A very vibrant industry has arisen around a set of young companies that tell IT how their systems are doing.
But that very vibrant industry could face some very serious competition from a strategy for a more comprehensive tool.
Observe, Inc., based in the sleepy Silicon Valley town of San Mateo, California, has built an analytics program that can take all the data that goes into the separate programs of Datadog, Dynatrace, Splunk, and many other companies, and analyze it collectively.
It’s a strategy the company believes will enable it to leap over the established vendors.
“Over time, people will realize that they can buy one thing instead of three,” Jeremy Burton, chief executive of Observe, told ZDNet in a Zoom interview.
“The analogy is, people used to buy accounts receivable, accounts payable, and general ledger separately, and now they buy financials,” he said. “And it’s almost an intelligence test, why would you buy them separately, because of the interdependence.”
The Observe program can consume all the same data as those other programs, and it draws connections between the clusters of data that those other programs can’t because they don’t see all the data.
“At the foundational data level, we bring these different data types together,” explained Burton. “Because we bring the data together, we can relate data, and because we can relate data, you can quickly get additional context for the problem that you’re looking at.”
That is important because every company now is running on applications, some even use an app as the front door to their business, such as retailers. “If there’s a problem, it’s a big deal, and they need to resolve it really, really quickly,” Burton observed.
Imagine, for example, you have a problem with your e-commerce site. Normally, such an issue sets in motion a whole chain of applications.
With Observe, it’s one program.
“A problem with your e-commerce site normally generates a ZenDesk ticket, and if someone investigates that, you want them to go straight to the logs for the session,” said Burton. “And from there, maybe, show me the database transactions related to those logs, and show me the builds that have gone into production, and show me the developers that have checked in code for those builds that have gone into production.”
“So, immediately, I’ve got a lot of context, so I should be able to triage and find the issue more quickly. That’s what we’re trying to do.”
The company’s name derives from the concept of “observability,” coined in the 1960s by scientist-inventor Rudolph Kálmán. Kálmán’s insight, as explained in his landmark paper on control systems, was that if any engineered system can be controlled, then one can work backward from the output of such a system to find out what is causing it to produce its results. You could infer what was going on inside the black box.
Analogously, Observe takes a total-system view of what’s happening in the interaction of computers.
“Observability, the space we’re targeting, is a collapsing of log analytics, which is Splunk, Elastic, Sumo, and a bunch of others; metrics monitoring, which is Datadog, Wavefront, and a bunch of others; and APM [application performance monitoring], which is New Relic, AppDynamics, and a bunch of others,” explained Burton.
That’s the theory, but what has made possible that ambition is Snowflake, the cloud database company whose headquarters is a five-minute drive down El Camino Real from Observe.
Observe has garnered attention because it is one of a crop of new young companies building applications on top of Snowflake, which is itself a hot young company, one that some believe is an Oracle killer.
Snowflake is the data store that lets Observe tap into all the different data formats that each program such as Splunk usually has to collect in a separate repository.
It’s no accident that Burton is on Snowflake’s board of directors. He found his way to the job at Observe through having seen Snowflake early in its history, back in 2014.
Analytics vendors such as Datadog will try to expand their wares to capture more and more functionality, but Burton predicts such efforts will be difficult to engineer. “I feel like I’ve seen this movie several times,” he said. “The big company tries to package things together, and tries to cross-sell, but the technical integration often takes a decade.”
Burton spent many years in established enterprise tech firms. He was head of marketing for EMC and Dell and head of services marketing at Oracle in the 1990s. Observe is his first startup gig.
Burton’s former direct report at storage firm Veritas in the early Naughts, Michael Speiser, went on to be a managing director with venture capital firm Sutter Hill Ventures.
Speiser helped to bankroll Snowflake and he tried to convince Burton to run the company early on, but Burton was skeptical.
“They said, you’ve got to see this company, they’re building a database, and I thought, everything’s going Hadoop, Hadoop is going to rule the world,” recalled Burton, referring to the open-source file system that became a big part of Big Data in the last decade.
Nor was Burton blown away by Snowflake’s low-rent digs in San Mateo. “It was nine guys, in this pokey office, no air-conditioning.”
Ultimately, former Microsoft executive Bob Muglia was brought in as CEO of Snowflake, followed by former ServiceNow chief Frank Slootman, the current CEO.
Not long after he turned down the Snowflake job, Burton had an epiphany. He began to see how Snowflake was becoming a platform, on top of which other companies were building, including Observe.
“Mike [Speiser] had funded a few companies, Observe, Sigma [Computing], and Lacework, that were all building on top of Snowflake, and I thought, hmmm, this is interesting.”
What Burton put together was that Snowflake was making the same transition that Oracle had made in the 1990s, from just selling database licenses to being a platform on top of which Siebel and SAP and others built.
“I thought, Snowflake is clearly going to do well, but the second act is going to be the ecosystem,” he recalled. “These data platforms create a wake that the partners can draft off of.”
Burton found the idea of working with his former colleague, Speiser, appealing. “He is one of these solid guys, never has a bad day, he’s got this infectious enthusiasm, and I always wanted to do something with him,” said Burton.
More than anything, though, what convinced Burton to come to Observe was the ambition of the founders. “The vision they had was, we want to ingest everything, all the unstructured data that machines generate, all of it, we want to process that and make sense of it,” he recalled.
“I thought, well, if I’m going to go do a startup, I’ve got no interest in working at something which is a feature of another company that gets sold after five years for $100 million bucks; if you’re going to put your heart and soul into something, let’s shoot for something big.”
Fast forward, Snowflake is has become an enormous platform on which Observe can build its tool. It is now an $83 billion public company following its IPO on September 16th. Based on his many years at Oracle and EMC, Burton has realized the true essential advantage of Snowflake.
“It’s a very simple thing,” he said. “They separate storage and compute.” The data can live in whatever cloud storage system a client wants, such as Amazon’s S3 service.
“You can dump data into Amazon S3, terabytes and terabytes, and then, because the architecture is massively parallel, you can spin up 128 nodes in Amazon EC2, you can load the data in parallel from S3, and you only pay for a couple of hours of processing to compute it.”
“It’s so simple, you wonder why someone didn’t think of it before,” said Burton. Burton makes an analogy to how Oracle’s database became the more-efficient choice in the ’90s.
“With Oracle, it was row-level locking,” he recalled. “Because transaction processing was the thing, you could lock it at a granular level, and therefore you could get more users on it, and so it scaled better.”
As for Observe, the company has every one of the analytics vendors in its sites, the “plan for world domination,” Burton jokingly calls it.
The obvious objection is that trying to do everything could make a company a jack of all trades and master of none.
Burton is convinced the synthesis of data will overwhelm the value of the specialization that the existing vendors provide. “Our value is definitely going to be more the correlation and integration of the data versus the absolute best in any one area,” he said.
“So it may well be the case that a company like Splunk would do a better job purely on log analytics, or a company like Datadog purely on metrics monitoring.”
It comes down to economics, just like the example of finance apps, said Burton. Today, a company has to hire professionals to correlate data between the tools, professionals who are expensive and hard to find.
“The best job in America is writing software, the second-best is fixing it,” he observed.
“Our sense is if we could correlate the data using Observe’s software, your more-junior members of the team, even, could help troubleshoot these issues.”
Burton is aware that every one of those successful companies, including Splunk and Datadog and Dynatrace, are looking to expand their wares, to grow their fiefdom to include more data. They are starting from their niche and seeking to be the one analytics that Observe is fashioning. That move is in fact one of Wall Street’s favorite reasons for recommending shares of Dynatrace, say, on any given day.
It will be hard for the companies to work backward from their Balkanized view of things to the totality, Burton told ZDNet.
“I feel like I’ve seen this movie several times, the big company tries to package things together, and tries to cross-sell, but the technical integration often takes a decade.”
Of course, where the rubber hits the road is shipping, and version one of the Observe program hasn’t shipped yet. About 15 customers have been testing the product for a little over a year, with workloads really ramping up in the last six months, said Burton. The product is expected to see its first commercial release by year end, he said.
Observe has raised $38.75 million. Of that, a Series A round was $15 million, the balance of it being debt financing, which helped to reduce dilution. Using debt was Speiser’s brainstorm, said Burton.
“What he said is, I don’t really want to dilute the team until the product’s out,” Burton recalled Speiser advised. “We can push out the Series B to when the company’s more valuable, and when that debt converts to equity, you don’t get as diluted.”
“It’s a favor from Mike, more than anything,” Burton said with a laugh. “Most VCs would have just taken another chunk of the company and made off with it!”