Co-Founder, Chief Technology Officer & Director, Datadog
Datadog: From a single product to a growing platform by Alexis Lê-Quôc, CTO
At The Family, we believe that anyone can become a great entrepreneur. Find more info here: http://www.thefamily.co/
By Alexis ( / alq , CTO at Datadog (https://www.datadoghq.com)
You can find the slides here: https://www.slideshare.net/_TheFamily...
Learning & adapting as your startup grows is fundamental. After 8 years and $148M raised, what has Datadog’s CTO Alexis Lê-Quôc learned? 📚
Datadog is a monitoring and analytics platform for teams to boost their web performances. With 300 employees, they monitor +100,000 servers today in 42 countries all around the globe.
Alexis is their CTO,...
About Alexis Le-quoc
In a 2019 talk, Alexis Lê-Quôc, co-founder and CTO of Datadog, discussed the company's evolution from a single product to a platform. He described the company's focus on providing real-time visibility into application and infrastructure behavior for engineers. Lê-Quôc noted that the company learned that offering free trials, or a "try before you buy" approach, was effective even for enterprise software, as it allowed customers to engage with the product with minimal friction. He also stated that the industry's shift toward containerization and serverless architectures, along with global cloud adoption, has increased the complexity of monitoring and observability. In a 2014 presentation, Lê-Quôc detailed Datadog's real-time data analytics architecture. He explained that "real time" in this context refers to a 10–30 second latency between metric collection and display, and that the system's architecture must account for the asymmetry between high inbound data volume and lower outbound query volume. Lê-Quôc noted that the company uses durable queues at intake to avoid backpressure and that, under high load, the system may trade precision for latency by dropping data points, as trends survive sampling. He also remarked that the company had not fully automated anomaly detection across all metrics because blanket detection produced many false positives.
Transcript (48 segments)
A
Alexis Le-quoc0:10
Thank you, thank you very much. Okay, cool. So I'm going to talk about... this is 'Teaching an Old Dog New Tricks.' In this case, it's teaching Datadog new tricks. I want to talk about our journey since 2010 on the road to learn, to discover our customers, discover our products, understand our ecosystem, and eventually really understand ourselves. So, there are a few people who know about Datadog. Any users in the room? Alright, thank you. Thanks for your business. For those who don't know what we do, we are in monitoring and analytics of application infrastructure. It's like engineers, you know, customers with customers like Airbnb and Twitter and so on. It helps them understand what's going on with the app from inside, in the backend mostly. We now have... I can't disclose the exact number, but it's on the public website. We say thousands of customers, which is true. And we have really tiny customers, literally two people in a garage, and we have really large customers, you know, large Fortune 10, Fortune 5, really large enterprises. So that's been... obviously we didn't start like that, so that's kind of an interesting thing to share. And really, we try to build a platform. After years, I think we tried to build something that is a platform now. I'll spend some time discussing what it is.
Okay, so nine years ago, we started officially in 2010. Nine years ago, there were actually fewer people on that, and most of them are still selling company. We were a handful of people. We didn't know anything about enterprise customers. So we are technically an enterprise software company. Enterprise software, you're thinking on a sell to enterprise customers, and we really didn't have a clue. We didn't have any customers. We tried not to walk there. We didn't really have any money. I mean, basically using credit cards to... and I'll get into a bit more specifics. So I want to talk about primarily four things over these nine years. I want to talk about customers, because I think this is something that we, as a tech company, as techies, we know how to write code and so on, and quickly we discovered that we needed to understand those people known as customers, otherwise there would be no business behind. So there was a lot of learning, and that learning continues. I want to talk about the ecosystem also, what we learned about, because you're a company but you're not in a vacuum. So sure, you have customers, but there's partners...
So let's start. The journey starts in 2010 to 2013. I won't necessarily get too into a lot of details because it's... we're now very good. I'll say that as old entrepreneurs experience, you go through highs and lows, and you never, at least we never got a very clear signal that that's it, you know, we nailed it, now we just have to execute and everything is going to be there. When we started in the early days, even to some extent today...
In 2010, I think we went to Mountain View, we applied for Y Combinator, and we didn't get in. It was an interesting experience. It stung and it didn't feel good when somebody tells you no. But we learned something there. We learned to focus on what we were going to sell to customers, what value, what problem we're going to solve for them. This is our webpage from 2013 roughly. At the bottom, nice graphics... 'Teams who write and run applications at scale and want to turn the massive amounts of data produced by their apps, tools, and services into actionable insights.' Okay, does anybody understand what this means? Has anybody ever woken up in the morning and said, 'Yes, I want to turn massive amounts of data into actionable insights'? Not really. So in reality, we were trying to hit all the points, but it didn't convey what we really were, because we didn't know. Part of the journey is ultimately to understand who you are as a company.
There were two things that are missing. Let's see... no laser. We had a lot of small players, a few large players, legacy like HP, IBM, CA, BMC. If most of the names mean anything to you, don't worry, they don't after. Real active enterprise software that's sold for millions a year and never works, everybody's pissed about them. That's the old-school IT. There was a lot of mom-and-pop small shops doing monitoring at a small scale, and there was open source. So we thought, 'Oh my god, if we're going to enter this market, it's going to be a slog, it's going to be tough. Incumbents are really big, serving a market we cannot address the enterprise.'
While we start on the cloud, there's no 'I' in the description I read before. This is where 'cloud' because in 2013, cloud wasn't real. So we thought it could run stuff, we could run our stuff, but we thought if enterprises read cloud and a product that thought it was a toy at that point, they thought it was a toy. So we didn't want to be associated with it. So that's how you end up with this mumbo jumbo that people hear, 'Yeah, I've heard Datadog, I want to try it,' but not sure what it does. So who we really were, it took us two and a half years to understand. Effectively, it took us two and a half years to write this sentence. 'Datadog is a service...' The value we delivered to them is we let them understand what's going on and not be alerted by the customers that their stuff is down, but we would tell them, 'Hey, your stuff is down.' And by 'we,' I mean the tool we had built. Getting to that realization and eventually writing it down was a really important journey. That's maybe when something clicked. The way it manifested itself is when you're effectively building stuff that wakes other people up. When we built that, people would come back to the product. Before we had that, people would say, 'Yeah...'
You who use us, we're part in a small way of your daily life. It's not necessarily the best moments, but we're part nonetheless. So then we said, 'Okay, cool. So we're SaaS and we do monitoring and we understand cloud really well.' That's another way to think about it. So three years to get there from 0 to 10 customers, small, month-to-month. A lot of traveling, a lot of events, a lot of demos, going from city to city mostly in the US, a little bit in Europe, schlepping the displays and computers and connecting everything and saying, 'Oh, well, let me tell you about that.' That was the early years.
There's this notion of observability: being able to see and understand what's happening inside a computer stack. If you've ever been into a data center, any application looks like a bunch of blinking lights. That's an app because it's a bunch of servers connected, they just blink, that's all they do. They heat up the space and they blink. As a person, you can't do anything with just blinking lights and heat. You need a tool to tell you, 'Okay, your app is up or down, it's slow here, your CPUs are about to burst into flames, you're running out of disk space,' and all that stuff. You need some software to tell you that, and that's us.
Our product, which internally we called Metrics, measures anything: CPU, memory, number of people clicking on a site, and so forth. We let people measure anything. It could be temperature. We have people doing random stuff with solar panels and so on, but that's a different story. I'm going to fast forward to 2014. I want to talk about three different categories which I alluded to earlier: customers, ecosystem, and ourselves, and the kind of changes that are happening. So in 2014, we went to AWS re:Invent. Prior to that, we were in events...
The entire industry. So in 2014, remember I told you that we didn't want to use 'cloud' in our description initially because we thought cloud was not real, it was a toy. Interestingly, in 2014, we see enterprise users show up at re:Invent, which means they're curious, which means there's something. Maybe they won't use it right away, but there's this notion that maybe that's what the future is. They want to understand. We had our first booth, really small. Among those enterprise customers, we don't see any banks, and that's kind of okay. Maybe the large manufacturers are showing up. I remember meeting with...
They have significant firepower in enterprise. So that's after the show, that's what we learned in terms of ecosystem. Interestingly enough, that's when Docker hit 1.0. In that year, in 2014, I gave a session at re:Invent on monitoring Docker containers. It was immediately sold out. It's not like I had something very interesting or groundbreaking to talk about. It was like, 'Okay, well, containers are this, this is how you monitor containers,' very basic. There was a line to get in. I'm not a physically good speaker, so it's not like I do magic tricks. But there was a line to get in.
Integrating in a customer's environment was really important, in particular when it comes to APM. APM is application performance monitoring, looking at the health of an application in a very particular way. That's something that resonated with customers. The learning we had is: first of all, we need to understand Docker. We need to understand Docker because we need to understand about it. In 2014, it wasn't clear whether it was real, it was just a buzzword. It could be a fad that comes and goes, but we had to make the bet. So I said, 'Okay, well, let's invest and spend time.'
We need to integrate really well with cloud, because enterprise users start to show up. The thinking was: if they show up, they're interested, which means they're going to try it. If they're going to try it, we are well positioned because we were born in the cloud to help them understand what's going on in the cloud. So we should continue to invest there, and we've never stopped since. It's been material for us. We should explore APM. APM was this particular way of looking at application health. That's not what we were doing at that time, but we saw that a lot of our customers...
Because that's what else is not... they, as in hypothetical day, but your user in the company, what else are they using? What other problem are they solving? And then of those connected things, one makes sense. You could say, 'My users are using Slack.' Okay, well, so are a lot of people. For us, would it make sense to think about building a chat product? Probably not. We're not in that communication collaboration space. In our case, we saw APM. New Relic is one of the established players in the space.
They think they need us and them to get answers. That gave us some idea. Another thing we learned, and this is more a go-to-market thing, is enterprises and the way they were adopting cloud is they were sending engineers. Not IBM, because that's kind of their play, but take Nestle as an example. They would send engineers, like software engineers, to AWS re:Invent. This is a different way to think about how large companies buy technology. There are two ways: either an engineer goes and tries it out and says, 'Okay, that's cool, we should use that.'
So we're going to meet the top of the food chain and convince them that they should buy your stuff at scale. Or, and that was relatively new, you have the bottom-up approach, which is you've got to convince users one by one, even inside large enterprises, that they should adopt your stuff. That's the thing that didn't exist before. Open source opened the way there, where there was a bit of a guerrilla adoption. Then cloud interestingly followed the same path. AWS, because the people at the top considered cloud a toy. It didn't start as a five billion dollar contract. It got started, and then the developers said, 'Wow, that's cool. I don't have to wait for six months or twelve months to get a piece of metal to run myself. I can get it right away. Let's do more of that.' They got to a point where companies realized, 'Oh, you know what's going on? We have all these credit card charges for AWS. What's this?' Interesting. And that's how they got hooked. So we followed somewhat the same path. Even to this day, you can try our product for free without talking to anybody. Literally, you don't have to enter... well, yes, you have to enter your name and email just so you can log in. We don't ask you, nobody's going to call you right away. They wait a little bit.
Keep building it, and it's our workhorse and so on. But we only have one. That's where 2015 comes in. In 2015, from a customer standpoint, there's an interesting thing that happens. There's a bank, a retail bank in the US called Capital One. They don't have a big investment bank, so pick a retail bank. They announced at re:Invent, 'Yes, we are all in the cloud.' This is like, 'Wow, this is a significant shift in the mindset.' Because now the banks, which in the eighties, nineties, early 2000s were very conservative...
Industries heavily regulated. So it essentially has the effect of telling other people in the industry that cloud is okay. Because if you're a large enterprise CIO, you don't want to be the first to say, 'Yes, we'll go to the cloud,' and it fails and you get fired. It's a lot easier if somebody you can point to and say, 'Look, Capital One does it, so we should do it.' It validates the choice you're making. For us, that means it's now serious business. People do not equate cloud to just...
Something called Kubernetes. It's here, which sits on top of Docker, it orchestrates Docker containers. It reaches 1.0. It's really the orchestrator. It's what makes or breaks your containerized environment. There are so many containers all over the place when you containerize your application. There's so much chaos that unless you have a piece of code that keeps it up and running and schedules, starts, stops, then you can't operate. At that point, we see that come up. Kubernetes comes out of Google, so they have good credit for it. It's not there...
We need to understand this piece because we're about monitoring. Monitoring is about observing, so you need to understand what you're observing. In this case, if a customer has a Kubernetes environment and you plug in their data, we need to tell you something interesting out of it, otherwise we're not doing our job. The other piece that was interesting is Lambda is announced. It's actually small, it's not at re:Invent, it's small. Lambda lets you execute a piece of code without any servers, or at least you don't have access to servers. It's like a blog post at the end of the year, after Thanksgiving. But it's interesting because it's a new way to think about running applications. For ourselves, we discovered what we decided is like...
That's better for us. I mean, it's bad when you're like that, so be it. So we will learn enterprise cloud adoption bottom-up. I talked about that. The term of art is 'land and expand.' You put your foot in the door because you convince a group of people in a large company to adopt you, and then you try to sell around. They talk to you, you help them evangelize their use inside the company. You provide them training, you make them happy, you make them the hero of the day thanks to us. But we don't want to be thanked, they have to be...
The environment is getting more complex with all this stuff that comes up: Kubernetes, Lambda, Docker, and so on. It's hard to keep up. Actually, everybody wants to learn, but it's a lot easier if we can give people some introduction to new technologies. It'll be good for us because they'll create repeat visits to our blog, for instance. It'll be good for everyone because they'll learn, and then you provide pointers. DigitalOcean, I think, is the one that really popularized that: create a lot of content as a lead strategy, good quality content, not 'Hey, buy Datadog,' but 'Hey, here's how you do this.'
It's a strategy to gain leads. In other words, containers add complexity. It's an evolution of technology. It's a technological evolution that's changing the way we think of applications, and it happened really fast. 2014, Docker 1.0. 2015, Kubernetes 1.0. People started to invest a lot of mindset and time thinking about it. So we thought we've got to invest time, we've got to invest people's time there and develop a lot of integrations. But also, when you think about...
Absolute middleware database never changes. Every year you add machines because that's how fast you can add machines. Now we're talking about containers that come and go every minute. That means the sheer complexity of things changing in an environment starts to exceed what a person can understand. There are too many of them and they change too fast. What that taught us is we're going to have to invest in machine learning. Machine learning can mean a ton of things. The way we think about it is how do we reduce the complexity of that environment? How do we tell you the important things, the patterns, the things that keep happening, or the things that never happened and now are starting to happen?
With that complexity, we're not just going to reproduce the complex environment in front of your eyes with a bunch of things on the screen. We need to cut through that, reduce, reduce, reduce, simplify, simplify, simplify. Otherwise, you're going to get lost, and then we're not doing our job. The other thing we learned, and we learned by doing, is creating this APM product from scratch is really hard. When you start up, you create a product for your product-centric startup. You create something, it's hard, but it's not the same hard. It's really hard because when you start, you have nothing and you're creating something out of nothing, and you don't know who you are, who your customers are, what your product is. When you start to have some momentum, creating a second product is really hard because not only do you have...
The existing product. Even to this day, as years pass, it's harder and harder. It's why I think large companies just don't innovate. It's too hard. The inertia, the weight of the existing, is such that it's really difficult for people to keep innovating. As part of our modus operandi, our way of being, we said we'll try to create a product every year, no more or less. We've got to do it, otherwise we'll just revert to inertia. Inertia will bring us back to just keep doing what we know how to do, and that's it. So at that point, we have the second pillar of our suite.
A group of products that are in the same screen is not a platform. A platform is when the products are so intertwined and interwoven that when you use both, you feel like it's more valuable than if you use one or the other separately. It's really difficult to achieve. When we built those two, for instance, one of the things we struggled with is how do you go from one screen to the other? What path? How do you connect things? If you're able to create these bridges...
Billion times bigger and bigger deals. We signed bigger deals. At least in the US, I think Europe and Asia were trying, could see it picking up. The US things started to really ramp up. We also could see AWS was not the only game in town. Now Microsoft with Azure really surfaced, Google with Google Cloud, which was aware that there was something they should do or just how to do it, or they built the technology, just how to sell it. Nonetheless, they were trying. For customers, what that meant was...
In 2014, the CEO of one of the largest US banks said, 'No cloud, it's over my dead body we go to the cloud,' because they were too scared of data leaks and stuff like that. In 2016, I think when that guy says, 'You know what, we've got to go to the cloud.' That's a very powerful moment because it means a lot of people are watching and they're going to wonder, 'Why is he saying they should go to the cloud? What did he and his team see in the cloud that we're not seeing?' It gets a lot of people thinking, 'Maybe we should go,' and then they rationalize the decision. In the ecosystem, Kubernetes...
The right horse. In truth, we also bet on the other horses, they lost, and we just pulled back and said, 'Okay, the team that was working on there was another one called Mesos, they said just don't work on it, just leave it as is, and we'll patch and support Mesos once in a while, but we're not going to actively support it because it's not going to be actively developed for a while.' In 2016, serverless was born. I don't know how many people know about serverless. I've heard several here. Again, this notion that you don't need machines to run code. It's not true, you do need machines, but that's all delegated to the provider. Interestingly, Google in 2016...
Large-scale companies built on that, but somehow in 2016, serverless catches on. I have no idea why. I don't know why in 2016 it resonated and why not in 2010. But you have to be on the lookout. You have to try to understand what's happening around you. For ourselves, we have to start to build that APM product on top of the metrics product. We always think, 'Okay, what's next? We've got to keep building, but what's next?' We thought about logs. Logs are everywhere. Applications log a lot of stuff to files. Everybody has the same observation. Now we sell...
Questions they are asking themselves. What answers do they need? Can we provide these answers? If so, then it's more of a product discussion: can we provide it, at what price, and so on. But this is how we think about expanding. The importance of the platform is that our logs offering has to mean more when used with the others than using just three things next to each other. Enterprise 2016, I think we always want to start our...
Steak. But I know each time that I've gone on this trip with our sales people, they have meat steak. So it's really... and you know, maybe you'll correct me and say that you don't understand, it's price else. But I'm telling you, this is the way we do it. We give people a list of accounts, 'Okay, you've got to get us business in these big accounts.' So you have to go there. I would go or talk to people and do meetings, understand the environment, and close the sale. Now it helps, of course. What's really helpful is when you have a bottom-up or land and expand...
The evil engineers adopt the product. They have small adoption, and enterprise sales can come in and say, 'Hey, we know you use us. How about we think about rationalizing a real good option across the enterprise?' That's one approach. Those are called 'calling and discovering new accounts.' Then we sell inside globally, which means inside sales that we've done so far, which really means somebody comes inbound, largely somebody comes to our website, somehow discovers us, they try, and during the trial we pick up the phone and say, 'Hey, Mr. or Ms. Prospect, do you want to become a customer?' That's the inside motion.
It's a global phenomenon. We have an option around the world, and we have customers around the world. At some point, we used to have sales out of Boston in North America that would sell around the world. The problem with that is if you want to pick up the phone and an Australian prospect tries your product and you want to reach them, you're going to stay up real late because it's going to be the middle of the night. So that's limiting. So we always try to sell globally, which means to install inside sales teams around the world, starting in Europe and then maybe some in Asia. Serverless is new, Docker, which means it's a better...
APM. What we learned when we added the second product is integration was really an important part of it. This is somewhat specific to what the product does and how it's usually deployed, but it was a friction factor for us, so we had to work out a lot of that. We also learned that everybody uses logs, so everyone needs logs. Everyone uses something, and you're close to it. What you offer is not very different. That's something you clearly think, 'Hey, maybe we should consider offering it because why not get the revenue instead of someone else?' So we still have two products, and we're effectively working on logs.
In 2017, we see large global companies migrate to the cloud. Which means we'll see, and this is maybe what's driven to some extent global cloud adoption, is global companies adopted it. But it means that now we're going to find customers outside of the US. We were very US-centric initially. By the way, we were founded in New York. I've lived there for 20 years. But that's promising because that means there's a market outside of the US. Luckily, the world is not only the US. In terms of ecosystem, there's still...
We know where the community is going to win. And about ourselves, we start to see: look, if we start offering logs and APM and metrics, and you can see everything you need to see in one application, is this what people are looking for? So you ask people, 'Does this vision resonate? Do you want it?' What we discovered very early on is, particularly if you're not trying to sell, if you're just trying to genuinely ask a question, 'What do you think of this idea?' then people are willing to engage. They told us, 'Yeah, look, we have 50 tools to do the stuff you're doing because of the legacy.'
I want to consolidate. So that's both a threat because there are 50 tools and behind these 50 tools there's a company that's fighting for its life. But it also means there's pain there. Somebody's telling you, 'I'm tired of this, it's painful, I want it over with.' That really has driven our platform approach, which is to deliver something such that the sum of the parts is greater. The value of the whole is greater than the sum of the parts. That's what I consider a platform to be: something where when you use everything...
It was predictable, but nonetheless, the learning there wasn't completely obvious. It's obvious in hindsight, but not when you face it. APM became generally available, reached a certain level of maturity. So now we really understood which of all the technologies in APM we need to support, which ones made the most sense: Java and .NET. Log management. We looked when we thought about starting a log management product. We looked around, who's doing what, what's available. While doing that, we ended up looking at a lot of companies in the space, and we discovered that's a tricky problem.
Promising, and one of them was the Paris-based company called Logmatic, which we bought that year. Instead of saying we're going to build it ourselves, we thought, 'Can we buy a company and start with what they have, like jump start?' Because as years pass, as the company gets bigger, it's harder and harder for us to do strong products. So that's what we did. To this day, we sometimes acquire companies for their team, product, and know-how. Sometimes we build it ourselves. All options are on the table. Finally, one note is that we decided...
Things can be like... it's a good experience. If you're in our position, we are lucky enough that we dogfood our own stuff. If you can use it, it's great. So at that point, we have three pillars. We can do logs, we can ingest traces, we can process metrics. 2018, last year, things start to solidify. The technology stack is now starting to solidify around cloud, containers, and serverless. It's how people think about their future applications. 'Okay, cloud is not really else, no rhetoric. You've got it. It's containerized. I'm not going to...'
We need to support that stuff really well. Last year, I think public clouds, that's it. There is no question that it's a toy or fad or something like that. It's now everybody has a cloud strategy as part of digital transformation. It's a buzzword, but when you say, 'Hey, I'm not going to go to the cloud,' people look at you and say, 'Why?' In five years, it's turned completely around. That's a very powerful signal. In the ecosystem, Kubernetes has won the orchestrator war, and serverless is here to stay. I think this is...
You can write your application logic in small pieces and load that up, and you don't need to run servers anymore. In some cases, in a lot of cases, it's not true, it can't work. But there's this idea that we can get there. It'll take a few years, but the industry may get there. It's not just some random experiment. We talked at length about a platform. We really spent time on that. That's something we invested a lot of time in last year and continue now. An interesting piece that we learned...
I try and I'm frustrated because there are so many sites where you go to this site, so many companies that built it. If you do it right now, I'm telling you, it's not a good idea. But you can go back and change it. You go to their website, you try to understand what they do. There's a high-level vision statement which may or may not resonate. Ours didn't resonate back then. Then there's a little cute animation of people running or things moving, and then that's it. Then there's a 'Try' button. You click, and then, 'Oh, I've got to fill out the form and somebody will call me or send me a PDF.' That was okay in 2005.
A screenshot of the real product is a huge improvement because you can look at it and say, 'Okay, now I get it. I kind of get what it's trying to do.' Ready to try before you buy is essential, with minimum friction. That works for enterprise, for large enterprises. That's fundamental in the way people buy technology now. They are not going to go into a long cycle before you can test it. They are not going to tolerate long implementation cycles. They want it now, they want it all. That also applies through 2017. The reason for that is when you get your prospect's attention, you get thirty...
It costs... okay, fine. Now let me try it. They enter their name, login, some instruction landing page, and then we have five, fifteen, thirty minutes of their attention. That's it. If we can't deliver something there and then, it's not because they don't like what we do, but just because they have a million other things to do in their day. I get bombarded by things to try. That's really even in the cases where you could think that it's going to take a lot of handshakes and meetings to demo, that's great, you need that still. But if you don't have this easy try-before-you-buy...
We have a fourth pillar, and basically this year we'll add a fifth one. So we'll have to rearrange the slide, but that's our path. What hasn't changed in nine years? I tell the team, 'We build the problem.' I don't really build any these days, but you build a product. We sell a service. We're a subscription service. The product is there to serve a need, serve a person, and serve a need of a person. So we still need to focus on the customer, and we have to focus on time to value. Time to value is how fast, how much effort, and how much time does it take for a customer to get something valuable out of the product when they first start to discover us.