Databricks CEO on Unifying Data & AI Agents

Yeah, that's a great question. And thanks for having me. Excited to be here. So, you know, we're just seeing acceleration in all of our business. Like, I really mean like we see acceleration in our core business. And what are we? Why are we seeing acceleration in the business? We slice it by Europe, by United States, Latin America, Asia, everything is. And we slice it by different clouds, no matter how we look at it. It's just, you know, the business is growing. So that requires you to invest, you know, to keep up. So we're investing in the core business. So that's the data platform, the security and all of that. So that's the huge investments going in there. On the AI side, I think that the biggest, and maybe we'll get into it later, is that, you know, I think the biggest opportunities, I think guys here. So I think we don't need more intelligence. We need context for the AI. The AIs without context are useless. So that's our Genie product. So, you know, there's a big research team that just does research on reinforcement learning and advanced AI techniques. My researchers, we're hiring hundreds and they're very expensive, you know, so it's almost like thousands. It feels like it, like other employees that you would otherwise hire. And then, you know, then GPUs. You know, so there's a lot of investment goes into that. So we're very excited about that. Happy to talk about that. That's the thing I'm most excited about. And then we're getting into new markets because what's happening basically is that the thing that people call SaaS apocalypse. You know, people interpret that in extremes. But really what's happening is that the barriers to entry for other people to come in and compete in any business in software is just lowered. So we're getting into new markets as well.

Yeah, I mean, we launched Lake Watch which brings us into the security market. And the security market is itself going through a big revolution because what's happening is that, I mean, we all heard it, but Mythos. But actually it started before Mythos. You know, it's already the case that the number one attacker, according to Hacker One's top list, is, you know, this person called Expo. But if you look behind it, it's actually all automated agents, and it's been dominating charts even before Mythos. They found, you know, they used OpenAI's 003 model to hack the kernel of Linux. So like this has been going on. So there's an swath of agents coming attacking these businesses. So cyber is a big risk. And the existing businesses that do security protection, they're sort of in this legacy model where you have to ingest certain data. It's very expensive. They're not going to be able to keep up with agents. Right. So, you know, we got into that market now, maybe we would have gotten into that market anyway. But with AI, it's compressing almost the time it takes for us to build. It's making it a lot easier for you to come in and try to disrupt.

Yeah. For multiple reasons. Right. Writing that software is ten times faster now thanks to AI. So we can just produce the competing software very quickly. We could not have done that five years ago if it wasn't for AI. So that's like one. But the second is that the AI threats themselves have changed the landscape. So the way the incumbents were actually approaching security, you need to change those assumptions. So that also always opens up an entry point for existing. And then thirdly, it's not, you know, specific to this time we don't have any security business. So we don't have any innovator's dilemma. We don't have to protect some revenue. We can just, you know, launch something and we can, you know, it's sort of your margin is my opportunity as well. It's used to say.

Yeah. No, we've stepped up a whole team. You know, a whole team.

I mean, unfortunately, I think it's going to be in the hundreds. But, you know, the way you want to operate in this area is actually very small team that can move very fast. It's actually preferable. So actually it's a, you know, so if it was 1000 it would be a terrible answer. That would be we would go nowhere. You know, I think it's going to get to the hundreds. But you want to keep it small and fast and nimble and execute quickly, because in this AI era, big teams slow you down because it requires coordination between lots of people. And that then becomes the actually, you know, slowest or lowest common denominator for progress. You want to unleash the agents and let them go fast.

Buy our new security product, like Watch. Just kidding, just kidding, just kidding. But actually, only half kidding. So, you know, these things are merging, right? So we were a data and AI company, we're focused on that, and we were not doing security before. And people would tell us like, oh, make sure that no one breaks into the data and make sure that the data in the AI is. But now people are saying, hey, how do we know that the agents are also not doing something weird? So actually whole markets are collapsing. So us getting into security is not just happenstance. That market is kind of collapsing with the data and AI market that we used to be in itself. So all those things that they say is absolutely true. So, you know, you unleash agents. Well, should agents act on behalf of you? How do you know that they're not going to do something that they're not supposed to do? So, you know, I think that ultimately, this is why I think actual humans are going to continue to be in the loop for a very long time. You know, a lot of people talk about like, you know, AI disruption is going to, you know, displace jobs, 20% unemployment. So I actually think it's not at all the case. I think that we're going to, we need to be firmly in the loop because many of accountability to your point. So we're going to ask, you know, if there's a, you go to the doctor and they say that, oh, you know, we have to amputate your arm. You want to know who? Like, if that was a mistake. Who approved it? Oh, some agents, you know. Sorry. It's like, oh, well, I lost my arm. Like, that's not cool. So, you know, so I do think that there needs to be like, okay, well, it was actually signed by this doctor. Did they look through the result? Do they looked at the traces? Do they understand what was going on? Why did they sign off on this? Same thing with our kids. We send them to school. We don't want, like, oh, you know, they were trained in the wrong stuff. And now your kid just went crazy because some agent thought it that. Okay, I'm okay with that. No, we don't know. Who was the teacher. Who was the person in the room? Why didn't I oversee this? Why didn't they? So I think we can have humans in the loop, and we need to have this. And it comes also oftentimes down to the legal issue of, you know, we took the legal risk here. So this is going to continue. One way to think about this is just to look at the past. You know, we could solve self-driving kind of airplanes a long time ago. You know. But if I told you all that, I'm going to give you, like, a huge discount, 90% discount. And, you know, you can go with all these airline. We just have, you know, one slightly drunk pilot that only has like 50 hours of, you know, flying experience, but it's really cheap. None of you will get on it. You would say, oh, that's terrifying. But, you know, we already have these autopilots in the planes. So why do we still want humans in the plane? Because we still want someone to sit there, and we don't want to just delegate everything to the, you know, to the sort of box, to the box. So yeah, I think that and that's one of the focus areas with our security product. Yeah. And I don't think it's solved by the way if you. I was joking when I said, just buy our product and we'll solve itself. This is going to be a journey we go through together. As humanity, as AI permeates society. Just like when internet came and there was all these, you know, Nigeria letters and fraud and all of those kind of things, we learned how to deal with it and we figured out how to handle payments. You know, I was a kid in college and they were saying, no one's going to ever swipe a credit card online because, you know, that could just disappear. And they could. Now we only shop things, so we figure out techniques to deal with it. We'll have to go through that together.

Yeah. You know, my financing would kill me if I do that a lot. You know, this isn't a, there's no quiet period when you're a private company and you have no plans to IPO, as you told my colleagues, this is what I keep telling my financing. But, you know, they still give me a hard time after these meetings. So, let's just say that's significant acceleration that we never would have thought we would have acceleration from the 40, 65% Y-o-Y increase. So there was, it was faster than 65% in Q1 over the year earlier.

And I can tell you that, uh.

Yeah, yeah. And also, I can say this, it's also when you look at by different product lines or by different regions or, you know, we have many of the AI companies, our customers, like OpenAI have shared that they're, you know, a customer, Databricks, they use Databricks, you know, 800 million users to make sure that their, you know, nothing crazy is going on. You know, they use us for that. So it's back to the kind of safety thing that we discussed. But, you know, it's okay. Well, what if we remove some of it? Maybe it's just a few big AI companies that are using us. Anthropic, you know, use Databricks. If we remove those. Still the same thing. Acceleration in the whole business. So why is that? Why is the business accelerating so much? I think one of the main reasons, and this is maybe one thing that people are not sort of thinking through, which is the agents are now starting to behave. You know, they're starting to actually act throughout the ecosystem. So a month and a half ago, I shared that we've now seen more queries come to Databricks by agents than we see by humans. So it's like this tipping point where, you know, it was 100% humans, I would say three years ago. And now we already have over 50%, databases launched on Databricks. Yeah, it's over 81% now. Agents and these agents are just way faster. So they just want lots of stuff where consumption based pricing model. You know, our revenue goes up when that happens. So that's one of the main drivers behind this.

Well, I mean, you know, without picking, they're all partners and customers of ours, so I have to be careful. You keep getting me into trouble here. But, you know, I would just say this that the biggest shocker for me is that it used to be that, you know, GPT-4 comes out, it could dominate for six months or so on. And then, you know, just not long ago, the saying was that, you know, frontier models only last three months, a quarter. So even if you have the world's best AI model, it'll only last, you know, a quarter. Okay, that's kind of surprising. So on the most shocking thing now is that that number is not one month. Like you get the idea for one month. Like, you know, maybe I should say like when Gemini came out in November last year from Google, it was like everybody was blown away. Everybody switched to Gemini, you know? But right now, people are not talking about Gemini, right? Who are they talking about? Well, I mean, like, you know, people are, okay, you're very good at your job. Look, people are very excited about Codex, but also very excited about Claude, you know, and I think Gemini 3.5 when it comes out is going to be very excited about that too. Here's the. So number one was this shrinking time of how long a model, in some sense this question even won't matter soon. You know what was the hottest model yesterday? What is it today? Well who cares? Tomorrow's a new day. So. But the fact that this shrunk was interesting. The second thing that's really been a big surprise for me is that, it used to be that the developers are changing models because they want the latest model. The model just speak English or any other natural language, so it's very easy to switch between them. You don't need to worry about, you know, oh, I'm changing everything. No, it's just, you know, it's a probabilistic thing every time. The answer is different anyway, so you might as well just pick the cheaper, smarter thing that came out just yesterday. But this was developers. Now we're seeing that corporations, IT departments, leadership is freaking out about the costs. You know. And they're saying this is insane because, you know, when you see these companies, you know, got $30 billion of revenue just in one quarter out of nowhere. Well, someone is paying for that. So on the other side, the freaking out that, you know, how do we put cost controls in place, right. How do I, so one thing that has a new trend that has emerged. So we put this thing out. It's called the Unity Gateway, which lets you basically, it's a kind of a, we did this for security reasons goes back to our security, which is what you need to control, which AI your company is using. Yeah. That was our purpose. Like, you know, to make sure they're not using some model. You don't want to track the data. Did they swipe a credit card? But what the main thing people are using it for is actually, you know, can I put a budget on it? And the thing they're doing is they're using more and more open source models, right. And what they're doing is they're using this new paradigm that's called the advisor model. So you use a small, not so intelligent, very cheap, very fast open source model. And then you give it a tool because these AIs are very good at using tools. You know, they can go type stuff and you're going to word processor. The tool you give them is access to a smart big model with the big brain and say, hey, when you get stuck, use the smarter model. And this way, you know, people were on this increase of exponential increase in costs and they're able to completely flatten it even though usage is increasing. You know, cost is flattening to the extent that cost is an issue.

People that want to talk about it. Okay. But yes, absolutely. You know, it's Chinese models, open source models are absolutely dominating. You know, and when people have the choice to use them. Yeah. It's like, hey, would you want to pay $30 million or $1 million? Well, you have to have used a Chinese model. Okay, fine. You know, it's like, you know, it might be like, oh, I want to use a Chinese model. And we have concerns and so on. It's like, well, it's going to cost 30 million versus 1 million. It's like, okay, I took that.

Well, I mean, open source is just this force, right? You can't fight it. You know, it's just and these are like, you put the weights out there and the whole world start using it. By the way it starts reappearing. So for instance, Cursor, put out the model Composer. Well, it became clear that they actually trained Composer on top of a Chinese model. So Composer would not have been able to be trained without, Kimi model that came out of China. So there's also this happening. So then some of our customers, like we do not want to use a Chinese model. We're okay using Composer. You know, I was like, okay, here you go. So. Yeah. So it's I do think that, you know, in the end of the day, it's kind of like, you know, money talks, you know? Sure.

Yeah. Look, I just think that there's, if you have, like, big dislocations happening and there's big things happening, you know, election year, there's, you know, there's macroeconomic indicators that, you know, could be great, could not be great. We don't know right now, you know, there is uncertainty, you know, with energy and other things going on in the world. And then there's this mega IPOs. And no one's ever raised that much money that any of any one of them, the amount that they're raising in the IPO has never been raised ever by any company before. So, you know, I think it's better to wait and see and, you know, and maybe I'm wrong, actually, and it'll be a great year, but I don't lose anything by waiting. You can go later and, you know, go at a time where, you know, you just have more stability. Like, as everyone knows, businesses prefer predictability and no big surprises. No surprises is good.

We might raise again. I think that, you know, there's going to be much more interest after, you know, these three gigantic private companies go public. You know, the way it works is that private capital, people who, you know, have to invest in private companies, they can't deploy in public stocks. Right. So, I think there's just going to be a much bigger supply in the private markets for, you know, companies like Databricks, Stripe, others. So, you know, I don't think there'll be a shortage of capital for us.