AI Agents Are Now Your Database's Main User | Reynold Xin, Co-Founder at Databricks

Hey Richie, thanks for having me.

Yeah, I think we're actually getting closer than we ever have and maybe already here depending on how sort of widely adopted these things are. And as a matter of fact, with like at Databricks right now, it's just not just me, like me personally have completely changed the way I run analytics. It used to be the case that whenever I have a question like for example, hey why did a product adoption metric suddenly spike up or spike down, I would ask somebody who reports to me and they talk to somebody on the data team and then that person talks to somebody who reports to them and then before you know it, it would be like two weeks pass and then some answers come back. These days I mostly just open Genie and I ask, hey what's going on, and I would say like 95% maybe even like 99% of the time you can tell me what is happening. But I think in order to get to that point there was a lot of groundwork that were laid down both in terms of product development as well as from the adoption of building up the proper data model, the ontology and all that. And it's not just me that's been doing it, like I think even the physical security team or sort of the real estate team at Databricks are doing the same thing, like they're not technical people at all. So like I think to some extent the future is already here, it's just not evenly distributed.

Yeah, so we did a recent branding to kind of consolidate a bunch of different things kind of under the Genie umbrella. But I think the easiest way to think about it is Genie is an agent that really helps you with a lot of enterprise work. And how it differs from I would say a lot of other agents out there is a lot of other agents last year, you recall, they kind of try to memorize what exists. But Genie actually computes because I think vast majority of the enterprise questions are to aid some decision making and it's actually somewhat quantitative in nature. It needs to be grounded on data. So Genie is designed specifically to address that. You can do a lot of what general agents do also but it's where it really excels is asking questions like hey build me a revenue model that can forecast how my revenue will grow or hey how are my sales going across the different geo distributions, are there any special things I have to be paying attention to. And this kind of changed how we work internally, also kind of changed I would say a lot of how our customers work. And think of it kind of as I think it's basically what BI really should be. And instead of just going to dashboards and asking a team of armies building dashboards, Genie is the way to actually answer business intelligence.

Yeah, I think there are many pieces that to get to as I was saying there's actually a lot of legwork required to lay down a foundation to get to this point of self-service analytics and the humans are an extremely critical aspect of that. So we're not advocating hey all the data engineers, data scientists, analysts you're out of a job. As a matter of fact we think they are essential in creating the foundation and this foundation includes about a very important part of why Genie can work is this thing called ontology, Genie ontology. So what it does is try to actually index every computation ever happened in the past. So for example if your company cares a lot about revenue like every company does but revenue is a very difficult metric to actually compute. And it's certainly something you don't want to get wrong. At the same time, there's probably tens of thousands of different metrics that different teams and different departments care about in a company. You don't want to manually still create all of them. But largely, I would say anything that really matters, you have a lot of historic workloads like you probably have a lot of SQL queries, a lot of dashboards that kind of reference revenue one way or another. So the Genie ontology kind of goes and indexes all the past workloads and then trying to define hey here's how you compute the different metrics and here's what you mean by for example ARR because every company might define annual recurring revenue slightly differently. And it sort of pulls all of that together and creates a graph. I mean it also allows humans to actually augment the graph and by combining this sort of curation based augmentation, auto discovery of what exists out there we can actually really power the AI to do its best work because AI models now have very good reasoning capabilities, what they don't actually have is the context for everything. So Genie ontology creates that, provides a context to all the AI and with a context now, one it can actually do it faster because it doesn't have to sort of go hey let me crawl something and then see if there's a definition here, oh this links now through somewhere else let me go crawl the next thing. It's kind of like one of my co-founders Ali and CEO Databricks has this analogy of imagine you go back to 2000 and then the way search engines work is it opens this one website first. It searches the website to see if there's anything on there and if there's something that matches the link, it opens the link next and goes to visit that link and just keeps recursing and then after two hours it shows the 10 links on google.com search result. That's kind of how in a way state-of-the-art how many agents work. The ontology breaks that kind of in a way give you the search index, the knowledge index, knowledge graph of your entire enterprise. So it substantially improved the velocity and substantially improved the accuracy because you don't have to like go figure out okay what exactly is that revenue. So with all of this I think humans has a very important part. One is data modeling, the AI agent if you feed it junk, junk will come out. If you actually feed it really great data with the ontology you can get amazing results. The second one is there is this a little bit of a manual curation component and we have found actually many organizations are kind of religious about whether they should do any manual curation. Some are in the camp of absolutely no manual curation ever. Some are hey manual curation would be the most important thing could get us to the so whatever from 95% accuracy to 99% accuracy. We're not trying to force our customers one way or another. We kind of feel hey it's a very useful tool. You probably use it. You should use it internally at Databricks. We do. So obviously humans have a very important part in that manual curation maybe for tier zero. So metrics and assets. And the other thing is I think once you have your data set actually get to what we call a gold layer. So ready for business metrics. All the system work pretty well but there's a massive typically massive amount of data engineering is required to go from your raw data which might be collected from tens of thousands of different systems into the gold metric and any of this middle layer could break at any given point. We are actually working on many things to improve and sort of self-remediate if for example a pipeline could break but in practice I think you still need humans to actually babysit and think about hey what happens if something breaks, what's the implication of that, how do I actually fix it, how do I make it more reliable in the future. So we don't think humans would go away, we actually think humans will so more and more focus on the more important task and kind of be relieved from hey can you help me answer this question by writing a specific query or can you help me answer this question because I don't know where the data is. I think the humans should spend the time to build the ontology and infrastructure to the point that it becomes largely self-service for any business questions.

So the semantic layer is actually now part of ontology, it's part of ontology. Ontology actually builds on that. It's one of the signals that the ontology pulls from.

Yeah. So the we should talk a little bit about it right now. Which is ontology. But it is true the AI makes mistakes about so does humans right? How many times do you just trust blindly hey this person is going to come back with the right answer but it turned out so was wrong. That happens all the time too. So I think it's not necessarily the case that hey AI has to be completely perfect where humans like make mistakes all the time. But certainly I think if you just were to ask a coding agent, you just give the coding agent or a generic agent so access to all of your maybe estate by plumbing through APIs they're too much context that it will confuse the agent a lot and then it start hallucinating fairly frequently. So what we have found we actually have a whole AI research team whose focus isn't on hey how do I conquer and build models where AGI there are far smarter than human beings. The whole focus is on how do we actually create the system and the product that can give the best answers on kind of enterprise-like questions. And we even created a sort of standard benchmark OfficeQA which is now used by many of the frontier models also. They actually run eval against that specific benchmark when they announce their new models and those are basically grounded on actual questions. We see that enterprise work like large workers would be asking the so the ontology one is it finds all the context but more importantly than having all the context is actually it finds the right context for any specific question and with the right context that are hopefully actually small you can actually substantially improve accuracy of the model and substantially reduce the possibility of hallucination. Now nothing is perfect just like humans. I don't think we'll ever have 100% correctness rate and it would never be wrong. So we're not saying you should be asking Genie a question and put it into your maybe quarterly filings and submit it to the SEC but what our experience is for vast majority of the questions these days and you don't need to ask like another person of the data team to help you answer it's largely self-service analytics.

The way we think about this problem is that definitely you need a different interface for people that are not very technical. Now that specific interface could actually be used by very technical people too. But the more technical people often they have a different jobs to be done like for example they want to see code they want to be writing code. Whereas if you show like massive amount of code up front to maybe somebody who have never written a single line of code, doesn't never even heard of the word SQL that would be too daunting right. So what we ended up doing is we actually created a whole new interface in Databricks called Genie One and when you open Genie there's nothing else all it does just hey here's like a box for you to type in kind of like Google search type in your question and then Genie One will actually go help you generating a report or answer or do whatever you want to do. And that interface is used heavily by everybody including technical users and non-technical users. But we also leverage the same underlying technology like Genie ontology and all this and we have a separate product called Genie Code and what that does is Genie is the so the AI system in the traditional Databricks interfaces for far more technical users and it understands for example here's a notebook and here's like some cell with code and the user is trying to do for example build a linear model to predict some revenue and it understands hey here's like Spark trace cases and there's a failure in the pipeline, how do I remediate it? So they kind of vary and give you far more technical detail. So we kind of built two different products where the more simplified one is usable by everybody is more for simple analytics and the which is Genie One and the Genie Code is the one that's facing more technical users and so heavily integrated with existing or pre-existing Databricks interface.

Notebooks are going to become even more important and if you think about it every chat interface like chatbot interface is kind of a notebook. They don't position it as a notebook, but it's really a notebook, right? It's no like a question is a command you run more or less and then it comes back with some results. The results could be semi-structured that weave in with charts and tabular data structure. It's not very different from notebooks. As a matter of fact, we're seeing many same kind of feature requests that our customers were asking for on the notebook product from like 10 years back. Now they're asking for exactly the same set of feature sets for the chatbot product. Like for example, they want to be able to share a report or set of an answer generated by the chatbot. Makes sense. And they want to be able to reorder sometimes the questions. Again, that's like one of the big thing we added very early on on the notebook side. And they want to be able to actually turn maybe a sort of a answer into a recurring thing. So kind of turning into a production job because this is something they want to see maybe every day or want to send it out to a mailing list every day. That's something we actually added 10 years ago in our notebook product. So we figure there's going to be a lot of similarities. Now maybe it's not going to be the identical user interface because notebooks target the slightly more technical persona like notebooks a little bit more code-heavy but we think so the form of the notebook is going to be very it's going to become even more popular and maybe we won't call it a notebook but it's effectively sort of a notebook form under.

Amazing stuff. I mean, this kind of makes sense, right? If you sort of in every enterprise work some sort of collaborative work, notebooks are great for collaboration and notebooks great for interactivity as of looking at inspecting results as you're writing code. That's not very different once the programming language becomes English.

I think this is actually one of the most overlooked points when people think about AI is they think, hey, if I just have access to all the data, everything will be great. But then often you don't want to give the AI access to all the data because you don't know exactly what's going to happen and who's going to be using it. One of the thing kind of a very early decision we made in Databricks is the there's only going to be a single governance layer which is what we call Unity Catalog. Every permission is defined the Unity Catalog layer. So if for example I've given Richie access to this but not the other thing there's no way for you to actually circumvent that regardless of what interface you come in from. And this is very different from I think a lot of the past BI tools which is hey because of extract process that BI tools often do in order to speed up performance they carry a separate permission model. So it ends up being hey maybe Richie should not have access in the actual underlying platform but then in this specific BI tool suddenly now he can actually get in and then before you know it somebody forgets to actually update the permission policy and maybe that was intentional in the beginning but later something changed and it forget to update the permission policy. So in Databricks there's only one single layer for governance which is Unity and in Unity you set the policies and then it's enforced everywhere. Your AI agents basically on your behalf and as a result it can only see what you can see if you can see the HR data you can or maybe more likely you can maybe see the company sort of employees or org chart but you can't actually see for example or more sensitive PII, SSNs and so salary information and all that. Everything will be respected and you just can't get around it.

Well, the way we did it is actually the agentic layer always access data through the data foundation layer. So the agentic layer never opens up on its own. It goes through this narrow ways. As a result, the narrow ways enforces everything. I think that's how security system needs to be built. By the way, in general, I think the because agents are pretty new concept. There's a lot of innovation happening. I think the industry as a whole have overlooked how important governance is and how important security is and I mean especially on the personal side right if you look at OpenClaw and Hermes agents it's effectively once you give it access you can do anything and that's super scary and I think a lot of best practices kind of maybe enterprises have learned in the last 30 40 years of building kind of data systems are largely we're relearning a lot of those in the industry and like having schema is a good concept because it can help you identify what columns are PII can enforce access control on those just having a single markdown file for everything is not a great idea nothing wrong with markdown files but that's probably shouldn't you shouldn't treat markdown file as a database I think rest of industry will have to sort of relearn a lot of this unfortunately but the way we've been doing is we we we spent a lot of time basically building data governance in the last few years. We just thought hey there's such a critical aspect of making agents actually work and deployable. So we build it in a way that hey you just can't circumvent there's no separate path to go in. You always go through a governed data foundation layer.

I think we will have to figure out a way. And I think a lot of it has to do with also introducing maybe different governance layer like you cannot have the agent itself police itself because it could actually hallucinate it could decide to do things like it's very difficult to explain exactly what happens like the same way we don't trust a single human being to enforce themselves 100% there's always some sort of other guardrails or access control I think the same thing will have to happen with agents so in a way basically the governance layers can't be in the agent layer. It needs to be below the agent layer or has to have some other agent enforcing it. By the way, what one of the thing that we actually announced was this thing called OmniGen which is actually Mate's like he's personally driving all of this and what OmniGen does is actually allows you it's less about data access control but allows you to declare policies or agentic governance like for example you're allowed to do this but not the other thing. And it has a separate supervisor agent that's trying to enforce and look at, hey, is this agent now going outside of the guard rail that I've already set up. I think that's like one approach and it's it's it looks pretty promising so far, but we'll see. To how I'm sure there will be more approaches that come out but it just can't be the case that five years from now or three years from now everybody just decides to give sort of their personal agent or enterprise agent permission to everything and then hope for the best.