Our next guest has spent his career building the invisible infrastructure that powers enterprise software. And now he's helping shape the future of enterprise AI. As CTO of IBM software, Anand is focused on one of AI's biggest challenges, connecting foundation models to the trusted enterprise data they need to deliver real business value. Before IBM, he co-founded Stepzen, led groundbreaking work in databases and information management at IBM, and then helped scale Apogee through its IPO and its acquisition by Google. Few people have spent more time thinking about how data, APIs, and AI come together at enterprise scale. Anand, you've been a longtime friend and I think first-time caller. Welcome to the show.
Thank you so much, Satyen. That was I didn't quite recognize all the words of praise that you said. It was embarrassing. I think I turned a bit red but no thank you very much for your kind words and as you know we have been chatting on and off and through some mutual connections we said let's kind of talk a bit not just among ourselves but also to a broader audience. So, for everybody else, I went to Satyen and first said, "Satyen, please come on my podcast." And he said, "No, no, he'll only come on my podcast if I come on his." So, this got scheduled first. So, here I am, Satyen.
Yeah. The best kind of quid pro quo. So, this is a two-part episode. And so, maybe let's just jump right in. So, you've been through I mean, you've obviously done an incredible amount of work research around databases. You founded companies around APIs. Certainly Apogee was a big investment there and all of that kind of comes to bear in orchestration in this AI moment. Maybe talk about a little bit about this moment. Is this kind of the same as many of these other prior revolutions or is this different?
It's really really different. I mean I feel it in the way I work and I'm sure you feel it in the way that you work. But the good thing Satyen is and that the thing that you're trying to do at Alation or we are trying to do at IBM and others is that the foundation is still the same. The interaction changes right so we had primarily built stuff for either automated flows or humans who would kind of swill chair or not swill chair based on all the fantastic work that you have been doing Satyen. But in this new world, if it is AI and agents that are doing it, does it represent kind of a different modality and everything else remains the same or something in the base also change? So those are the kinds of questions that we have got to kind of always answer. But the fact of the matter is that the data systems are the same, the systems of record are the same, the problems of integration are still the same. The problem of which is the authenticated piece of data here is still the same. So problems remain the same but the interaction paradigms change completely and because of that perhaps some of our old layering needs to be rethought.
Yeah. So I think you mentioned that a lot of these fundamental problems are similar. These problems of providence, these problems of understanding, these problems of metadata. And I think the convergence of having AI do things in the way that a human would have done things meant that sort of many of these estates, these operational estates and these data estates, you know, APIs and technical consumption versus human consumption, like all of these borders seem like they've collapsed in my mind. And that fluidity feels like it's forcing both a steps function change in speed but often when that happens you have this sort of corresponding dynamic where the form of what matters changes too and so I think like a lot of this work at least what we're seeing is that a lot of this work around metadata and insight generation and governance used to be side of desk work and now it's entirely and completely operational at least for the most progressive organizations and to me that shape of that work means that you have to almost rethink what an application is and how it functions. I'm old enough to have lived through a few changes right so if you recall why did APIs come about because people had kind of physical retail then they went to e-commerce and then Steve Jobs came out with this and they said we can kind of create either a third different silo or why not let's kind of build an API that will allow us to kind of cut across everything else and that kind of gave the API boom and that API boom is now lasting for at least 15 18 years but the point that I took out from there is that every time there's kind of a generational shift something changes in the infrastructure also okay all I was pointing out was that the importance of your data your metadata your integrations your this is the single source of truth etc. The importance of that still stays because anything that is exposed that is not correct or good etc just amplifies the fast agentic reasoning on top of it presumably in some incorrect directions if you have some mistakes the importance stays but obviously the interface when the predominant thing is a machine as opposed to humans or predefined has to actually change too right and today for example MCP is the wrappers around APIs right and if you kind of poke into it that's what it is most of the case but is that the best thing right who knows right who knows whether that's the best thing if skills is coming about what is the role of that can you actually there are people who are now saying that CLIs are the better way in which some of these skills and mechanisms actually operate so I think that the interfaces will change but the importance of good hygiene around data and everything else doesn't actually go away irrespective whether it's humans pre-integrated workflows or now agents that would be my assertion
Yeah I agree with that and I think on some level you have these systems of record like classically Salesforce or whatever system ERP workforce management whatever it is that you want and have. And then you effectively then bolt on all of these things onto the CRM in order to be able to sort of gather some of this data. And historically, it's just been impossible to make this data correct because no sales person is going to enter all full fidelity of the conversations that they've had. And so now all of a sudden you just get like garbage in these fields and the data is either not correct or not accurately describing reality. And so you have this kind of world where the operational systems are important and are driving things, but they're not necessarily always accurate or correct. And so the challenge is these problems have existed and remain but now the importance of solving them is very very very high. But they're still very hard to solve and I guess how do you see this forcing function evolving in terms of getting this stuff right because it's been always side of desk it's always been expensive and it's always been something that a CIO doesn't prioritize and so how does one actually think about where money is put and time is put in order to fix this stuff
No no that's really awesome point the way I like to think about it is data for AI which we just talked about that if the interactions are changing etc etc but also AI for data which is can you actually leverage AI to kind of get over some of the hurdles that you have had in the past with respect to how do you actually cleanse and standardize and make things better. So I like to kind of think about it in those two framings. I don't know whether you think about like that but let's talk a bit about how can AI actually help us my view with respect to some of these gnarly data integration and data cleansing problems and everything. So there are kind of in my mind there are two things that it is actually quite remarkable how the AI that uses the data is actually fairly tolerant of bad stuff. Okay. So, previously if my API had a slash member slash ID and I accidentally type slash member slash ID as opposed to slash member/ ID then it will just kind of return whatever it is. I don't even know what the code will be probably some 400s or something like that. But today AI can say okay I'll type slash member ID and it'll come back with that code and it'll kind of reason among itself and say oh I got it wrong okay and therefore let me kind of readjust it. Obviously we are paying more to the AI gods with respect to tokens and all that stuff but they kind of self-correct around stuff that might be bad. So, in some ways if you believe that the users of all of this stuff are shifting towards AI, you have to be less prescriptive with it being exactly correct. Okay, so that's my view number one. I don't know whether that makes sense to you or not. But then the second part about is that for the things that you have to get right, AI does actually help. And I've just kind of I've been thinking a bit about this in various formats. So number one is that the classic problem of resolving unstructured data and structured data is a perpetual problem, right? I mean it's like a problem that always exists. Is it possible that if you kind of think about kind of an embedding space so that your structured data and unstructured data are kind of landing up in some embedding space? Is it possible for you to actually unify some stuff that you couldn't have unified before? Is it possible for you to kind of think about certain kind of new fusing techniques, correlation techniques, analysis techniques that would allow you to kind of fine-tune a model. I mean obviously for so for example we had somebody in my podcast Wana who is the CEO of a company called Kumo that got acquired by Nvidia and he was talking about how he's taking kind of standard transformer architectures and then applying to the data problem to kind of solve some of these things associated with it. Right? So I think that the opportunity to leverage AI to actually get around that very very laborious and error-prone tasks so that you can expose to the AI use cases or human use cases something that's much better and much cleaner and then depend on them to actually work around not perfection. I think that is in my mind some really really large opportunity.
Yeah, I think that AI for data I love that expression of the problem statement is a huge opportunity. It does shift the work I think like what we've experienced is that you can apply many of these techniques but you still end up at some level of error. I mean ultimately it's a function of your eval and what you call good but it's very hard to get you know like anything you know whether it's web reliability or whatever you know you can get to the first 90% 85% relatively quickly and then thereafter it just becomes harder and harder you know like every order of magnitude becomes more than an order of magnitude harder and so you have this kind of so now the work shifts to like the last set of quality guarantees and then many of these processes are dependent on very high fidelity if people are willing to automate it like in the human offline process maybe that they're not willing to automate it but in the agentic world they have to get sort of extraordinarily high fidelity to trust it in order to make it move forward and I think people are struggling with this problem in its essence like because they just don't know how to think about what they can trust and what they can't trust because they don't like this kind of eval thing is a very new way of thinking for people. I guess going back to some of these new techniques I mean you mentioned this kind of classic structured and unstructured data like dichotomy and applying transformer models to structured. I guess how do you think about you know the way I've always thought about it is that in some ways LLMs have kind of developed this new pipeline like on some level your typical structured data pipeline is basically the structuring it's essentially taking the world of unstructured reality and just superimposing on it boxes of things that we want to just count or label you know that's like a very simplistic way of thinking about the world even
Yeah and like ML features are effectively the same thing it's like okay I'm just taking some things that are unstructured. I'm just going to like glom on these features and start counting them or start measuring them in some fundamental way. But this new kind of LLM to your point this embeddings world is a very different way of modality of thinking about the world. I guess where do you see I mean IBM sees so many customers and so many people and so many different organizations. Where are people in absorbing this kind of way of thinking about kind of using LLM to sort of help do some of these classic problems or deal with some of these classic problems? Are we very early? Are we mid-stage? Like where do you feel we are
No, I think we're very early. Right. So I think that what's really going on is that with your help with IBM's help and others people have gotten a bit of their handle on their data estate. They've got kind of the integration flows done. They've exposed kind of the right catalog perhaps through you etc etc you know with all its kind of issues and everything else. So the quick hits then is actually changing the application paradigm around the piece of work that has already been done as opposed to using that new paradigm to change and do something slightly different in the data infrastructure. Right? So the very first thing that people are doing if I may say is to say okay can my data estate deliver the context that's needed for these applications to actually work right I mean we all know that what Anthropic knows is not what our customers know inside and therefore being able to deliver that context is really really important. So I think that the first sets of hits are happening on this delivery of context. Okay, which is not fundamentally saying how do you kind of do all the hard problems of AI for data. The second set of hits that are happening is to say that look the application paradigm the agentic paradigm is a very very different paradigm. It's try this, try that, go this, change chain of thoughts, figure this out, etc., etc. Therefore, exactly to your point from 15 20 minutes back that therefore it changes the interaction paradigm and therefore what infrastructure we need to do in order to go instead of kind of pre-wired stable flows and everything else to okay can you please produce this piece of information for me or can you do something else for me and everything else and therefore what that means is some other constructs have to be done which would allow for pipelines to be created on the API or metadata to be assembled and everything. So I see these two happening in kind of big way. Just to kind of summarize delivering context for these applications and then seeing whether these applications can be used. You can change some of the infrastructure in support of these applications. And the idea behind it is that look there's always been kind of a long tail of integrations and metadata stuff that has never been done before, right? Can you actually make some of those things happen in terms of kind of fundamentally thinking about can I think about the primary data integration and catalog creation and everything else is done through AI and therefore what really needs to change with respect to either the embedding space what's really verifiable I mean obviously AI works very well when things are verifiable as opposed to not verifiable right so what does it really mean in that space and I think that is something that I haven't I mean obviously all of us are thinking about it but I haven't seen it kind of yet make a big impact in our customers probably because we as vendors or at least us from an IBM perspective haven't followed Alation that much we haven't kind of made a big deal out of it
Yeah I think this context thing is obviously super topical and everybody's talking about this idea of sort of context and centralization of context and provision of context and then to your point. I think the second conversation and I think that's an interesting conversation because it's obviously what drives fidelity and all this governance and context and governance and context. I mean you can build an agent in 20 seconds getting all the governance and the context right it'll take you 100x longer.
Absolutely. But you've thought a lot about this infrastructure question like what are the primitives and I actually think that's like I mean you're probably about as deep as anybody into this question and you talked a little bit about MCP and maybe some of these like sort of ephemeral pipelines like what are some of the ideas floating around in your head around what are the areas that could experience change and how people ought to think about some of these primitives differently than they were before.
So if you really think about it, the fact of the matter is that data is created everywhere and it's like you're always kind of bailing water to try to kind of get it in the right place, okay, for all the applications that you got to do. That's kind of the perpetual paradigm. Is there a way to think about it slightly differently? Okay, which is to say that look enough metadata. Okay, as opposed to enough centralization of the data. What was the reason for centralizing the data? Obviously you got an opportunity to kind of clean it but there's also an opportunity out there Satyen to then run these really really complicated queries okay which are not federated across multiple systems and you can do a table scan as opposed to something else and what we have done is typically we have mixed and matched the two. We have said that look we want to kind of bring stuff in one place so that we can both kind of standardize it and we can build those various tiers of my warehousing hierarchy or whatever the new data lake hierarchy but also I can now optimize and run stuff really really efficiently. Okay. If the paradigm is shifting towards no look yeah you've got to run these long-running queries but in the agentic interactions really it's kind of short short burst of queries first let me find this and then let me find this oh this didn't get me there let me now find something else so instead of let me kind of run a mega query fixed report etc etc I'm trying to discover my way through to certain things then the paradigm changes and perhaps this concept of centralizing everything in one place for being able to efficiently run queries may not be as important in this kind of new workloads because they're not really executing something really really fancy out there. They're just kind of finding their hill climbing their way through the right answer. So in that case some of the primitives that you have thought about which is let me kind of move stuff etc etc don't have to be thought about it. More importantly, what has to be thought about is how do I kind of express what I can do? How do I kind of then efficiently execute on it? Give the kind of reliability and everything else. But in both of those cases without that metadata, the agents are basically groping around some space trying to figure this out and therefore that particular part of it still remains important. Whether that needs to be accompanied with kind of data movement etc. I don't know. Okay. So that's one place where I think that it may be the reemergence of federation with kind of good strong metadata space on top. That's one view. The second view that I have is that fundamentally fundamental problems that we always solve try to solve for is that as much as we think that the data is clean when it ends up in our databases okay it isn't right and the question really is what is the effort so for example I mean just certain just go back we used to write when we would do this kind of if you even go back to something like NLTK which was natural language some Python library it would be very careful about look I've got to clarify on what my synonyms are and what my this is in order to for example find out right Anand may be spelled in two different ways and so therefore this or that and you do the expansion everything else and then Google came about right if you all recall in kind of the old days and other you had to do that anding and oring and everything else in order to kind of get there. And then Google came about and said, "No, no, screw it. Just tell us what you think you want and we'll try to figure out what you want." Right? And that worked well. That worked well because Google had that huge amount of data that it could figure out that Satyen and Anand is probably a misspelling of somebody with etc etc. So I think the same thing kind of applies now which is that if you now think about and look at kind of the data fragments and other things that are misspelled or incorrect or others nowadays I don't go about when I'm doing an agentic interaction certain to ever bother doing a spelling mistake. The only reason I correct spelling is because I don't want it to think that I'm a doof, right? But I don't really need to correct spelling because it kind of figures it out. So I think there's something really big out there which says that can we apply some of these AI techniques that have been applied on large corpus of data to structured data. Maybe it is based on embedding space that Satyen with a Yen and Yen end up in the similar space and therefore they're probably the same as opposed to kind of old stuff. But I think that space is ripe for kind of a phenomenal amount of innovations and that's where my friend Wana is trying to do to apply some new transformer models and all that for structured data. So those are kind of two examples that kind of come to mind.
Yeah. No and both are really interesting. I mean in the first case we've struggled with kind of this federation question for a super long time like you know do you have to move the data and you know there's always this conflation between kind of the centralization of data and the centralization of knowledge. We've always talked about sort of centralizing metadata obvious and look that's obviously talking our own book but we've always talked about the centralization of metadata and you know now context whatever you want to call it versus the centralization of the actual physical data but even now that's kind of getting super like muddy because now it's like well context can also be operational data so now is that to be centralized or not I mean there's like all of this like new same same same battle all like in a new location. Do you feel like there's an opportunity for I guess do you think the compute opportunity in federation is a new opportunity? I mean with Databricks versus Snowflake you've seen just massive centralization and now there's kind of this Iceberg table thing kind of coming along and so do you think that maybe we'll start to see like is that the end of the story or do you think there might be even yet another story in the world of compute where federation is a more real thing and I guess are there things that you've seen where that's been more or less promising?
No, no. I actually strongly believe that two things are happening. One is that agents are being asked to do increasingly complex tasks. I mean you know this. I mean, when I used to code, I would do kind of the old cursor joke was that the only keyboard you need is a mic and a tab button to accept whatever answers kind of come back, right? But it's not even that case now, right? I mean, I don't even have to accept the answers. Now I'm
Giving kind of an arm and all that stuff, and it's actually figuring out and reasoning through what needs to be done, just in the case of coding but in tons of things. Today I was trying to do some research on some other topic and it spawned some agents and it kind of came back with something and I said I don't agree with it, etc. whatever it is. But the point there is that the model, the kind of the, if you think that agents are just going to do the same thing that you're doing except machines instead of people, it's kind of boring and I don't think that's going to happen. Yeah, it might happen but it's kind of just that shift. The real change will be they're doing something different that we haven't done before and in that case my view is that the paradigm of the chain of thought reasoning, which is really what where we are actually going, AI is really going towards that, is so different than kind of predetermined stuff and to me it's not at all clear that all the structures that we built to answer repeatedly the same queries or certain associations, so smart that he has thought about a day-long query that you want to optimize. Okay. So I think that the paradigm is really, really different. Now obviously the two paradigms will stay together for quite some time but I'm pretty clear that the dominant paradigm would be more ad hoc agents kind of thinking through some sets of issues and obviously once something gets repeated you want to kind of relegate and execute it repeatedly but the number of tasks that will be done in the new way will exceed the number of tasks done the old way.
Yeah, it's a really interesting construct. I guess we've done all this work to your point to get kind of this last mile fidelity right, you know exactly the right, you know IBM is always IBM in the database, it's never you know international business machines or you know Intel business mash, it's like always just this one canonical thing because by having this one canonical thing I can you know count it exactly and I can you know identify it with it. Exactly. And I'm and then I know what to do as an idiot.
Select select count star where where name equals a single single code IBM. Yeah. No, but then you also have to unfortunately you still have to do what's called invariance on caps. I forget whatever it is, right? Sometimes IBM will be small IBM, sometimes be big IBM. So you still have to do that kind of manipulation at query time.
Yeah. And now to your point, you don't have to. And so then this kind of construct of like, and it's relatively like to your point it's not even very expensive for the LLM to figure out that these two things are the same thing and therefore like the level of sort of last mile fidelity you have to have is relatively lower than it used to be. And then you know to your point like these sort of task defined, you know outcome defined agents can go do a whole bunch of stuff and therefore they can you know spin up and spin down databases if they need to, they can you know build tables and kill tables, like all these things that you know people used to have to do in a fixed way these LLMs can do in a sort of ephemeral way and that's super interesting. I guess it means that now the start point has to be really high fidelity or you know that the implication of all that is that whatever you're starting from that has to be very high fidelity because otherwise something is going to go wrong if your instructions and your inputs are wrong.
Agree. Now we are all kind of system builders. So I mean I don't want to kind of undermine the fact that while a capital IBM and a small IBM can be reasoned together by the LLM, the fact of the matter is that it is expensive to send the query down to the database to kind of figure out right how many different variations of IBM should I ask for from the database, right? So I'll get kind of this big long list of ORs etc etc right so there are still hard problems to be solved but I think those hard problems can perhaps be solved in a slightly different way as opposed to just kind of standard indexing, just embed every row okay and when you embed every row IBM and small IBM, big IBM and small IBM kind of end up in the same space but exactly after that obviously without that kind of fidelity everything else is basically garbage and as much as the agents and AIs kind of change, they're improving on their hallucination right, hallucination is a big problem two years back, it's a smaller problem now because there's enough RLHF that's happened and everything else but the fact of the matter is if the data doesn't have high fidelity any amount of lack of hallucination is not going to solve the problem.
Let's switch gears a little bit. So, let's talk about, we've been talking so much about IBM like as a semantic construct. Why don't we talk about IBM as an actual construct? So, you're at IBM, I mean it's in some ways, you know, it's an institution that's been so central to this change. I mean so many data management and sort of manipulation and centralization technologies have been acquired by IBM over the years.
Built and acquired. Built built and acquired.
Built and acquired. Built and acquired. Totally. Built built.
And I mean you know obviously IBM research has been incredible in its sort of all of the thinking that's been done. So there's a lot of heritage here that speaks to this exact moment. So maybe tell us a little bit about this. Tell us a little about IBM like you know everybody kind of feels like they know IBM and heard of IBM and I guess tell us about what IBM is today and tell us about like what Arvind is doing today and what you're doing today.
One is it is incredibly, I mean you know this we know this that the technology moves faster than the landscape of our customers right so you would say oh my god everything is now AI it isn't right it isn't oh everything is a lakehouse no it isn't right so there's kind of a huge amount of look these things work let's kind of tinker at the edges of it as opposed to let's try to make something different and that's why for example from a data perspective we've got a huge Z business we of course have DB2s we have data stage we have everything else that you would know of, Informatica and everything else that is kind of centered around the fact that we basically run the business for a large part of the data real estate. So that's one right that's one and the way that is getting modernized is that the core system of records is not changing but in the case of DB2 for example we now have something called DB2 genius which says that now look clearly the most painful task about databases is not the excitement that you get out of it but really managing that damn database right and we have always talked about autonomous databases and autonomic databases and all this stuff but now it can happen okay because AI can actually observe and tweak and modify and everything. So around that kind of stuff, the basic problem that we are trying to solve is to say that look, how do we ensure that the people that surround it are actually able to deliver phenomenally large value by leveraging AI. Okay, so with some things called AI edition. So that's one.
The second one is what we spend a lot of time on which is how do you kind of think through the new structures? How is data becoming relevant for this new application in the world of AI, right? So obviously how do you kind of deliver context? How do you kind of build a vector store? How do you kind of collect information from everywhere else? How do you leverage GPUs? Our customers have GPUs of the wazoo, but GPUs after three years kind of go down in terms of their relevance for AI, but can they be leveraged for databases and everything else, right? So, how do you kind of do some of the things that we spend a lot of time in this and deliver something for kind of the new AI and applications out there. So that's kind of the second one.
And the third mode really that we are on is to say that look in this kind of new world we have spent a lot of effort in getting data correct in the right place and at rest and everything else. But is it the case that data in motion and data at rest? We talked about unstructured data and structured data. Again the problem of data in motion, data at rest has never been kind of fully satisfied and that's why we went and acquired Confluent and to say that look is it the case Confluent has some really really fantastic pieces of technology so I would say these are my these are kind of the three pillars: modernize things that our customers bet their lives on, second is build new infrastructure for delivering into the AI applications and third is combine with it. Those are in my mind kind of the three top pillars of our data strategy.
