I think we have AGI. I think we have artificial general intelligence. We really haven't.
You hear these 95% of projects fail, but that's actually what you want.
I think the LLM is a commodity. People are not saying that, but it is a commodity. Like you can get gas from this gas station. You can get gas from that gas station. Doesn't matter. Just compare price.
There is an AI bubble. Okay. So then Glean is also in the bubble. Everybody's in the bubble. No, I would say there is a bubble. I would say those three camps.
Yeah. There's a super intelligence quest camp.
I would be very worried there. There's a second, the researchers doing the, you know, that's definitely not in a bubble. They're like the...
Yeah, they're super sober. Nobody cares about them. And then there's the third camp, which is us trying to make this valuable. We're not in a bubble in a sense that we're not spending huge amounts of capital on what we are doing. We're just trying to get actual economic value inside of these organizations.
Two legendary builders, Aliant. I'm so thrilled to get into this with you because both of you have seen every super cycle I've lived through. Internet, mobile, cloud, data, and AI. Not just through the super cycles, but also through the hype, the trough of disillusionment. And this time it's different. Today we're going to chop it up on the state of AI. You know, let's start with a 20,000 ft view. Take stock of where we are. AI, we've seen consumer AI, billions of users. ChatGPT said the guns went off three years ago. Cloud perplexity. ChatGPT people use it in the room. On the SMB and developer side, you've got hundreds of millions of users with cursor and codex and cloud code and so on. Enterprise, on the other hand, there's a lot of divide. It's hard to see a lot of fog of war. On one side, you've got models that are earning math benchmarks and science benchmarks and engineering benchmarks. But on the other side, you've got the MIT report that's saying 95% of AI deployments don't work. What's the reality? Bridge that gap for us. Lay it out as you see it. View from the top.
So, I think first of all, I think we should know that people use AI in their personal and work lives both. So there's not so much of a divide. Like, you know, everybody in your company is probably using ChatGPT and Claude and other tools on a daily basis. The thing that I feel is happening in enterprise is you hear these 95% of projects fail, but that's actually what you want. Like, when you are actually experimenting with new technology, if all of your projects are failing, that means you didn't try enough. So, I think when I read the study, it was not a surprise for me. We're going to actually see hopefully similar stats next year too, because we want everybody in the industry to be really eager and experiment and actually figure out how to get benefits from this technology.
This would make you guys by default the 5% of AI that is working, which is one in 20. Maybe you go to the 5%. What is a use case that is working and not just working like it's saving me time, but like it's working and it's transforming my company? Something that you can take to the bank, to the CFO, while the CFO will notice it but the legal won't shut it down.
I mean, look, we're seeing a lot of use cases that are working. It's just that you have to... it's not just you can unleash the agents and it just works. It's an engineering art. If you're going to have a company that's going to be really differentiated, like my company or your company or anyone's company, and you want to beat the competition, you can't just quickly put something together and think that your competition is not going to do the same thing. So, that's going to be something that needs evaluations. It needs something that you're going to productionize. It's going to take effort. You need a great team around it. But, we're seeing a lot of them. I'll give you some examples. Royal Bank of Canada built agents with us that basically take as soon as an earnings report comes out. So equity research analysts, their job is to put together these reports that say like this is a buy, this is hold, and so on. The agent goes, gets the earnings report, gets all the previous earnings reports, gets all the competitors' earnings reports, gets everything that's going on in the market, does the full analysis, the news, everything, puts it all together, and it can get the equity report out in 15 minutes from the earnings call. Industry standard is two hours. Of course, it's going to get commoditized and others are going to do that as well, but that's actually a really important use case that we're seeing in finance. So that's a finance example. It's like and there's lots of examples like this. Sifting through hundreds of thousands of documents, SEC reports, so on. That's finance. Let's switch gears. Let's go to healthcare. Healthcare is completely different. In healthcare, we have a customer, Merck, in the life science space created a model called Teddy. Teddy stands for Transformer Enabled Drug Discovery. This is a transformer model, kind of just like large language models that can predict the next word, but it instead can figure out which genome is missing if you remove a genome. So it really understands the gene regulatory network and can really start telling you what's happening with gene expression and so on. So this is really important for drug discovery. It's the beginnings, but this is going to actually help us do things that we couldn't do before. Let's pick retail also. So I'm picking different healthcare is one, I gave you finance, the RBC one. Let's go to retail. 7-Eleven agents that completely automate the marketing stack. Actually, the marketing stack is going to get disrupted pretty heavily. So these agents can basically prepare, they can segment the audience like this segment wants to hear this, and they can prepare all the marketing material that's directly targeting you guys, and they can put the campaigns together and do that. 7-Eleven was doing this before as well, but we're seeing this at Databricks as well. More and more is being done by agents and being automated. So you can just do it faster and you can segment more fine grain because before you had to create the content for the groups, that was a heavy content creation, something that was human manual labor. Now you can actually do that much more, like you can have all your web materials completely customized for a target group. So these are examples where it is working. There are also lots of examples where it's not working. Even with Databricks, we're not just the 5%, we have some of that 95% too.
Ali, follow up on that. These are great examples. Thank you. Maybe if you were to take it a layer up, what is common across these use cases or these organizations or these CIOs that's making these use cases work? Is there something that we can pattern match?
Yeah, look, I think the LLM is a commodity. People are not saying that, but it is a commodity. Like, when I took econ classes, commodity was when it's interchangeable. Like you can get gas from this gas station or you can get gas from that gas station. It doesn't matter, just compare price. LLMs have become that way. It doesn't really matter. This one is better right now, next week that one is better. You can't even keep up anymore. So they're a commodity. So it's not about that. It really comes down to your company. What data does your company have that's special that your competitors don't have? Can you leverage that and can you build AI that really understands that data? Because that's not a commodity. There's not an AI out there that understands all your business processes in your company, your secret sauce and your data. That's not a commodity. In fact, that's closer to the 95%. It really comes down to that. Or if you have a complicated process that just your company has, this is how you deliver your product and services in your company. And it's that portion can be disrupted with AI somehow. If you can do that now, you can get ahead of your competition. But it comes back to what makes your company special. Unfortunately, a lot of companies are just building commodity stuff. Like you should not be building that because it's a thing that every company can do. It's not special to your company. That's the problem in a lot of the industry. Another problem in industry is a lot of demo-ware. It's really easy to make cool demos with GenAI, and therefore we're seeing a lot of cool demos, but that's all they are.
Yeah. Well, you know something we say around at Ultimator quite a bit is your AI strategy starts as your data strategy. Yeah. So you got to get the data house in order first. And you know there's a lot of reasons for use cases that are not working. Maybe give us an example of the 95% of an AI bet that either of you had at Databricks or at Glean that did not work out and why it didn't work out.
It's actually an interesting thing. You know, with engineering today, you build systems and never before have we been in this mode where you start with a great idea and it doesn't seem like a good idea anymore within two weeks, because we see a new development that happens. So there are numerous failures in engineering on that front. For example, some of our fine-tuning work, building models for a specific use case within our product, didn't really pan out for us. Ultimately, the choice was that we can go with already built models, whether they are small open source models hosted on Databricks or one of the large foundation models, but from a corporate use cases perspective. So actually, we are also in many ways in this mode where a lot of our work, I would not say fail, but it actually takes much longer to generate success. There are things we are actually trying to automate a lot of our business processes internally. For example, one thing that I want is in our company, I want everybody to actually know exactly what their top priority for the week is, what they want to work on, and maybe we want an AI agent to actually first tell them what their priority should be, and we want it all to be documented, and we want a system which then rolls it all up, and I get a view every week where I can actually quickly see what are all the different people working on in the company and are they aligned with what I want them to work on. And this simple thing, companies have always tried to have this, as CEOs you always want it, and it's always hard to make happen. We thought that AI would simply magically do all of this work because it has all the context inside the company to make it happen, but I still don't have it. So things do take time. To Ali's point, AI is just one more tool that you have in the toolkit. It does not suddenly make building complex enterprise systems something you can build in one day.
You know, the last time enterprises got this excited about a tool was called RPA. And we know how that ended. It unfortunately fizzled out. And somebody in the audience yesterday was like, hey, how is this time different from RPA? It seems like the same movie, bigger budgets, better actors. What's different this time? How is the nature or the architecture of the technology different from the previous automation cycle?
Yeah, I mean, first of all, RPA didn't capture my attention at all, so I actually can't... What is it? So I think I would not compare these two technologies at all. What we're seeing now with AI is so fundamental. When we saw it first, it was basically magic, and we couldn't believe that this is a machine that is doing this work. Machines simply cannot do these kind of things that we saw them do, like writing on their own, having emotion, understanding emotion. So it's fundamental, it's different, and that's why I don't think this technology is going to fizzle out. You don't have to be a financial expert or a deep thinker on business. This is obvious stuff. All of us know, all of us feel it. All of us can see the capability of this technology, and we know it's special and it's going to be around.
Yeah. You want to hear my RPA? Please. You know, I mean, it was rule based, and the problem with it, especially if you want something that automates what's going on on your desktop and automate the work that's happening, it's just that there's too many unexpected things that happen, and it's just hard and brittle to set it up. It wasn't learning ever. So there was zero learning. It was like you tell it exactly, here are the rules, and if it got something wrong, you need to go back and expand the rules. Here, you have something that's learning, so it can improve, and it can generalize, and it can understand the patterns and do pattern recognition. So that's the fundamental difference between these two. Now, there have been many startups that have failed in the generative AI space that tried to replace RPA with generative AI models. There's many startups that failed, actually, that I know of, some high-profile ones. It's because the paradigm we live in today with AI is still problems. The biggest problem is that you bake a model, and that's where it's learned everything it needs to learn, and then you freeze it, and then you launch it, and then maybe you give it some context, but that's it. It's frozen. So therein lies the problem. We need an AI that really can sort of continue learning while it's using the desktop and clicking around. So I do think this problem is hard to nail, but I think Arvin is right that there's no comparison at all. It's like rule brittle rule based stuff versus learning agentic system. I think it's going to nail it perfectly. But we haven't really nailed computer use yet.
Yeah, working on it. The number one shift is this move from if-then-else statements to a more generative solution that figures out the solution. And so you're trading breadth for maybe determinism. That seems to be the difference. And you know, there's a lot of CIOs in the room, we've got here, and they've got budgets coming up to plan. If you were giving advice to them, like, hey, based on everything I know from my customer base, here is one thing or two things that you've got to figure out and align incentives on, or it could be a reliability problem or design. What advice would you have for CIOs who are thinking about their AI budgets right now?
Put it on Glean. Spend more. Yeah, put it on Glean. But I think the one thing which is important in AI market today is that it's very new and there are many players. In fact, every software company is also an AI company now. You can go and check their websites. So, I think it's just hard to actually figure out where to allocate those budgets. What we tell people is that I think the winners are yet to be identified, and so experiment with more vendors, do shorter term contracts. And while that's easy to say, it's hard to actually implement because every product that you try has a cost that you have to pay to make it even tested. So you have to also pick products that are easy to test. Those are the ones that don't require you to spend the next 6 months trying to implement something and you have no idea what's going to come out after that. The products of today, the products that are built with the right AI, they should work very, very quickly for you.
Crawl, walk, run. We're going to take a peek into the future. Shifting gears, you know, one of the things that keeps investors like me up at night is a quarter trillion being spent on Nvidia on the semi side of things. Assuming that is just 50% of the capex, you're spending about half a trillion on capex, and then you've got to earn about a trillion dollars of AI revenue for all of this capex to be worth it. And just to put this in context, the entirety of the software industry earns about $400 billion of revenue. This seems like a physics problem at this point. How do you think this plays out? You've got to make about a trillion dollars of revenue to justify this present spend that's already happening. How do you think this shakes out? Maybe we start with you, Arvin.
Wrong person to start with, but you know, I'm an engineer and I shouldn't really think too much about who's spending what money. We're here to build our product and add value. So in some sense, I've not really thought too much about this problem. But if you think about AI, AI is not actually extending software in a marginal way. It's a different product, and in fact, it's actually going to grab a lot of revenue that today is in the services industry, which is 25 times larger than the software industry. So there's a lot of spend that is going to move. The spend that you see happen on AI is actually those service dollars that are converting into AI or software dollars. But with that said, maybe you have a more informed view on this.
Yeah, let us just do our engineering, right?
That's really good framing. And the way this manifests in the world is there's a data layer, there's the intelligence layer which is where camp 1 is presumably producing a lot of great models, and then there's the software layer where the users engage with.
Where do you think value will accrue if you were to design 100 units of value across these three layers? The data layer, the intelligence layer and the software or the application layer. Where do you think value occurs in the next five years?
All right, this is a tough question. I mean, I think all those three layers actually are very fundamental.
I thought you were going to add a few more which are not influence semis but...
Yeah, because I feel like as Ali was saying, the models are going to be available to all of us. They are going to be commodity and it's going to be hard to see that the more spend goes to them versus these layers on top. But it's hard to come up with where the most value will be. And I also don't know if it changes from today's technology architecture where in a pre-AI world, any enterprise application and data systems, you have data systems, you don't have enough of that intelligence layer today, and then you have the application layer. So some dollars will shift into it. I do think that the intelligence layer is actually going to be pretty thick. Maybe it'll capture half of the enterprise value.
Yeah, I think there are more layers in the stack depending on how you want to do it. But as I said, the LLM is a commodity. As Arvin said, you can get them like you know, but that doesn't mean those companies are not going to be valuable. They can be very valuable. TSMC is very valuable. They're going to be kind of like these fab-like companies. But they're interchangeable, and we've never seen something like that ever. I have not seen all these people just switch LLMs in one day. That's not the case with your iPhone versus Android or your Windows versus your Mac or Google Sheets versus Excel. That's a huge religious battle inside our company. But LLM, because it's a commodity, it just speaks English or any language you like, and it gives you different answers every time. Might as well just try the cheaper one, the cheaper commodity, or the flight to smarter commodity. You can't even really tell the difference. So what is special is the data that you have. If your company has data that it has actually collected that your competitors do not have, like Glean is amazing, but if you remove all the data from Glean, there's no use to it. So it's all about the data that you have. And can you secure the data also? If we're going to have agents running around accessing this data, like, oh, that's his HR data. Oh, here's Purva's salary information. Oops, I blurted it out to all of you. So how do you lock it down? How do you make sure that there's governance? There's also a lot of worry about what if it's using a Chinese model? What if it's accessing this information? What if it's sharing this information with a competitor? What if it's interacting? So the governance security layer is going to be super important. But I do think most of the value will accrue to the apps.
