Hello again, and welcome to the Inside Products Podcast. I have with me today two very special guests from a recent acquisition that we made of a company called Galileo, Yash and Atin. And we have Vikram, who is the third musketeer who is not here today because he has had a new addition to his family. So he's enjoying well-deserved time with his family. So we'll make sure that we have him on the next pod. But welcome to both of you.
Before I get started, I actually-- you folks have had an amazing background. You were at Google. You worked at DeepMind. Talk to us a little bit about that. What did you do over there? And what learnings did you have before we go into how did you think about founding Galileo and what happened from there?
Yeah, so before we started Galileo, I spent about a decade working at Google, building conversational AI systems. We started with very rudimentary speech-to-text systems. And there was a time when we were able to convince Larry, Sergey, and the business that conversational AI is going to be extremely, extremely important. That's going to be the main way by which the world--
People interface with tech.
--consumes technology. Yeah. And so we've gone over a whole host of models over the last decade, whether it's right from Gaussian mixture models to RNNs to DNNs to transformers and then to large transformers. And I think the big unlock that LLMs, which is the large transformer architectures, have had is we didn't have to really hard-code the natural language understanding parts into a hard-coded workflow that initially these assistants were built on. And on top of that, these models became multimodal. And so we were part of this early team that was building out these models and scaling them. We worked with Denison and team as well. And on top of that, our team powered about 20-plus products at Google, whether it's the Google Assistant, Google Cloud, and Google Meets, and the cars and the home devices and the phones. So--
Yeah, the same voice engine that you would actually--
The same conversational AI engine. So it's the voice and natural language understanding along with actions that you can do in different modalities. So I think that's--
Were you in charge also? Did you also think deeply about tone and--
Tone, punctuations, and then even quality of the generated voice. How can we make it--
How can you make it more human-like?
--sound more natural? Exactly. And so that was adopted by contact center businesses and the meeting-- in COVID, we saw a huge spike in these Google Meets and Webex and all of these products. And then conversational AI, transcripts, and now-- we take that all for granted now, but that was a tough time for us to scale this technology to long-form speech and long-form conversations. So that's where I come from. And towards the end of my student--
You currently run research and core engineering--
--in this team. And then often you run the product and the engineering side.
And so Atin, what was your background?
So I started my career in the earliest days of Siri, where I worked on some of the first versions. This is--
Tom Gruber, yes. And this is still in the ImageNet, AlexNet days, where deep learning is still kind of in the labs. But really learned how to build end-to-end productionized language systems before deep learning models were a thing. Spent a few years there, and then really joined a very interesting team at Uber back when Uber was on a tear. They were growing. And they had barely any AI footprint. It was a team of six or seven people. It's known as Michelangelo. And we literally built out their entire AI system.
Was this before Monic or during Monic's time?
This was before and during.
Yes. But we really were a few of the folks who built out end-to-end AI infra and the foundations and blueprints which have been adopted pretty much across the globe. We were one of the first to build feature stores, which are data systems for machine learning models, as well as monitoring systems, where I really saw the battle scars of how do you make AI models do the right thing and how do you measure them? And then met Yash and Vikram, who were solving the same problems at Google. And we sought out together and we said, how do you solve the trust problem in AI?
For unstructured data and language models in particular. That was the-- yeah.
So unstructured was one of our key pillars. And we said that we want to build for the next 10 years of AI, not the past 10 years. Because we had seen deep learning models get cheaper and cheaper to productionize. So we saw that coming. We didn't, of course, foresee exactly how it would manifest. But we kind of feel like we hit the nail on the head on the unstructured vision. And not only was it exciting for us because it was future-facing, but we feel like we spent the first couple of years before ChatGPT happened really building the foundations of understanding AI models.
Yash and Atin, what I'm going to do is I'm going to take a step back and say, if I were to think about how observability for AI is broken down in my mind, the three distinct components to it. Number 1 is the observability for the infrastructure itself. So how are you going out and measuring resilience of the infrastructure? GPU utilization, uptime, all of those pieces.
Networks. The second area then is agent behavior observability, where you can actually see how the agent is behaving, seeing if they're behaving in a way that you don't want it to behave. And when that is happening, can you dynamically at runtime intercept to alter that behavior? And then the third area is around tokenomics. Is the consumption pattern of tokens for these agents within the boundary conditions that you want to actually have, or is it actually going out completely in a different stratosphere, which could then create a very, very different economic impact for you?
Am I missing anything in that? And then how are you folks thinking about how Splunk fits into that entire picture?
I mean, I think, Jeetu, that these three things literally cover the entire agentic stack. While infrastructure seems-- we sometimes ignore infrastructure and think that it's commoditized. But as agents are scaling, we are seeing customers hit infrastructure bottlenecks very quickly. As we all know, just the networking impact of agents is just 450 times that of a human. And so our customers are actually asking us for an end-to-end observability across not just the agent behavior, which we talked about in quite depth already with evals and enforcement, but how are the infrastructure components dealing with agents? What network accesses do these agents have? What are the security layers that we can put on-- because without observability, you can't have security. And so if we want to run agents, whether it's in a desk-side computer or it's in a server rack or it's in the cloud, without end-to-end observability, we can't enforce the right guardrails around the agentic processes. And the final piece is tokenomics, where tokenomics is actually bubbling up quite a bit now as something that's of importance. But we've always tied the end outcome, the ROI of agents to the spend that we're making. When we talk to customers in the past in the first couple of years post-ChatGPT, we've had so many dinner table conversations with CIOs and saying, hey, I spent $15 million on LLMs. What's the ROI? Are we shipping more product? Are we actually moving the needle on our revenue? And the answer has been no in the past. But now I'm starting to see that change, where there is measurable revenue. There is measurable impact. There's measurable productivity. But the thing that's missing is the infrastructure to measure that itself. How can we quantify and tie the agent's behavior and the tokens spent to the business impact or the ROI.
To the value that you are going to receive.
Because what leaders want to actually do is we don't want to spend less on AI. We want to spend more on the right pieces that are giving the right ROI. There are a thousand things going on with AI. There's a lot of tinkering going on.
So give me a concrete example that a customer can say, if I have these class of problems, I should be thinking about observability for AI with Splunk.
So there are three classes of problems. When we think about who needs observability for AI, most enterprises today are having some form of agentic behavior in their workflows. These are tools and productivity tools and coding agents for their developers. That's the biggest spend today, and that's the biggest amount of agentic activity. That being said, the first-party and third-party agents, those are the other two categories, where first-party agents are the ones that enterprises build themselves. And that could be around their internal workflows, but also consumer-facing products. And third-party agents are the entire agent sprawl we are seeing in every SaaS provider having their own agents.
And you want to make sure that you provide observability for the entirety of the estate of all agents.
It has to be. We cannot just have blind spots around, oh, I measure my first-party agents, but what about the rest? But the point I was trying to make is the agentic behavior in products and services is only growing. And today is just the early days. As more and more workflows in our tools, in our products become agentic, the spend and the observability needs for agentic on first party, third party will increase. But today, as we stand, we cannot ignore the coding agents and the tools that vertical as well.
So I'll come to you, Atin, on the product side of the house because there are so many products that actually use these terms that sometimes it's hard for a customer to say, I can't tell the difference. So here's my question to you. If I'm a customer at a bank or an insurance company or a pharmaceutical company, pick an industry or a vertical, what is a concrete example where what you do and what we do with Splunk Observability for AI can meaningfully change the game on the kind of value they harness from AI?
Absolutely. I'll give you a concrete example, say, of a healthcare company. They're building workflows to make critical decisions on, say, prior authorizations of expensive drugs. They're using GenAI to solve it, and they're building agents to automate this whole gnarly process. And it's ripe for disruption because these workflows are just filled with unstructured data, PDFs, documents. And LLMs are-- for the first time, we have technology that can make sense of it and take actions for it. So we have the ingredients. Once you build an agentic workflow, a whole host of things can go wrong at many layers, including the three layers that you mentioned. I'll talk about the first layer, which is tokenomics, and second layer, which is the observer.
The behavior. Starting with the behavior, one wrong mistake that the agent makes because of a loose prompt or a wrong instruction can lead to a bad action, which in this case would be the denial of a drug for a patient in need. So the results can be catastrophic because of wrong behavior. Then there's the cost aspect to this where tokenomics comes in. And I'll tell you how layer 2 and layer 3 are related. There's the absolute measurement of tokens that will be needed to take this action, including parsing all the PDFs, the docs, the insurance docs. All this requires tokens, and we can't escape that. But the question is, how do you do it efficiently? Because these models are probabilistic. They can go wrong, and they can spin into loops. So it can be very expensive to get this action done. So the problem to solve is, one, measure the total number of tokens it took to take the action. And two, whether the action was taken or not, which is also a key part of tokenomics, but the insight and the signal for whether the action was done right or wrong comes from the layer below. So that's a--
OK, so I measure the key actions taken with the use of tokens. What do I do if I find out that-- what's a good scenario and a bad scenario in that particular instance? And what does our technology help with on that front?
A good scenario is that there's action completion happening, which is the agents are doing yep. What the instructor action is, which is to approve or deny the drug and do it with high saliency or high accuracy. The bad scenario is, of course, doing the wrong action. But the bad scenario could also be your rewrite action.
Very consumptive on tokens, while you're taking--
Extremely consumptive, extremely slow, because whether it's a healthcare workflow or an e-commerce application that can deliver goods to you and issue refunds, the architecture of agents behind the scenes is the same. And they have the same proclivity to the errors that they can make.
So why did you folks feel like being part of Splunk would actually be additive to your mission? What was it that Splunk brings to the table that you felt like you didn't have?
I think to my point earlier, the thing that we're super excited about is that Cisco is building the critical infrastructure for the AI era. Splunk, as a business overall, is looking at solving the digital resiliency problem. And agentic or AI resiliency is the next chapter in this entire business. We also heard from our largest customers, some of the largest Fortune 100 customers, that they have an entire digital resiliency stack, whether for application monitoring, for enterprise security. And then as more and more services start becoming agentic, they asked us as, how is your platform going to tackle all of it? Because we want the entire digital resiliency to be in one end-to-end flow, in one end-to-end solution. And so from a product and technology perspective, what we've built is extremely additive as well as strategic from Splunk's customers, as well as Galileo's customers.
This is truly a full-stack observability solution from infrastructure observability, application observability--
And then going into agent behavior, tokenomics and pulling it all together in a way that's one cohesive interface in Cisco Cloud Control.
Absolutely. And apart from Splunk's capabilities also, Cisco has the AI Defense aspects, the DefenseClaw and AI Defense pieces that actually fit in very well with the story, as well as ThousandEyes and security solutions. With all of these capabilities, Cisco can truly provide the solution for critical infrastructure for AI agents. I'm not saying that lightly because one bespoke point in time solution is not going to solve the overall problem to build. And all of this allows us to build the fabric for digital resiliency, agent resiliency across agent software.
And whether you take the example that took Fable down or the healthcare example I just gave, these are anecdotal examples of singular workflows that, of course, have the same failure patterns. But really, when you imagine an agentic fleet running in production, that happens across multiple devices, multiple data centers, multiple availability zones in the cloud. And to detect a security issue in an agentic flow, the data can come from anywhere. So providing, number 1, the unified data fabric to be able to-- a unified interface across all your data. You need that to solve the security problem. You need observability on all this data to solve the security problem. It all blends into one another. So having this full stack really is saying that I have the key ingredients to solve agent reliability. And Cisco, from what I know AI is the only company in the world that has that.
And to your point, Cloud Control stitches it all together. And we're super excited about where we're going with the direction. Every product in Cisco can be operated--
