Everybody, welcome back to Moscone West. You're watching theCUBE's live coverage of AMD's Advancing AI 2026. I'm Dave Vellante. We have a special guest here in our live coverage. G2 Patel is the president and chief product officer of Cisco. Awesome seeing you. Thanks for jumping in. I really appreciate it.
So great to see you. It's the best ones are the spontaneous ones, man.
Yo, you got it. Well, first of all, Cisco's on a roll. I mean, you know, we have seen you really focus the team. You're a builder getting the product really to where it needs to be and so congratulations on that. Give us the quick update and I want to get into the news.
Yeah, I mean the update is I think we're in a quiet period but last quarter was fantastic. I think the general momentum of innovation has never been better than what it has been over the course of the past two years. I'd say in the past two years we invented more than we had in the previous 20 years combined and I think that's going to dwarf in comparison to what we have in store in the next couple years. So I think the team's excited. They're kind of, you know, the only thing is we're not getting enough sleep. So like we got every once in a while like take a Saturday off or something.
Well, and I know you're in a quiet period, so you can't say much, but I mean, Lisa laid out the sort of the TAM, the growth, the demand, where the demand is coming from, claiming that this is really the first year that inferencing is sucking up more compute than training, which we've been saying for a long time. It's finally happening, but there doesn't seem to be any end in sight to the demand. Certainly, the memory shortage looks like it's going to last through '27 and beyond.
There's less than 1% of the world that's using agents, right? And if you believe that agents are actually a step function improvement from a chatbot where you can actually have either your personal productivity or your company's productivity materially change as a result of agents, I don't see how you don't stay in continued kind of supply shortage for a pretty long time period. And that's what I think we're seeing that play out.
Yeah, the signals are very positive. We saw, I mean notwithstanding IBM's disappointment but that's really not, I don't think they're a bellwether for AI. I mean super micro doubling its margins and obviously you know growing, you're going to probably see something similar from Dell, you're going to see something similar from Nvidia and clearly AMD is on a rocket ship. You got an announcement here that you were up on stage, we're talking about distributed inference. Take us through the news here.
So basically what you're starting to see happen right now, Dave, is the pattern of inferencing is fundamentally changing as you move from chatbots to agents. So in the chatbot era, when humans were the ones that were driving the engagement, you ask a question, you get an answer. The patterns of inferencing were very spiky. What you're now starting to see in the agentic era is these agents are working 7 by 24 on your behalf. They might actually go out and talk to other agents. And so the pattern of inferencing is much more durable and sustained and not as spiky. And it's actually the volume is higher. And so what you're starting to see is every customer I talk to right now has a couple major concerns. Hey, I'm concerned about security and safety and I'm concerned about the cost of tokens because these tokens can start to run up in cost and you might not have had enough time to go out and budget for all of those. So we need to make sure that as an industry we lower the cost of tokens quite dramatically. And so one way that that's happening is you're not just going to have inferencing run in the cloud. You'll also see a whole new set of models that emerge for inferencing running everywhere and locally. And one of those kind of patterns is a whole new class of computing that's emerging called deskside computing. So you work on your laptop right now and that's a human working on a laptop but you might have a deskside computer where the agents are running. And by the way this isn't really a new phenomenon. It's just taking more effect right now.
I do have that by the way.
We do have that and if you remember during when OpenClaw first came out Mac Mini sold out. Right.
And you're starting to see that same pattern which is like there's going to be more inferencing that happens on the desk side. Now, by the way, if you have enterprises that start to go out and deploy this, you can't just deploy and pray that everything works out right. You need to make sure that it's managed effectively. You need to make sure that it's secure. You need to make sure that the agent behavior is monitored. You need to make sure that you figure out whether the cost of tokens is out of hand and if so, what do you need to do to make sure that you can contain the cost of tokens? All of that apparatus needs to be put in place. So what we have done is we have our unified management plane that you know well, Cisco Cloud Control, and what it does is it can go out and look at all of your inferencing capacity that you have in the cloud, all of the inferencing capacity you might have in a private data center as well as your desk side computers in a single management plane and then be able to say well what do we need to do to ensure that you have the best cost efficiency and performance and security when you're running in this hybrid inference model which is what I think is going to be the prominent model as we move forward.
So you're doing the routing there.
So in this particular case AMD does the specific intelligent routing of the models. They have the actual compute. We provide the network bandwidth that's going to be needed. We provide the security and safety that's needed. We provide agent behavior monitoring so that if an agent's starting to go rogue, you can actually make sure that you can sequester an entire Halo machine. And we provide token economics so you can figure out how many tokens are being used. Is it more efficient than running it in the cloud? What do you need, which agents are using, which amounts of tokens, is that agent starting to show patterns of excessive token usage? If so, do you want to contain it? All of that kind of telemetry is provided in Cloud Control.
You guys provide that telemetry, that observability, that's fundamental to what you guys do there.
Okay. So we have early customers using it right now. But you know the Cloud Control is going to go general availability in the US in early fall and so we're excited about how customers can lower their cost, improve their efficiency for tokens.
It's interesting. I mean during the pandemic everybody was working remote. They just threw everything to the cloud. There was spend, spend, spend. 2022 comes around, we're coming out of the pandemic and cost optimization was the big theme. Now you're seeing that, you know, token cost because the volumes are higher, right? And the numbers are getting bigger. So most companies don't have a few hundred million just lying around that they can go out and start spending on tokens. They have to plan for it. But yeah, at the same time, we've seen token costs come down 90% a year. I know there's a big delta between, you know, the most expensive tokens that you're getting through APIs in the cloud from the frontier models and some of the Chinese open models. That's like orders of magnitude delta. What do you make of that? I mean ultimately is this a short-term trend with this token cost optimization or do you see this, I mean you talk about it, Ali Ghodsi is really pounding the table on this. Do you see that as an ongoing concern or ultimately are tokens going to be essentially free?
The way I see it is it's an ongoing way that we will operate. Just like we have a finite set of resources with humans and we have to plan out what those humans are going to be able to do, you're going to have a finite set of resources with tokens and you're going to need to make sure that you can put those tokens to the best use so you can get the best outcomes for your organization. And so that's what's going to happen. And I think the reason that I don't think it's just going to be a short-term trend is because yes, you might have seen the cost of tokens go down by 90%. But the demand signal of how consumptive these agents are on infrastructure has gone up way faster than that. So your total cost is still going up within the organization and that's going to be a finite bucket and we need to make sure that that finite bucket is being applied in the best way possible.
So I want to run something by you sort of the macro. If you look at spending on IT broadly across the world, the organizations spend about four or five percent of revenue on IT. We forecasted that's going to double or triple by mid next decade. Why? Because you're essentially going to be scaling without the labor that you used to have to require to scale. We're not talking about wiping out jobs. That's really not the point. The point is you're going to shift investments to tokens whether it's through APIs or you're generating, you know, spending on capex on your own. And that is going to be where the ROI is. You're not seeing that big trend today yet. You're seeing the token cost. You're not seeing the productivity hits just yet. We're at the early days. But do you buy that premise that over time you're going to be able to scale dramatically without as much labor and that's going to justify that doubling or tripling of that revenue going to technology?
So I have a slightly counterintuitive perspective on this one which is that as AI is starting to advance and automate workflows and do things that are in a much more effective way than what was done in the past, more human bottlenecks are emerging than they were in the past. So I actually think you're going to see more jobs as a result of AI rather than less jobs. The only difference is the dexterity for AI is going to be needed in every human being. So anyone who's actually not AI fluent is going to have a harder time. But the people that are actually fluent in AI today, they're in more demand. They're making more money. They're busier than ever. And you're seeing this across the board. So I think you're going to actually see that being that so that's one dimension. The second dimension is we have overly focused just on productivity. I think we need to focus on what class of problems could we never even dream of solving before that we can solve as a result of AI. And I think that's a far more important question than just pondering what can we do to go out and reduce the costs for humans.
Yeah. So by the way I would agree with that. I think we're going to have more jobs. My guess my point is and I want to test this on you is we're going to be able to scale dramatically faster than we would in the past with a lot less labor. That labor is going to have tools and be more intelligent per unit of output. You might have less labor, but you will have more jobs than what you had before.
I would totally agree with that. And when I talk about productivity, I have a very simple measure of productivity. Revenue per employee, right? And so you're going to have much more revenue per but you look at what Uber has done now. Of course they have a different business model where they're
Okay I got to ask you it's interesting you said about only 1% of us are using agents, you know building agents and using agents. We're in this bubble right and so
