Hello, everyone. Welcome to Cisco's Inside Product podcast. Today, I have with me Guru Shenoy. Guru is our product leader for hyperscalers, neoclouds, sovereign clouds, service providers. It's a big job, my friend. Welcome.
Thank you, Jeetu. Yeah. Pleasure to be here.
Yeah. And so we're going to talk a lot about what's happening in the industry with AI, with data center buildouts in these four kind of customer constituencies that you have, specifically in data center networking.
And so walk us through what's happening. Set the stage for us. Why is it important for what's happening, specifically in service providers right now? Because that's actually a huge area of growth, in addition to hyperscalers. And so what's happening over there, and why do you think it's seeing a resurgence?
Yeah. No, look, we are undergoing a massive transformation, as we know, with AI. And it's having major implications on traffic. And if you look at where the last three years or so have been, a lot of the energy around AI was around training, mainly the hyperscalers, building big data center networks. Now, what we are starting to see--
Which we were involved in.
We've been heavily involved in that, helping those buildouts and investing in technology and so forth. But now we're starting to see the consumption of AI, if you will, inferencing, the agentic economy arising. And so enterprises, consumers are all starting to use AI. And so all of that AI is now starting to get delivered, most of it is over telco networks, service provider networks. And so we are starting to see massive shifts in traffic, not just volume, but changes in the patterns of traffic. And so service providers, they have a huge opportunity here because they are in the middle of this. They're the backbone of the AI ecosystem, as we like to say. And so delivering intelligent fabrics to serve these new AI workloads, participating more actively, offering potentially things like inferencing cloud. So there's huge opportunities for service providers. And that's what's causing a resurgence because, post 5G, this is the next big transformation. And this is a durable one that we see. So there's a lot of energy amongst the service provider community--
So you don't believe this is a bubble which is going to go away at some point because--
Absolutely not. I'm 100% convinced that this is a durable trend because we are seeing value. We're seeing products and services that are actually adding a lot of value. That was hard to see in the 5G era. 5G was all about, hey, we need better latency. But no applications ever arose to fulfill that need, if you will. But here, it's a different ballgame altogether.
The demand had to follow the supply on that one, whereas over here, the demand is almost instantly serviced as a result of the supply. And we are in a supply-constrained world right now.
Big time, yeah. We're not able to keep up just in terms of the infrastructure that needs to be built out to deliver all of these new capabilities that the agentic world is demanding.
OK, so I want to talk about each one of these four markets real quickly. But let's start with the strategic shift you said is, hey, the amount of volume of traffic is going up exponentially. Give us some stats on the volumes of traffic that are going up.
Yeah, so it's interesting because very recently, just a month ago, we published our AI impact on the WAN report. And it's the first of its kind. What we actually did is we didn't just look at third-party sources and so forth, but we actually ran measurements with sensors in some of our service provider customers to identify what does AI traffic look like on the internet today. How is it growing? What is the nature of traffic distribution and patterns? And this is a public report that we've shared. And there's some really interesting stats. First of all, we're starting from a very low base because inferencing is only now starting to take off. But the growth rate is phenomenal. We're seeing 10x year-over-year inferencing growth rates. And obviously, at some point, it's going to slow down a little bit. But it's going to go on for quite a while.
I actually don't think it slows down because you've got about 0.1% of the world that's using agents right now.
So you're only going to see that compound over time. It's not going to slow down at all.
You're right about that. The slowdown will happen many years. And by that time, it's going to be a significant portion of the overall traffic. And we haven't even started seeing AI in a lot of video yet. Once that happens, we're going to see even more. But we are seeing this incredible growth rate in inferencing traffic, 10x year over year. Moreover, we characterized how does this traffic behave, because now we have agents that are actually driving. And it's things like people using Codex or Claude Code and all of these capabilities. That's one of the big contributors to the AI traffic. And we found that agents actually generate 450% as much traffic as humans do when they perform web transactions. And that's simply because they can do things so fast, they need a lot more context to reason and so forth. So we see this clear sign. Moreover, there's some interesting characteristics about the traffic distribution itself. There's a lot more upload that happens with AI traffic instead of just...
Because your markdown file, your skills file, your memory file have to keep getting uploaded.
Exactly. You need to send it to your inferencing model. And then that needs to be processed as context. And then it needs to respond to it. And then imagine, as we start looking ahead, we are already starting to see signs of physical AI, with a lot more robots and 3D spatial sensing and so forth. That's going to introduce a lot more video in the mix. So you'll have a lot more video upload, as well, to your closest inferencing location. A lot of it will happen on-prem, but some of it will go over a service provider's metro network into a locoregional inferencing cloud, and so forth. So upload is a lot more. Traffic tends to be a lot more distributed, as well, in the sense that these agents-- and imagine a world where you have millions, if not billions, of agents. Every enterprise application agentic-enabled, every consumer application agentic-enabled. All of these are talking to inferencing cloud endpoints. They don't just talk to one cloud location to download data as we might...
It's very chatty. It goes across multiple areas.
Very chatty. So the net effect of all of this is that the network over which all of this is serviced, a service provider's network, needs to adapt significantly to handle that traffic load, to handle the multidirectionality of traffic. A lot of it will be latency sensitive. We don't think latency is going to be the only factor, like everybody thought in the 5G age, but--
Basically, latency, performance and power will be the three, right?
Those are going to be the three primary ones. Absolutely.
So Guru, one other question that comes up a lot, especially in data center buildouts, is around energy efficiency, because if you think about it, we just don't have enough power, compute and network bandwidth in the world to satiate the needs of AI. And that's been a pretty commonly known fact at this point.
We're doing a lot on making sure that these infrastructure elements are built in a sustainable way and that we actually have very, very high efficiency on power consumption. Talk to us a little bit about that. What's happening? What are you doing? And give us some stats.
Yeah. So power has been probably the number one constraint when we talk about data center buildouts. Now we're starting to see some others, but power continues to be one of the key ones. And so the desire to build extremely power-efficient network infrastructure and compute infrastructure has been a very high priority for data center builders. So what we have done from our side is it begins from the silicon itself, building extremely power-efficient silicon. And this is what we pride ourselves upon when we built Silicon One, making sure that the silicon itself is super power-efficient, but then also all of the software that goes on top of the silicon and the systems. The nature of systems we build, because we are now building extremely high-capacity silicon, we have the ability to deliver in small, fixed form factor boxes one, two, three RU that would earlier take big modular chassis to do, 50 terabits of routing and more. That's a huge difference, because a chassis, with all the fabric cards and line cards and everything--
The size matters on power consumption.
Big time. And it's 8, 9, 10 kilowatts for the chassis, and we can deliver that in less than a couple kilowatts when we talk about fixed systems. So there's a lot of innovation there. But that's not where it stops. That's actually only the start, because we have optics and optical that is being integrated more and more.
One of the big things that we've been pushing for a couple of years, and now the entire industry has really started embracing it, is something we call routed optical networking. And that is this idea that you bring together IP and optical layers together so we have coherent pluggables sitting in the routers. And that means you're eliminating an entire layer of hardware--
And coherent pluggables are the ones that go data center--
Interconnect. And so you have these-- rather than having the big transponders, you can just actually--
--have a bunch of pluggables within your--
We can drive distances of hundreds and thousands of kilometers just with these little pluggables that sit inside the router. And so it's 120 kilowatts in a transponder card versus 23 kilowatts sitting in the router. Huge power savings when you put these coherent transponders in. So that architecture, and then more and more optical integration into our systems is something that we are really focused on. And then additional things, now we have already started building liquid-cooled systems, highly efficient liquid-cooled systems in our 50T generation, 100T generation, and so forth. So there's been a huge amount of investment in just the core technology needed to support infrastructure buildout.
And then for the likes of neocloud, sovereign cloud builders, who also buy software from us, the software that's integrated on those systems, we have capabilities in our software, as well, for intelligent power optimization, fan control, all kinds of things designed to make sure that we keep the power profile, the real-time consumption of power as low as possible.
So yeah, this is just scratching the surface. There's a lot we're doing here.
And then I'm assuming that because the advanced packaging of the chips, where we also make our own SerDes and kind of high-speed IP, that actually really helps with power efficiency, as well.
Absolutely. So if you look at what goes inside a chip, you have the packet processors, you have the I/O, you have the SerDes, you have the buffers. So because we control the entire technology stack there, the packaging and the design, we can bake in power optimization and sustainability--
As a core characteristic.
--as core characteristics of those designs.
Yeah. And then there's a whole discussion around how can service providers be more effective participants in intelligent token delivery, not just over-the-top token delivery. So there's a lot of opportunities to rethink how network infrastructure behaves.
The way I try to talk to-- the way I think about the business that service providers are in, because a lot of times, service providers will ask, hey, what do you think we should be doing--
--I think you have to be in the path of token generation.
And if you're just in the consumption point of token generation, it's marginally interesting. But if you're in the path of token generation-- by the way, the amount of kind of infrastructure that they had laid out with 5G is actually-- this might be the mechanism for them to go out and monetize all of that, that they haven't been able to fully.
A lot of the infrastructure-- that's a very good point, actually, because-- both great points. One, how can they be in the middle of token generation and intelligent token delivery, and not just be providers of over-the-top tokens. But also the infrastructure to deliver all of this, by definition, is very distributed.
Because unlike training clouds, which are a few hyperscalers, big data centers, gigawatts, the inferencing ecosystem is all kilowatts and megawatts of capacity, from a power perspective. And telcos have already built a very rich set of distributed locations, points of presence or PoPs, as they call them, where they have just the right amount of space, power-- 20 megawatts is plenty-- and the fiber to deliver all of this. So you're right that they can actually realize a lot of the investment--
Capex has already been spent--
--can actually be monetized now.
Can actually be monetized now.
OK, so now let's talk about Cisco and what Cisco can do to help. And let's talk about each one of the class of customers. Let's start with hyperscalers, because they're the most sophisticated on the kind of things that they're doing at scale. And then we'll go hyperscalers, neoclouds--
--sovereign clouds, and then service providers. So let's start with hyperscalers. Tell me why hyperscalers are coming to us. Tell me why we've actually gone from what was virtually, for AI workloads, almost zero in orders taken three years ago, to we thought we were going to do a billion last year. We ended up doing 2.3.
We thought we were going to double that this year at five. We're going to end up doing close to nine.
Why that level of surge, and what is happening over there? And why is Cisco uniquely differentiated?
Yeah, no, a lot of credit to our engineering teams, to our sales teams, just a whole cross-functional set of teams for having pulled that off. But if you think about what hyperscalers are looking for, they build-- first of all, technology is paramount, making sure that you have innovative technology, leading technology. For example, they're looking for 50 terabytes, 100 terabytes, which is beyond, advanced optical -
One thing I've actually told people is you should expect, over the course of the next few years, we should have a one-petabyte switch--
--with a trillion transistors.
It's not unrealistic at all because we are already in the 100T era. And as you can imagine, we are working with our customers two or three generations beyond.
So having the right ingredients for delivering technology-- silicon, optics, liquid cooling, various kinds of optical packaging-- NPO, CPO, XPO, all of these-- we have all of those assets. We have our own in-house silicon with Cisco Silicon One, which has been a phenomenal investment, and it's really paying off now.
We started that a decade ago. And it seems--
--like a genius move now, but--
I take no credit for it, but I think it was-- you have to thank all the people that made that happen.
Two of the things that have worked out really well for us, Silicon One, and then our Acacia acquisition gives us the capability to bring coherent optics technologies. Not just pluggables, but a lot of those technologies are now coming together. Silicon and optics are coming together. So technology is a big piece of it. We have that. We are also able to build the kinds of systems that hyperscalers want, because a lot of them want very bespoke systems. They co-design with you. So you need to have those skill sets and those abilities. We have that. So we're able to deliver that to them. They look for diversity of supply chain, of technology. We have that. Silicon One and our optics is something we build. We own the entire supply chain right down to buying wafers from TSMC. So that is an advantage. We also engage--
Yeah, we're not just building ASICs. We have a full customer-owned tooling and customer-owned toolkit that goes direct to TSMC, so --
And then, of course, we engage with them on a variety of different business models. They buy chips from us. They buy systems from us. We give them software support for what they want. Some of them put SONiC, others put FBOSS. So we are able to engage with them in a variety of different ways. And so the combination of technology, the ability to engage the way they want us to engage, co-designing, just the supply chain, not to be underestimated, the value of our supply chain--
--all of these factors together have been very useful.
And scale really matters on that dimension.
