The Critical Infrastructure Behind the AI Boom | Cisco’s Jeetu Patel

The adversaries and the threat actors that cause cybersecurity attacks or cyber attacks have the exact same tools now that you and I do, Connor. And they can use those tools to get far more sophisticated and far more at scale in systematically attacking society and the critical infrastructure of society.

It's such a pleasure to be on your show, and I love the name of your podcast. Chain of Thought. That's so, so creative.

I believe in that because I feel like we should not underestimate the human mind. You know, like we've — thousands of years have gone by. There's been a bunch of technology shifts that have happened, and humans have continued to find a way to stay relevant during each one of these kind of disruptions. And, you know, if you think about what are humans really good at doing? Humans are amazing at gut instinct, intuition, ability to be able to make decisions with incomplete data, having empathy. These are all things that are really hard to have a machine get trained on. And it's a very interesting industry strategy that had transpired where the industry took an intentional, you know, approach of making AI very skeuomorphic in nature, right? And so it was like every single thing was equated to a human rather than being equated to tooling that humans use. And I think, like, you know, there's a very eloquent argument that Jensen has that, hey, this is not — this is not a tool, it's work. And I think that's — I actually like the way that he frames that, but I do feel like level one maturity that we would have is AI is going to take our jobs of thinking — like just pure maturity of thinking spectrum. Level one maturity is AI is going to take our job. Level two maturity is someone that uses AI better than me is more likely to take my job than AI taking my job.

That is a real risk, right? And then there's mitigations to that risk that we should talk about. And then level three maturity is when people realize in the next few years it's going to be really hard to imagine me doing my job or me getting my job done without AI. And this is not even a crazy concept. Like, the spreadsheets came out and everyone thought that every finance person was going to lose their job, and now you can't get a finance job without understanding the spreadsheet. And I don't think that AI is going to be any different on that dimension. Like, we will not be able to get our jobs done without AI. And so then the question is, what is AI good at, right? Like, one way to think about this is there are a bunch of jobs that we just don't have time to do that we can actually perform out to AI to get done. That's going to be the first category of jobs we'll give to AI. The second thing is there's going to be a bunch of things that AI is better than humans at doing, you know, which is like, you know, crunching data of large volumes, making sure that you can actually aggregate the insight from that. AI is going to be better at doing that than a human would do it. And then the third one is jobs that humans can't do that only AI can do. And so I feel like there's on each side humans are going to be good at a lot of things, AI is going to be good at a lot of things, and when you put them together, that's when the magic happens. And I think it's a false narrative to say humans are going to be — this is the end of creative contribution of humanity in society. We're going to be sitting on a beach staring at the ocean and that's all we're going to be doing for the rest of life, and AI is going to do everything for us. I just feel like that's jumping the gun a little too much. And so I find it to be a very kind of optimistic future from the dimension of — and I'm not delusional about it. I think there's risks to AI, but the risk of all jobs going away, I think, is diminutive. Now, what I think will happen is some jobs will go away, all jobs will get reconfigured, and entirely new industries will get created. And we should think about that framing as we go into this market. And I think it'd be a far more productive framing rather than the nervous frenzy of everyone worrying that AI is going to have the entire corpus of humanity not be able to be employable for the rest of time. I think that seems exaggerated to me.

I think the compression of time at which you have to get — you know, the half-life of anything is compressing quite dramatically. And that can cause nervous energy, and that's actually very, very kind of legitimate. On the other hand, I don't think the storage of knowledge in our heads is going to be the currency of the future. And so if knowledge and intelligence is abundantly available, what's going to be the unique nuance that can differentiate one human from the other is how can you make sure that you get that knowledge extracted from AI with the right questions. And so asking the right questions, learning to learn fast, those are skills that are going to be far more important for my 14-year-old daughter to learn than just, you know, cramming a bunch of knowledge and memorizing it, which is something that I feel like, you know, the internet was the first phase of that, but now it's like the AI is a second phase. And the thing that I think is grossly underestimated about AI — and this is an area that I don't think people are talking about enough and they're not thinking about enough — is right now we think of AI as an aggregation mechanism for the corpus of data that's available in the world. So it's like, I'm going to go out and train my models with all the data that's available, and then I'm going to be able to get a succinct answer for a question I ask based on the data that's all available. I think that — and that's going to get me productive. That's the current kind of simplistic promise that's delivered. I feel like that's like 1% of the benefit. 99% of the benefits are going to come from original insights that will get generated that don't exist in the human corpus of knowledge that allow us to dream about solving problems that we had deemed unsolvable up until now. And that will then give us a whole new set of possibilities on things we can do for tackling important issues in society: you know, longevity of life, quality of life, curing of disease, you know, ending poverty, you know, kind of helping cure the climate crisis. All of these are kind of going to be done far better with AI coming up with original insights because humans had not thought of it. And that to me is an exciting future that all of us should be stoked about rather than being nervous of, because we'll be able to solve problems. Now, what should we be nervous of? What's worth being nervous about is — you've got, you know, like the adversaries and the threat actors that cause cybersecurity attacks or cyber attacks have the exact same tools now that you and I do, Connor. And they can use those tools to get far more sophisticated and far more at scale in systematically attacking society and the critical infrastructure of society. And so what we will have to do is make sure that we completely accept the fact that safety and security is a big risk, and we have to mitigate that risk because in the absence of that getting mitigated, I think there'd be a lot of harm that's caused in humanity.

So in the land of security, let me actually zoom out for a second and say, in this world of AI, what are the impediments that could actually hold AI back? And I think there's three big areas that could hold AI back. There's more than three, but there's three that we can — at Cisco we can squarely actually make sure that we can make a positive contribution to. The first one is an infrastructure constraint. There is just not enough power, compute, network bandwidth, and data center capacity to go satiate the needs of AI. And the constraint for infrastructure is directly correlated to a company and a country's ability to have intelligence. Infrastructure is the input. Intelligence is the output. And by the way, all things being equal, any company or any country is going to be better with more intelligence than less intelligence. Everyone needs more intelligence. So the first constraint is infrastructure. The second big constraint is a trust deficit. And what I mean by a trust deficit is these systems that AI is built on, or these models, are by definition non-deterministic in nature, which means that they're unpredictable. But we are building highly predictable applications, or we're wanting to build highly predictable applications on top of a foundation which is by definition unpredictable. And so we have to make sure that we can solve that problem effectively by saying, do I have visibility on what data is going in? Can I actually jailbreak this model algorithmically through a red teaming process that's algorithmic, not human-based? And once I've found a way to jailbreak a model because it doesn't work the way that I expected it to work in certain categories, in certain areas, I should dynamically at runtime be able to have runtime enforcement guardrails be put on it. And if I can do that, then anyone who's a developer, anyone who is building an agent, anyone who's building an application on top of a model is by definition going to not have to worry about building a security stack because all they'll have to worry about is coming up with the next best idea, and we will take care of the security because we will actually do that piece of it for them. So you can innovate fearlessly. So that's the second big constraint is a trust deficit. If you don't have safety and security right, people are not going to use the system. So safety and security become a prerequisite for adoption. And then the third area is a data gap, where right now, if you think about it, most models — the issue that people had with scaling laws was they thought that at some point in time we're going to run out of data, which was actually true. You're running out of human-generated data publicly available on the internet. However, there is plenty of evidence now that's starting to build up that synthetic data is starting to show very efficacious kind of outcomes for AI models, right?

And synthetic. So that's great. And then the second thing that's happening is machine data, which is data from applications and agents. And the more you have — for every human, if you have 10 agents or 100 agents, the amount of machine data on activity that gets generated in logs, metrics, events, traces, all of those things is currently time series data that is not effectively utilized by AI models for doing things like detecting infrastructure stability, predicting infrastructure outage, you know, making sure that you can prevent a breach from occurring. All of those things is where machine data can be hugely beneficial. And so these three constraints — infrastructure, trust deficit, and a data gap — are all three, it turns out, that Cisco can be squarely in the middle of and really help our customers out. And those customers could be public sector, they could be private sector. And essentially what it's doing is it's helping our customers generate tokens with the lowest amount of kilowatt power of energy and the lowest amount of dollars spent, because token generation ability is directly proportionate to a country and their ability to actually have economic prosperity as well as national security. And that same applies even for companies, where companies can be more financially viable and have greater security posture. So I feel like those three dimensions are pretty important to understand and say, what does the market need to do to ensure that we can actually drive these constraints to a point of, you know, where it does not become a constraint but it becomes a state of abundance? Because if you have plenty of trust, and if you have no infrastructure constraint, and if you don't have a data gap, then the potential of AI to alter the course of humanity for the positive and for the good is going to be meaningful, and that's the future that we should be striving for. And by the way, that future does not happen without humans. That future requires humans to be participating.

Yeah. So on the AI Defense side, just to close out on that, basically what it is is a mechanism for us making sure that we can get visibility on what data is flowing through a model, do algorithmic red teaming and jailbreak a model using AI Defense APIs, and then make sure that you can runtime enforcement apply guardrails. That's AI Defense, right? So think of it like almost like a very sophisticated version of an AI firewall. Going back to your model question, in the consumer world, I think you're going to see a few handful of foundation models be very successful: OpenAI, Gemini, XAI, you know, Anthropic in certain instances — they will all be very successful models. My guess is a few more will emerge, but you're not going to have thousands of models on the consumer side. You will probably have a dozen that'll actually have most of the volume of inference. On the enterprise side, I think it's going to be very different. You're going to have thousands of models. You might have tens of thousands of models. And what you will have is multiple models being used in conjunction with one another with an intelligent routing layer that allows you to make sure that you can optimize your COGS and you can make sure that certain queries go to certain models because that's costing you less from a compute resource standpoint, and then other queries might go to other models. If you look at Cursor, my guess is — I don't know for a fact — but 60, 70% of the queries now are not going to Anthropic. They might be going to their own models, and then they might have a certain percentage that goes to Claude. Why is that important? Because as you have more and more of these models, it's going to get not only more cost-efficient, but your efficacy of the model being able to, in a very small footprint, be able to give you exactly what you're looking for in a particular domain is going to be much higher.

That's right. That's exactly right. So what we discovered this pretty early, and we said, hey, look, security is an area that we care about deeply. We want to make sure that we build — wouldn't it make sense for us to build our own models in security because we've got so much data? And the key over here is not to just make sure that you have infinite amounts of data. The key is to make sure that you train it on the right data, you know, and so distill it down to just the right amount of data that you want to train it on where the efficacy of pre-training a model that is a, you know, open-source 8-billion-parameter model, we are seeing can now actually perform better than a 70-billion-parameter model. And what we're also seeing is that 8-billion-parameter model can be further quantized and you can run it on a CPU on a laptop. And it actually just has a very, very different kind of economic outcome. And so that in my mind not only does it solve the trust deficit, but it also solves the infrastructure constraint, right? And so then what we're doing is we're building out products for the edge that says, okay, if you have robotics on the edge, you might need to have inferencing on the edge, to your point, because of latency. And so then we build this thing, this platform called Cisco Unified Edge, which allows you to have networking, security, and compute bundled in a box that if it's a branch office, or if it's a hospital, or if it's a factory floor where you don't have IT staff, you can still just plug this box in and it doesn't have wires dangling. You don't have to go out and have someone special come in to get it installed. You can manage it centrally. It's just inference plug-and-play on the edge so that you have the full from core to the edge inferencing capability provided by Cisco. So those are the kind of things that we've actually been spending our time building because we feel like you have to build things where there is a true pain point and a challenge. And right now the challenges in the market tend to be around infrastructure, trust, and data. And so what we do is we take that as a core founding principle of our product ideation process and say, if we solve those things and make those abundant as a result of doing this, there's going to be a natural demand signal for it and we'll end up doing well. And we recently had our earnings for the quarter and, you know, we had a very, very successful earnings because you're starting to see that hyperscalers and, you know, neoclouds and sovereign clouds and service providers and enterprises alike are all starting to find that, hey, it makes a lot of sense to partner with Cisco because they provide the critical infrastructure for the AI era.

I think in my mind a partnership — a company's willingness to partner is very indicative of the company's arrogance level. And the reason I say that is you can't be arrogant to think that you're going to go out and build every single thing that's needed for a very, very large growing economy and the largest platform shift. And that means that you have to have enough humility to know that there will be overlaps between your partners. Sometimes they might be competitive in nature compared to what you might have wanted to do in an ideal world, but it is better to partner with the ecosystem and create an open ecosystem than to create a walled garden. And so that's something that we've actually thought pretty deeply about. And when I joined five years ago, one of my big kind of convictions over here was that we have to be extremely open in our partnering approach, you know, and so in anything that we do, even if it's a competitor, we will make sure that we extend the possibility for us partnering together. And frankly, it's worked out pretty well because the reality is I would rather grow the pie than keep trying to focus on getting the bigger piece of the same pie. And when you work with competitors and you grow the pie, everyone's happy. And this does not have to be a zero-sum game, you know. And so we've partnered with Nvidia, which we don't have that much of a competitive overlap with them, but it's a very synergistic partnership because they build GPUs, we build the networking. And the GPUs are — you know, the way I think about this is power is the constraint, GPU is the core asset for AI, and network is the force multiplier. In the absence of the network, the GPUs can't operate. Why is that? Because it used to be that these models were small enough that they would fit in the memory of a single GPU and have the processing speed of a single GPU be able to train the model. Then what happened is the models got bigger and you needed to train it on multiple GPUs. So you created a server with four to eight GPUs, and then the model got even bigger. So you said, 'Oh, these servers need to be put into a rack and stacked up, and that's what I need to train the model on.' And that became a rack. And then you said, 'Oh, I need to have a row of racks that need to be tied together within a data center.' And now we are at the point where you're not just doing — you know, the rack networking was called scale-up, the row networking was called scale-out. And now what you've got is networking that goes across data centers when two data centers that might be hundreds of kilometers apart will need to act as one coherent ultracluster for a training run. And we have created silicon and systems and optics to be able to have two data centers with two different power draws that might be hundreds of kilometers apart operate as one giant cluster, one coherent cluster. And when you do that, that's what's called scale-across. And those things are impossible to do without partnerships with others. And the way I think about a partnership is if someone has more than 20% of the market and you choose not to partner with them because you want to have all the revenue to yourself because you feel like you're arrogant enough to believe you're going to build everything, then all you're doing is getting yourself excluded from that 20% of the market. You're not doing yourself any service. So economically it makes no financial sense to not partner. I think it's a better way to live life to not just hate on people, and I think it's most importantly better for the customer because the customer has made an investment in company A and company B, and if we happen to be one of those two companies, it's our duty to make sure that we can actually work with the other company to protect our customers' investments. Because if we do that, the customer is going to say, you know what, I want to work with Cisco, I'm going to work backwards and do that. And so that's why I've always felt like partnering is at the core. So there's two core principles I hold very close to me, which is: first one is you got to build a platform, not just an individual collection of products that don't talk to each other. You have to have an integrated platform. And two, you have to have an open ecosystem. And if you do that right, then there's a pull from the market towards you, versus if you do that wrong, you have to push your products into the market, which is a much harder thing to do. It's a lot harder to win zero-sum games with multiple competitors, and you're always going to have multiple competitors. So —

Totally.

My rule of thumb is if I stay focused on what I bring a unique perspective to, where I have permission to play that I do better than anyone else, and then for everything else I partner, then what ends up happening is you just build a vibrant ecosystem. And that is truly the definition of a platform, where your contribution of your technology is so big that it actually catalyzes a movement throughout the entire industry, and that the industry makes much more money from your technology than you make from it yourself. And that then creates a level of, you know, value inherent in society with your contribution.

Let me actually talk about those partnerships just since you asked. So Nvidia, we've got a multi-dimensional partnership where they build out AI factories. We said, what about if we actually had secure AI factories? Would you want an AI factory or would you want a secure AI factory? It makes more sense to have a secure AI factory. So we actually have our security capabilities factored into the AI factory architecture, and then we have, you know, their reference architecture is something that we've continued to make sure that we can get certified against. That's one. We've also used — we just launched at our earnings — or not at our earnings, at GTC, you know, the 9100 switch where we can take that Spectrum X silicon and integrate that with our switches so that we can build switches with their silicon as well. Even though we build our own silicon, and the reason for that is because sometimes customers prefer theirs, and so you just want to give customers a choice. And so those are the kind of things that we've done with Nvidia, and we have a continued level of, you know, kind of tight synergy between our engineering teams that we can continue to keep partnering together. In fact, some of the executives at Nvidia are very dear friends of mine. And, you know, I think people like doing business with people that they actually tend to enjoy. In fact, yesterday we had our earnings call, and I had three or four of my competitors who just sent me a very kind note saying, 'Hey, congrats on a great earnings call.' And it just feels good to just operate that way, you know. With G42, we have a very — we are partnering very much with the sovereign cloud kind of initiatives that are going on, whether it be in UAE with G42, whether it be in Saudi Arabia with the Humane Group. And we want to make sure that we are continuing to partner with them so that as they do this data center buildout, that we become the network of choice for them. And so that's happening over there, and we will continue to keep doing that. I've been to Saudi and the UAE three times in the past six months. We met with, you know, the crown prince on both sides, and we've actually had a very tight partnership with G42, with Humane. We've known each other for many, many years. So I think it really helps. And then I'd say that even with the model providers, we will continue to partner with all of them because we want to be the Switzerland across the entire industry, and we want to make sure that we can provide network regardless of the model provider, regardless of the GPU provider. And we want to provide the networking, security, observability across the entire stack.

I think like power is a very interesting one because what's happening with power is rather than power being pulled to where the data center needs to get built out, what's happening is data centers are getting built where the power availability is the highest. And that I don't think is going to be a phenomenon that changes. I think every country is going to want to have their own sovereign data centers. And what we need to make sure that we do as an American company that believes in the power of, you know, what America brings to the table is we want to make sure that any and all of these data center buildouts that are happening worldwide are using American technologies to actually build these data centers out. And so whether it be in Southeast Asia, whether it be in the Middle East, whether it be in Europe, whether it be in South America or Canada, we want to make sure that we can help the buildout of data centers in addition to the data centers that are being built out in the US. And I think that the power scarcity will make it very important that these are global initiatives rather than just localized initiatives because you can't just always pull the power exactly where you need, which is also why this notion of scale-across that I mentioned is really important. Sometimes you might not have enough power that you can pull into a single data center, and you might need to have multiple data centers, but those need to operate like one data center that might be hundreds of kilometers apart because a training run has to operate with a number of GPUs that might not all fit within one data center. So if you need 300,000 or 500,000 or a million GPUs to go out and do a training run, and a data center can only host 50, 70,000, 100,000 GPUs, then you might need to have a coherent cluster of GPUs that might span data centers. And that's where scale-across networking is really important, where the technologies around security and around low-latency communication that's power-efficient gets to be very important because if you drop a network packet in a training run, you have to restart your training run. So you have to have, you know, technology that's built in the silicon chip itself that says I'm going to provide things like debuffering. So I can buffer the variance of the bit rate of the packets going in, and then that way if there's a little bit of a dip, I'm not going to go out and negatively impact the training run because that costs millions of dollars if you have to restart the training run. So those are the kind of things that require deep architectural forethought that we've actually put in place for building out, and these are like hard computer science problems and hard technology problems that Cisco does best at. And the way I like to help people think about Cisco is think about us as the picks and shovels company during the gold rush. We are the critical infrastructure company during the AI era where we just keep things humming so that everyone can get full potential from their AI investments that they make in —

Ain't entirely shabby. You know, it was a good time to be in the picks and shovels business.

Yeah, I think on that front, you know, what's really important is working closely between the public and private sectors. The public-private sector partnership is more important today than ever before. You know, like in the past six to 12 months, the number of government leaders I've met and gone out and visited is non-trivial. And why is that important? Because I think we have to learn from them on what their challenges are, they have to learn from the private sector, and then we have to make sure that we come up with joint solutions that can meet their requirements and that can also be technically feasible from our side. And so, you know, when you think about a sovereign — like the reality is there are two choices right now on who you can buy AI infrastructure from, right? It's either an American company or it's a Chinese company. Those are the two choices. And, you know, I have a lot of, you know, kind of sense of urgency on this particular topic because we have to move really fast because they're a very capable competitor in China. They actually can — they have done a great job in innovating, and their constraints are slightly different from the world's constraints.

Certain areas, and in certain other areas they might not be as viable a provider as we might be. Like for example —

For example, they don't build 2-nanometer GPUs. You know, they build 7-nanometer GPUs. The US builds 2-nanometer GPUs, but they have more resources on the power side.

And they have more engineering capacity, and so they can say, well, the inefficiency of a 7-nanometer GPU can be offset by unlimited power and unlimited, you know, kind of optimization of engineering resources. Those are the kind of things that it puts and takes that'll have to play out. And I feel like it's very important to keep in mind that as we build out the future, the US is a very formidable participant in building out AI capacity, data center capacity, not just for our own needs but for the global need worldwide.