Networking Becomes the Bottleneck: Rami Rahim on Building Infrastructure for AI at Scale

It's great to be here. Thank you so much for having me, Daniel.

Yes. Yes.

So, we were getting ready to make a big announcement at this show that unfortunately didn't happen in time. So we missed it literally by days. This is my first HPE Discover Las Vegas.

Um, I was at HPE Discover Barcelona, our European customers.

Yes.

Right. It's a great question. For the first part of your question, there is no doubt that there is an immense amount of focus and energy right now on networking. I wouldn't say it's emerging. I mean, we are living it right now. Many of our biggest customers, the AI cloud builders are really thinking about three things: space, power, and connectivity. And it's interesting because if you take a look at a typical AI data center, maybe only around 10 to 15% of the total investment in that data center is going to be networking. But it is a true force multiplier. That investment is going to determine how much you're going to get out of your billions of dollars that you put into compute. If you're running your network at 50% capacity, having blocking issues, congestion issues, guess what? That investment in compute is just not going to live up to expectations. So networking has become incredibly important again. It's essential infrastructure for the AI era. To the second part of your question, Daniel, about the integration, it's going great. Honestly, it's a lot of work. Integrating two large companies is not easy, but we are ahead of schedule. We told ourselves as we work through this integration process, we are not going to let ourselves get distracted from what's important, which is innovation and supporting our customers. We've been doing all of that and the integration work. You can see the innovation engine since the beginning of the close of the acquisition. It's amazing. It's honestly awesome to see.

The numbers are getting bigger primarily because the demand is getting a lot bigger. I mean, put the supply chain issues aside, the innovation and the process node advancement is allowing us to actually achieve far more bits per second per watt in networking and that's allowing us to build far bigger data centers today. And of course we are capitalizing on that right now in networking, moving more data. Let's face it, it's generating more tokens and that is what we need to move the data faster and generate those tokens.

Both of those key areas of opportunity are important to us and both are massive opportunities. Starting with AI for networks. This is just a new mode of operating networks across every single layer of the network from the campus to branch to data centers, wide area networking, security, you name it. The old way of operating networks, manual configuration, manual optimization, manual diagnosis of issues when they occur just doesn't cut it in the AI era. And so we need to leverage artificial intelligence to crush the cost of managing networks and to delight the end customer. I truly believe that we at HPE networking have made the most progress of any other technology company in that space. Our solutions across Aruba Central and Mist are just awesome and honestly light years ahead of anybody else. The other opportunity you mentioned is networks for AI. This just goes back to what we were talking about. There is an insatiable appetite for the production of tokens and that means you need infrastructure and networking is such a key piece of this. One thing that we have the ability to do now in HPE networking is not just the scale across routing solutions or the scale out Ethernet solutions that connect different racks together, but also the scale up solutions that we're developing in partnership with companies like AMD with a product called Helios, where the network is an open Ethernet fabric, first of its kind to connect GPUs together. That's a brand new opportunity that has come about because of the combination of Juniper and HPE.

Yes.

Both Nvidia and AMD are amazing partners of ours, very important partners. And we're working our solutions with each of these technology providers in different ways. With Helios, it is truly unique in that it is now bringing Ethernet and our own Ethernet technology with our software and capabilities, services into the scale up solutions for the first time. And I mean, you might have seen some of these stats, but scale up from a capacity standpoint can be sometimes 10 times the total bandwidth from a requirement standpoint as scale out. It's staggering capacity that's required for that layer of the network.

Sure is. We landed a great Q2 and networking was a big part of the success story there. And yeah, we're optimistic about the future for sure.

Well, let's start with AI for networks. We have now for a decade been collecting telemetry from real deployments in some of the largest, most complex environments, the largest retailers, largest banks, largest healthcare systems, and learning from those deployments and teaching our AI engine. It's called Marvis. So Marvis has essentially for the last decade been looking at real world examples of failure modes, learning from them, and that's gone into our algorithms that are now proactively addressing issues before they start to impact customer experience. That closed loop system that we have now built is way ahead of anything else that exists in the market today. You can see it in analyst reports like the latest Gartner Magic Quadrant for wired and wireless infrastructure where we are farthest ahead in both vision and execution. Then in networks for AI, we build incredible networking infrastructure. A lot of that pedigree comes from Juniper, of course, where we have a history of developing high performance networking. But I think the real unlock here for us is to combine compute, storage, networking, the virtualization, the automation capabilities across the stack. And that's how we achieve a level of differentiation that will be very difficult for the standalone networking companies to take advantage of.

Yes.

I'm glad you brought it up because we firmly believe that a self-driving network must also be a self-protecting network. So security is an integral part of all of our solutions across each of the networking layers. This is where again the combination of HPE and Juniper is really powerful because we have brought together an amazing set of ingredients. Best efficacy firewall, very high performance firewalls, network access control that's world class, the best SD-WAN solution in the market by a lot, a great cloud security solution, and we're bringing them together, embedding them into each of our self-driving networking solutions and working towards the future of universal zero trust network access which really every CIO wants a path towards. So security is absolutely integral. We've got amazing ingredients and we're executing rapidly there.

Yeah. You know, rewind about five years. A lot of the conversations that I was having with customers and prospects was to convince them that they didn't need a lot of on-prem controllers to manage their network. They could go to the cloud and it would simplify their life significantly. And now many customers have moved in that direction. It's much less of a convincing job. Today's conversation is about convincing them to do what you just said: take your hands off the wheel because you cannot keep up. You cannot keep up with the complexity of networking. And I have to say more and more customers are starting to do that. We're giving them knobs on an individual feature basis where we say, for this kind of issue, if it occurs, I don't even want to be informed. Let me know after you've gone and addressed the problem on your own. And Marvis will do just that. So it's amazing to see customers build trust in the technology and literally take their hands off the wheel.

Yes.

While we have made so much progress, there is still a lot more to do. I think we're probably around 70 to 80% of all issues that can occur in a network can be addressed proactively using a closed loop automation system. We are going to get to 100% in the next few years because of the advancements in agentic AI ops. It's truly staggering to see how fast this technology is evolving. And I think we're going to get there first. Now look at the networks for AI. There is no end in sight to the demand right now for the generation of tokens. What that means for us is we need to continue to push the envelope on silicon, systems, and software for scale across scale out and scale up. We're in this period where if you can demonstrate superiority in performance and power efficiency, you're going to get a big chunk of the pie. And that's what we're working towards.

Honestly, I'm not just saying it. It has exceeded my expectations. It's just an amazing event. Such a density of customers and partners to talk to. Loved doing the keynote address as well. So yeah, I'm looking forward to the next one already.

My pleasure. Thank you.