Matthew Murphy1:37:51
All right. Outstanding. Outstanding. Super fun to have Jensen here as always. All right. So, we've been talking a lot about connectivity. Jensen and I just covered this. So, let's dive in now, right? Let's go one level deeper. So AI infrastructure spans every distance. It spans from hundreds or even a thousand kilometers between data centers to just millimeters inside the package. Every one of those distances requires a different solution. It's a different technology, different engineering team. It's a completely different set of experts and in many cases it's a different supply chain. So these are not variations of the same problem. What you have here is fundamentally different engineering challenges and that's what we're going to walk through next.
All right. So let's start with the longest distance. Jensen referred to this. This is scale across connecting data centers together. Now every major cloud provider has hundreds of data centers around the world and all of those data centers need to communicate with each other. This is fundamentally a long-distance connectivity problem. We're talking about links that can span hundreds or even a thousand kilometers. This requires very specific, very complex technology called coherent modulation. At the heart of it is a specialized digital signal processor or DSP. It's designed to push enormous amounts of data across fiber optic cables over very long distances with extremely high reliability. There's only a few companies in the world that build these coherent DSPs, and we're one of them. Marvell has been a leader in this technology for many generations. We build optical modules that contain all the electronics needed to drive and modulate the laser and transmit data over long distances.
So, I've got a little show-and-tell here in my pocket. Not holding up a chip this time. I'm holding up an optical module. This is one of our coherent optical modules. This is an incredibly complex piece of engineering. At Marvell, we build the entire module. This is ours. It includes the advanced node CMOS DSP. It's among the most complex chips. Just the DSP alone that we design at Marvell, but it also incorporates inside our fourth generation silicon photonics technology. That's inside here. We've been developing that technology and in production for a decade on silicon photonics. It also includes our own broadband analog components that we designed which is designed in silicon germanium. So Marvell pioneered this technology starting with 100 gigabits per second a decade ago then moving to 400 gig and now shipping 800 gig in volume and later this year we'll be sampling the world's first 1.6 terabit 2-nanometer coherent optical solution. And that couldn't come at a better time. Demand for bandwidth has never been greater.
All right, now let's go inside the data center. So these data centers can be very large spanning hundreds of meters and they contain racks and racks of compute servers. Now each rack typically has a switch at the top with servers connected into that switch. Those rack level switches connect to the spine and then the core switches. This creates the network fabric that ties the entire data center together. And all of that is connected through fiber optic cables. Now once again optical modules drive data transmission over those fiber optic cables. But this time the modulation scheme is different. Instead of coherent technology we use a more power optimized modulation technology which is called PAM4. So the two key semiconductor solutions for this part of the market are the PAM4 chipset inside the module and then the cloud switching infrastructure that ties the data center together.
Marvell builds both. Starting with the PAM4 chipset, we build the industry's leading PAM4 DSP solution and also the high-speed analog components that go around them including transimpedance amplifiers or TIAs and laser drivers. These are also in silicon germanium by the way. And we've led the industry through every major transition of PAM technology starting at 50 gig, 100 gig, 200, 400, and 800. Then last year, we began ramping Marvell's 1.6T 3-nanometer PAM4 solutions, leading the industry's transition to 1.6T connectivity.
Now, for Ethernet switching, Marvell has a similarly complete portfolio of products from 12.8 terabits to 51.2 terabits. And today we announced our new 100T Ethernet switch specifically designed for AI data centers with the industry's lowest power.
Special announcement for Computex. We waited. So you put it all together, we provide a complete solution for connectivity inside the data center.
Now let's move inside the rack. The goal here is to connect the largest possible number of processors together in a full any-to-any configuration. In other words, every processor can communicate directly with every other processor. And Jensen talked about this. The first company to bring this architecture to market was Nvidia with NVL72 named for the 72 GPUs connected together inside a single rack. And this required a completely different approach to connectivity. There's a different class of switch and the ability to drive very high-speed signals over copper backplanes inside the rack. So today this is not the domain of optics. This is the domain of copper and the core differentiator here is the electrical SerDes technology not the optical. Now, Marvell also has leading electrical SerDes at 200 gigabits per second today. And we've demonstrated already over the last couple of years, 400 gigabits per second for the future. So, we're building this SerDes technology into our customers' custom silicon and their XPUs and also into our own scale-up switches.
All right. Now, let's go all the way inside the package here. We're not talking about meters anymore. We're talking about millimeters. And you might not actually think about this as a connectivity challenge, but today most advanced chips have multiple chiplets inside the package. So when you have 2.5D or 3D packaging, it's fundamentally a connectivity technology actually. And it allows these chiplets to sit very close together inside a package and communicate through ultra high-speed short-reach die-to-die interfaces. And Marvell has leading die-to-die SerDes and leading capability in advanced packaging allowing our customers to build some of the most complex unique multi-die chips in the industry.
So as you can see connectivity for AI data centers requires a very broad portfolio of technologies. Each distance requires a very different solution. And Marvell has the industry's most complete portfolio from millimeters to kilometers. Every hop, every distance. And it turns out having all of those capabilities under one roof is unusual. It's unique. When we go and compete, normally there's a different set of companies that we compete against in each one of these categories across these different distances. But this is what makes us unique. We're the one-stop shop. We're the leader across the entire connectivity stack. And that brings us to the next major challenge facing the industry.
So, what you probably notice as I described these different solutions in the last couple of slides is there's different solutions for different distances and that some of those connections today are optical and some of those connections today are electrical. And it's actually defined by distance. And so the connections on the left side of this chart are optical today. That means they use fiber optic cables to transmit light with complex electronics on either side of the cable to drive and modulate the laser that's transmitting that light. Connections on the right side of this are electrical. So they use copper cables or just copper traces that are printed on the circuit board or even microscopic copper routing inside the package. So the common theme here is copper. And in the middle you see the wall, the copper wall. And the wall is defined by the longest distance you can transmit a signal over copper. So before you have to move to an optical connection. So this is an important distinction because copper is simple and it's low cost and as Jensen said you want to use it for as long as you can. It's very practical.
But optics is more complicated. It requires lasers, photonics, complex electronics. So it's a bigger lift but it's going to be needed. And the copper wall, what I'm here to tell you today is it's about to move. It's going to move again and it's going to take over the rack itself. So, this is creating an explosion in demand for the optical industry. Incredibly complex engineering challenges are coming along the way. So, why is this happening?
So, it's not just somebody's preference to go do this. This is physics. The distance a signal can travel over a copper cable is inversely proportional to the bandwidth. So every time you double the bandwidth you have to cut the distance in half. Today the highest speed production systems in the world run at 200 gigabits per second per lane just to give you an example. So at that bandwidth the cable length is limited to roughly 2.5 meters. Now by comparison systems running at 100 gig could use about 5 meter cables and the height of the rack is about 2 meters. So once you account for all the routing inside the rack 2.5 meters is right at the limit. So when we move to 400 gig, we can no longer fully connect the rack with copper. So the wall is moving and it's moving now.
Going forward, even the connections within the rack will become optical and the whole industry knows this is coming. So we've been preparing for this moment, not just Marvell, but the industry. And you see this in Taiwan, by the way, and the supply chain and the ramp up that's happening. The ramifications for this are actually enormous because each time the wall moves one step to the right, the number of connections that you have goes up by at least an order of magnitude. So it's creating this explosion in demand as I mentioned and the optical supply chain needs to scale up massively and be ready. We've seen this movie before. Okay, I mean 20 years ago and I remember this when state-of-the-art was 10 gigabits per second inside the data center. It was 10 gig and we used copper cables all across the data center. Optics back then was reserved for just very very long distances. It was essentially like a telecom technology. But when the wall moved, the optics industry actually rose to the challenge. And today all the hyperscale data centers in the world, they're all optically connected. And as we saw in that transition, it did require new solutions. You couldn't use the same power hungry kind of telecom approach, which is where PAM4 came in. It's optimized for power, density, and reach and requirements specifically tuned to inside the data center. And Marvell was one of the key innovators there. So, we're about to see the same wave of innovation needed as optics moves inside the rack. And that's with a technology called co-packaged optics or CPO. You hear a lot about this now. I'm going to tell you more.
CPO is a technology where we bring the optical connections all the way to the package itself right next to the compute either the custom compute or the switching silicon and the fundamental challenge we're solving with CPO is density and power. Now remember the number of connections inside the rack is like 10x the number of connections between the racks. So if you just try to use the same optical technology used across the racks in the data center, you wouldn't have enough power. You wouldn't have enough physical space. You cannot fit all these standard optical modules and cables as they are today. It just doesn't work. It's not possible. So, the industry has been inventing this co-packaged optics concept which brings the optical fiber right to the package and it tightly couples the electronics that drive the signal over the fiber directly with the custom compute or switching silicon. So this is a massive change and it's hard because you're combining some of the most advanced technologies in the chip industry. Leading edge CMOS, silicon photonics, advanced packaging, optical interconnect, all manufactured in a small, tightly integrated system. So the complexity is very high, but it's the only way to continue scaling bandwidth and overcome this limitation that I talked about with copper while reducing power at the same time. So this is where the industry is headed and this is one of the reasons that Marvell has invested for more than a decade in silicon photonics, optical DSPs, all the analog broadband components around it and all the advanced packaging you need to pull this off needs to all come together actually in CPO. So this isn't some futuristic thing guys, okay, it's happening now and in fact I brought a couple of Marvell examples with me today so let's do a quick show and tell.
Okay. So, over here you have a traditional Ethernet switch. This is our 100T Teralink switch that we announced today. And you guys are the first to see it. Actually, everybody here in the room. You can see the switch in the middle of the board. Copper traces inside the PCB carry the signal to the front panel, which is here. And this is where all the optical modules plug in. Now, let's move over here. This is a CPO-based switch right here. Now, notice that there's still the switch silicon in the middle. That's right in the center of the die of the package. In this case, this is our 51.2T switch. And all around the edges are 16 3.2T optical engines. So, the 16 times 3.2 you get 51.2. So, this is the fiber is directly attached now to these engines. It's not to the front panel. So, we've completely eliminated the copper traces on the PCB. Light comes directly out of the package. Okay, this is a very very complex piece of engineering and it was very cool to be able to show this off today. Okay, so co-packaged optics is here and the industry is scaling up to meet the challenge and as we've seen time and time again, every time we reach a physical barrier, we break through it with technology and innovation. In this case, by replacing copper with fiber, because unlike electrons traveling over copper wires, the distance that photons can carry a signal through glass is largely unrelated to the bandwidth. So, as AI infrastructure demands even higher transmission speeds, and needs to scale to larger and more complex systems, spanning millions of processors woven together now, not thousands or hundreds, optical connectivity will increasingly become the de facto solution. So the real question becomes what does it take to deliver optics across the full AI infrastructure stack? What's it going to take? Well, it starts with recognizing there is no single technology for the entire data center. It's not how this works. There's no one-size-fits-all solution. There's no shortcuts. There's no easy way to the end here.
There's not a single architecture, modulation scheme, frequency band, unique technology that's going to do it all. There's no free lunch. That's why we are pursuing a bunch of different unique optical paths across every distance to get here. Each one of these technologies we have up here is optimized for a different design point. Each one enables a critical part of the infrastructure and addressing different requirements for density, bandwidth and power and integration all across the stack.
So if optical interconnect is the underlying technology for which next generation AI infrastructure is built, then Marvell is building the broadest portfolio with the deepest bench in the industry. But no company can deliver this transformation alone. And as Jensen talked about earlier, right, it takes an ecosystem to get here.
So, like I said, technology innovation is great. It's part of the challenge, but not all of it. But demonstrating this at scale is really what matters. And at this point, if you're just operating on a PowerPoint or a demo, press release, it's not going to get you there. Customers need solutions now that are ready. They're reliable. They need to be manufacturable and be ready to deploy at scale. So Marvell and our ecosystem partners have been doing this for a long time. We've already shipped hundreds of millions of DSPs. We've accumulated through our volumes tens of billions of device hours of data in the field. This experience matters because these products have to work not just in the lab but in the world's largest data centers at very high volume and very reliably for years. So that requires investing ahead in the manufacturing ecosystem. We've got to build the capacity and the supply chain infrastructure before the market arrives. This is why the ecosystem matters so much and it matters a lot here in Taiwan by the way. Now, one of our most important partners at Marvell in this journey has been Advanced Semiconductor Engineering or ASE. Now, ASE is one of the world's leading semiconductor manufacturing companies. They have more than 100,000 employees with operations in Asia and actually all around the globe with a decades-long track record of helping enable pretty much every major technology transition we've gone through in the semiconductor industry. Now leading ASE through this period of transformation is someone that I know quite well. He spent more than 25 years helping shape both the company and the industry.
Today I'm thrilled to have my next guest speaker come up which is ASE CEO Dr. Tien Wu. Tien, please join me on the stage. Thank you.