Chief Development Officer, Marvell Technology
The Futurum Group at Marvell's Industry Analyst Day 2023 with Sandeep Bharathi—Futurum Tech Webcast
On this episode of the Futurum Tech Webcast – Interview Series, host Ron Westfall welcomes Sandeep Bharathi, Chief Development Officer at Marvell Technology for a conversation on AI and the impact that it's having on the semiconductor industry as well as portfolio development, during Marvell’s Industry Analyst Day Event.
Their discussion covers:
The impact that AI will have on semiconductor technology and the market moving forward from Marvell’s perspective
How Moore’s Law is changing and how Marvell has shifted its technology and IP strategy to address this change
Advanced connectivity IP...
About Sandeep Bharathi
Sandeep Bharathi, Chief Development Officer at Marvell Technology, discussed the company's approach to AI-driven data infrastructure and semiconductor innovation during a December 2023 webcast. He stated that AI depends on compute infrastructure and data movement, and that Marvell is addressing bandwidth challenges through evolving signaling mechanisms, such as moving from NRZ to PAM4 and coherent signaling. Bharathi noted that Marvell announced the 1.6 Tera Optical DSP Nova and the 800-gig coherent DSP Orion, and described a 51-terabit switch with 512 lanes of 112-gig SERDES. He characterized Moore's Law as an observation rather than a physical law, and said techniques like 3D stacking and advanced packaging are used to maintain progress. Bharathi emphasized that Marvell's investments span standards work, foundry technology, EDA tools, and applying AI in chip development, and that partnerships with customers, vendors, and foundries are fundamental to their strategy. In a July 2021 talk on chiplets and data infrastructure, Bharathi argued that the explosion in data generation has made hardware and silicon "hot" again, and that chiplets are needed due to the prohibitive cost of advanced geometries and the slowing of technology scaling. He advocated for standardized interfaces and modular architectures to achieve cost effectiveness, while noting that modularity involves trade-offs, such as latency considerations. Bharathi stated that not every product requires a chiplet approach, but that it is becoming imperative for most compute and networking segments. He also discussed the role of thermal management, the move from copper to optics at higher data rates, and the need for collaborative industry approaches to solve challenges like power and packaging.
Transcript (17 segments)
R
Ron Westfall0:10
Hello everyone, I'm Ron Westfall of the Futurum Group and you're joining us here today at the Marvell industry analyst event. I'm very pleased to have a distinguished guest with us today, Sandeep Bharathi. To start things off, first of all, Sandeep, please share who you are and what you do at Marvell.
S
Sandeep Bharathi0:35
Yeah, thank you Ron. It's great to be here and to share with you about what exciting things we are doing at Marvell. I am the Chief Development Officer here at Marvell. I joined Marvell in February of 2019, so it's going to be soon five years. I lead the development and central engineering activities.
R
Ron Westfall1:11
Excellent, thank you Sandeep for that intro. Let's dive right in. The hot topic of the year is AI and the impact it's having on the semiconductor industry as well as portfolio development. From your perspective, what is it about AI that's driving portfolio innovation? What's making all the difference here in our ecosystem?
S
Sandeep Bharathi1:34
Yeah, excellent question. I think AI, a lot of people talk about training, inference, these large language models and many of the technical jargon that we have known since ChatGPT came online. But the other aspect, whether it is within the chip or between chip to chip or even off chip and board, that's the data movement challenge. The more data gets generated, the more data moves. AI is all about bringing data from one place to another securely, computing it, and storing it, which is our mission statement. No matter whether it's AI today, yesterday, or in the future, these challenges will still remain, and that's what we are intent on solving.
R
Ron Westfall2:46
Yes, it's very exciting times. I think one thing that people are very interested in hearing about is, for example, AI's impact on Moore's Law. Could you share your perspective on that?
S
Sandeep Bharathi3:10
Moore's Law is not a physical law; it was an observation by Dr. Gordon Moore in the 60s when he published and saw the trend that computing power doubles at the rate of 18 months to two years. Over the last four or five decades, that has been sort of right. Now as we move to the limits of semiconductor scaling, which means every process geometry by classical scaling meant that if you go from 28 nanometer to 20 or 16 nanometer, you could pack in two times the transistors. But now, people go to advanced technologies for three reasons: one is to pack more transistors in a given piece of silicon, the other is higher performance for the silicon, and the third is energy efficiency or lower power. Because the limits of physics are almost being reached, there are newer innovations for transistors. We had planar transistors that went to FinFET, now there are gate-all-around transistors or nanosheets, then it will soon be some other material. But you also have to integrate a number of these aspects. That is what AI is bringing up: how much more can we pack and how much more can we get in and out of the package?
R
Ron Westfall5:22
Yes, I agree. I think packaging is going to be a huge differentiator for how silicon is selected. That is an excellent historical overview of Moore's Law and what is going on today. Looking forward more, what is Marvell doing to take advantage of these capabilities and specifically in relation to IP networking capabilities?
S
Sandeep Bharathi5:50
Very good point to segue into. I want to move from what we talk about data movement to what is also happening: the analog mixed-signal characteristics or the scaling, for example, the caches of the SRAM or the SerDes architectures. They are not scaling as effectively as digital circuits. So what we need to do is really increase the bandwidth through different signaling mechanisms. One way to explain is if you have a 10-lane highway, you can only send so many cars at a time with so many passengers. But one way to double the number of passengers is to go to a different signaling scheme. We evolve innovations in signaling transitions so we can send more bits or gigabits per second. That's another metric we use to widen the bandwidth capability with the highest energy efficiency or low power. That is done by a lot of innovations in analog mixed-signal circuitry as well as different algorithms for error correction because when they go a certain distance, they lose fidelity. We need to build in DSP algorithms. That's what is very evident with our portfolio, but also innovations in signaling techniques and innovations in our analog mixed-signal portfolio.
R
Ron Westfall8:19
I can understand that. I think that does lay the foundation for drilling down more on a specific aspect of the portfolio that I see increasing recently becoming important, and that is the connectivity IP capabilities. Naturally, I know folks out there are very keen to know more about what Marvell is doing in that area specifically, and also any insight as to the direction you're seeing these capabilities going.
S
Sandeep Bharathi8:50
Let me step back a bit. The connectivity IP portfolio that we build, I talked about data travel within the chip or across the chip. The energy efficiency metric is called picojoules per bit: how much energy you expend by transferring a bit. If we go back, there was the classic NRZ or non-return to zero where your signaling is zero and one. Then we went to PAM4, which uses four levels to transfer bits. With coherent, it's really 16 different levels. PAM4 went from 56 to 112 and now we are at 224. What is very critical in all these is how effectively we can do error correction that we can overlay on top of the signal loss that is there when you traverse across chips. Innovations are happening on a regular basis. We were the first with PAM4 signaling with our inphi acquisition with 112 gig. Then we are now at 224 gig. We recently announced the very first 1.6 terabit optical DSP called Nova. On the coherent side, we introduced Orion, which is our first 800 gig coherent DSP. All this is proof of what we are able to do, and it's not showing any signs of slowdown because the need for AI means we're seeing the innovation engine moving forward.
R
Ron Westfall11:12
It's built in, and that makes for a great foundation for more innovation. This could be an opportunity to drill down on some of the applications that are benefiting from this, including data center interconnectivity. What else are you seeing on the application side that's really benefiting from these breakthroughs?
S
Sandeep Bharathi11:38
Let me draw your attention to what you're holding in your hand. Product placement perfect. This is really our very first 51.2 terabit switch. You can look at the size of the package. The thermal management is such an important vital piece. It's not always the bits and bytes, it's not always we go from 7 nanometer to 5 nanometer to whatever is the next process technology of choice. It is really a holistic design process of chip-package interaction, package-thermal interaction, thermal and power management, and how all this is brought together in innovative substrates, connectors, and what we call thermal interface materials to manage the thermal performance between where the die is and the organic packages. All this is really important. Whether you have a box or a module, everything has a power envelope. You don't have a product if we don't meet those specifications. That's where the innovations on not just the connectivity IP portfolio, not just process technology, but the entire platform become very imperative as we are addressing products for the hyperscalers.
R
Ron Westfall13:38
I am certainly impressed by this packaging. You can grip it in the palm of your hand, so what's not to like? I think that definitely sets us up, Sandeep, for the next topic that people are very interested in: what Marvell is investing in to drive future innovation.
S
Sandeep Bharathi14:10
There are several different areas. For example, standards are becoming an important piece. You can't go at it alone, so we are in several consortiums that take investment and need to know where the puck is headed. That's one area of investment. Another area is from a foundry technology perspective: what other things are out there? TSMC and the rest of the foundries are looking at various ways to get power. There's talk about backside power and how it helps the integrity. We are looking at these advanced things that are three to five years out. It's also EDA tools. For example, what AI techniques we should use in order for our chips to be better, faster, cheaper, lower power. What worked yesterday, it's the famous Einstein quote, or maybe attributed to Einstein: you can't solve tomorrow's problems with yesterday's solutions. So we have to be continuous in how we build the chips. Are we doing it right? Do we need to build models such that we help our design team, a very capable design team, to take AI algorithms into account so that there are fewer bugs? Because a bug is very expensive when found after the fact. If we can use AI to help us design better, we can reduce the time to market and improve the process. Again, there are different vectors of investment in advanced technologies to build silicon, software, how we integrate all these things together, and work with ecosystem partners because we're not going at it alone. We have a hybrid strategy: we will develop stuff that we are good at, but we are also depending upon the ecosystem to get other aspects that our hyperscalers and customers need.
R
Ron Westfall16:40
I agree wholeheartedly. In fact, that was my next question that you addressed superbly: what about the partnerships? Earlier today we talked about Nvidia, Arm, and F5 came up in terms of partnerships. What do you see Marvell doing in terms of partnerships to drive innovation?
S
Sandeep Bharathi17:10
Partnerships are with our customers and with our vendors, including the OSATs, because there are many different kinds of products that we will build. For example, let's take a look at chiplets. If you need to integrate chiplets, that requires a different business model. It could be our own or something from a third party, and thereby the ecosystem partnerships come into play, both in terms of our foundry partners, our customers, end customers who will have to get HBM chiplets from somebody else, and EDA IP providers. Sometimes the ecosystem will mean that we may have to do co-development. Bringing it together in terms of Marvell's process capabilities, IP connectivity, and packaging proposition, how is Marvell going to continue driving innovation across the ecosystem? Quite simply, make a difference with the partners and the customers out there. You have heard about what we are doing with silicon photonics. These are long lead time items that we continuously put out test vehicles. Sometimes these are to learn from them to further the technology portfolio. These are the various aspects by which we are looking at future technology. Some we can talk about and some we cannot.
R
Ron Westfall19:22
Thank you so much, Sandeep. Thank you. With that, thank you everyone for joining the Futurum Tech webcast. Please be sure to join our website to see our latest installment. With that, have a good semiconductor day.