In an AI digital age, energy capacity equals economic capacity. Why build a new data center and buy a million GPUs if I can't power them? You're going to see more and more defaults happening on many of those data center projects because the energy won't be there.
Whenever you have the technology to make something easy, that means the bottleneck moves somewhere else.
Nothing's a chip anymore. It's a rack. It took me three months to design it, but it's nine months until I can actually start to use it. Exactly. How many major new memories have we had over the last 30 years?
Zero. Memory innovation for the first time in 30 years is nigh upon us. I declared the death of copper about 25 years ago. Eventually, I'll be right for all of us hardware guys. This is like a renaissance in front of us. AI inference accelerator chips. And I'm sure I don't even know them all. Why are you guys funding so many of those?
You funded your fair share, too.
Historically, there have not been 100 competing processor vendors in any industry ever. This is a temporary thing. It'll converge back to a few.
I'm here with our esteemed guest and dear friend and former boss, Mr. Pat Gelsinger. Welcome, Pat. Pat is currently the general partner at Playground Global, but as you're very, very well known in the industry for leading Intel, for being the CTO of Intel, of course, leading VMware and many other things. So, welcome.
Hey, thank you, Ravi. Great to be with you and Guido, right? And to me, this feels just a little bit like old home, right? You know, it's like, you know, you know, it was like we superimposed a year of change, right? But you both look the same. I feel energetic. So let's dive in.
There has never been a day when you stopped being energetic. So there's no news there. But yeah. No, let's start actually from your Intel career, right? I mean, recently Andrej Horovitz created this Horovitz and Recurrent Academy, which was to find talented people between 16 and 22 and then put them into a modern educational setting so they'd be well prepared for either doing things on their own or joining the companies of today. You had a similar experience, like you went to a regular trade school and then jumped into Intel.
Yeah. Yeah. When we were what, 18 or 19?
You know, it was really a sort of magical period. You know, I'm 16 years old. I accidentally win a scholarship. I go to tech school, right? Skipped my last year and a half of high school, and Intel comes recruiting when I'm 18 years old. And, you know, I'd never been on an airplane. Had already fallen in love with computers at tech school. And the interviewer, right, Ron Smith was his name. He writes on his page, and I was number 12 that he interviewed that day. And if you've interviewed 12 people in a row, you can't tell male from female, you know, by the end of it, right? You know, I'm number 12. He says, 'Smart, aggressive, arrogant. He'll fit right in.' So I got invited to go, you know, come to Intel, and, you know, it just became, you know, really glorious. You know, starting as a technician, moved into the design team at the end of the 286, you know, engineer number four on the 386, you know, architect and design manager for the 486, all of that while doing my master's, my bachelor's, my master's and PhD work. So it was like the best career that you could possibly have. You're learning by day and you're putting it to practice at night.
That's right, that's right. You're creating a completely new era of chips. For I bet most of what you learned and what you built there was a big gap because you're breaking new ground.
Yeah. In a lot of ways, and, you know, it really was a pretty incredible time in the industry. And I remember, you know, one of the professor's classes, you know, he postulated a new carry look-ahead adder design, right, in one of my classes at Stanford. So I said, well, I'm working on the 386 carry look-ahead adder, so let me go try it. So I come back two days later to the professor and I said it doesn't work, right? And I started an argument with my, you know, professor at Stanford, and it was like, you know, it was like, what, you know, the next two class periods were just me and the professor arguing over why his design didn't work.
He had a book that was being published with that design, so he was very annoyed. It was very time sensitive for him.
And, you know, we started built-in self-test, right? You know, and I remember Ed McCluskey at Stanford was my professor, and he was sort of the father of that. And we, you know, had wonderful arguments around why most of his ideas were bad, they weren't practical for real chips. But I was putting, you know, built-in self-test to work on the 386. You know, John Hennessy, you know, was my thesis advisor, and obviously he did okay. I say, you know, I helped his career. He became president, and he's done okay since then.
Later on we'll talk about the CISC versus RISC debate, but I want to talk about the 486. Sorry. Go ahead.
I had the same experience when I was teaching at Stanford. You know, you come with something that you think is the truth, and then the student comes that works in industry and is like, well, actually not quite. So, you know, the next lecture you come back with like, okay, I looked this up. You're right. You know, this is outdated.
Yeah. And I tell you, you know, that's part of the set of students, that interaction, and I think that's part of what makes Silicon Valley so unique. Right. You know, is you have, you know, these so many companies, and, you know, entrepreneurship, it's like in the water here, right? You know, starting things up and new ideas and challenging the professors. And you can't be a professor at a place like Stanford and not expect to get challenged and enjoy it.
There's a good foot traffic in and out of that ivory tower.
Yeah. Now, the 486 was your big accomplishment, if you will. Obviously lots of people worked on it. And you guys broke some new ground in chip design there, right? And increased what functions the chip was supposed to do and so on and so forth.
Yeah. It was a pretty magic period as well because it was before what you would think of as the EDA industry. And, you know, the end of a little bit of the 386, but the 486 was really the first chip to implement what you would think of as modern EDA techniques. Yeah, we did a high-level design, right? A description and RTL description, but there was no Verilog. So we invented HDL, right? The Intel hardware description language. So, you know, I wrote my own language. Well, you had to build a compiler for that language, right? You know, so we created the compiler for the language. There was no automatic place and route, you know. So we had to invent that. And we worked with Alberto Sangiovanni-Vincentelli at Berkeley and some of his students for the first, you know, placement, the first routing, the first automated timing, you know, management. And so they created EDA to some degree, to a great degree. That, you know, and that was really one of the hallmarks that not only was the 486, you know, this compatible, you know, pipeline microprocessor, but we ushered in, you know, many of the foundations of what became the modern EDA industry. And, you know, it was really, you know, a pretty magical period of the industry.
Yeah. Yeah. So if you look at today, will people look back at Blackwell and say, you know, this was of the last generation that was done before AI took over the design process for micros? Are we in a similar transition right now?
I think there are certainly aspects of that. And, you know, I think, you know, when you look at, you know, things like Jalapeno today, right, you sort of say, you know, that, you know, that's sort of a first-principles use of AI in the chip design process, right, where you throw out a lot of those things. Now, you know, there are pieces of design that are still really hard, right, in that sense. And a lot of the analog, right, aspect, Certis is probably the best example of that, you know, are still not viable, right? You just need lots of silicon data, right, to get those. So you either get so conservative in your analog design, you know, that you're able to, I'll say, AI it, or, right, you got to do those the hard way, you know. But now many of the logic functions can really be done with AI tools and techniques in pretty incredible ways, right? Your transistor budgets are big enough, the design tools are getting smart enough, you know, a few experts guiding the tools and how to apply it. And I really think it will be somewhat like the 486 that way, right? Where you sort of look back and say, 'Yep, that was the beginning of a new era of chip design.'
Whenever you have the technology to make something easy, that means the bottleneck moves somewhere else. Yeah. Can we guess at this point where the bottleneck will be in the future?
There's some things which it seems like AI is incredibly good at, right? Like creating the software layers, you know, writing kernels, you know, a lot of the... I think a lot of the sort of being able to specify the objectives at a higher level, which then gets sort of translated into Verilog, right? And so what is the new frontier? What is the thing that will be difficult going forward? Well, you know, when you look at today, you know, so let's say, you know, Guido and Pat, we're off to do a great new chip together. We're building, right? You know, right? You know, and it's the one that's going to leap ahead because we have understanding of x86, you know, we have understanding of the AI workloads. We have understanding then of how to compose that into the right multiply, accumulate, you know, register structures, the perfect visibility model workloads of tomorrow as well, you know, and all of that kind of stuff. And, you know, let's say, you know, we turn our AI, you know, tools loose, we are great. Within three months, we have an awesome design, right? You know, for that, we've been a little bit conservative on all the analog components of it. You know, now we still have like a couple of really major bottlenecks today. You know, one is, right, we still have nine months of silicon processing time, right? You know, something like that, right? So I can design the thing in three months, but I can't actually get it into real silicon at scale for nine months. Okay, that sucks, right? So we have to really see that bottleneck improve if we're going to have this innovation because we know that...
About the validation, verification stages, or even beyond that?
Well, you know, first, I can't get anything out of fab, you know, in less than three months. And that's if I have, you know, like supercharged design flows, right? So I said, you know, how do I compress that, right? And then I got to get into advanced packages. 3D packages are getting complex. So that's like another three months, right? You know, and then I have to put it into a rack-scale solution because nothing's a chip anymore. It's a rack, you know. So we're nine months, you know, it took me three months to design it, but it's nine months until I can actually start to use it, you know. So how do we start to compress, you know, those aspects of design? And I think we need new forms of lithography, you know, to enable that. We need, you know, design flows that don't require, you know, $50 million of mask cost until I can get things into prototyping. So, you know, to me, how do you compress that to a month or two, right? Because I can now do the design in three months. How can I have that done in a month or two? Because if it takes me a year and a half until I actually get scale and software on it, okay, my understanding of the AI workloads is no longer applicable to the chip that I designed. Right. You know, so, you know, all of those...
You see that play out live right now with AI. Yeah.
Right. You said, you know, it's almost, you know, we can look at it historically. It was like, you know, Graphcore wasn't a bad design, but the world moved on, right? And, you know, in that. So all of those aspects somewhat obviate the fact that AI has made the design piece easy. All of those manufacturing scale issues to me become a huge bottleneck. Right. Another bottleneck, of course, is memory. Right. You know, memory bandwidth. Right. And, you know, for that, as I've described, HBM is a hideous memory. It's just the best one that we... You know, it's a terrible memory. You know, I get terrible bit density. I'm limited to shoreline bandwidth, the power issues, thermal issues. DRAM doesn't like to be hot. You know, it's terrible to give... AI is a memory-compute workload, yes. Right. So, right, you know, for that, so I, you know, I need to, you know, compress that development and embodiment into real scale. You know, I need to fix memory associated with it. So to me, that's another critical bottleneck for us to attack. And power, right? You know, power and, you know, thermal, and most of the power simulation aspects today are bad. Right. You know, you don't have good 3D modeling tools for power, power dissipation, hotspots, et cetera. You know, my voltage in, right? You know, I'm essentially guardbanding almost my full power envelope, correct? Right. So essentially I'm losing, you know, 40% of my power in guardbanding. So I got to get much better at power modeling, you know, and then my power dissipation techniques are bad, right? So, you know...
Delivery model has to change too.
Yeah, so all of these things, you know, to me, those are the, you know, the next big bottlenecks, right? You know, how do I, you know, improve each one of those? And now, right, you know, if I've done that, okay, now we're on to an exciting era for what's next. So, you know, thank you for those great AI tools, but, you know, I got big challenges even when those are fully deployed.
And so we've seen sort of a rebirth in the number of chips and chip architectures and so on and so forth. Thanks to these workloads.
Yeah. It's like there's 100 AI inference accelerator chips, and I'm sure I don't even know them all.
Yeah, exactly. Yeah. So, so...
Why are you guys funding so many of those anyway?
You funded your fair share, too.
Yeah. So, so are we. You know, they're all, to their credit, making different design assumptions, either in the components that you talked about or in the part of the Pareto curve that they're targeting. So do you think there's room for all of these, or are we in a world where, partly because of what Guido talked about, AI tooling, we now have the ability to build more different kind of chips? And the TAM is enormous whichever way you want to... Even more provocatively, historically there have not been 100 competing processor vendors in any industry ever, right? So is this a more heterogeneous future? Will we see... Is this a temporary thing? It'll converge back to a few.
Yeah. And, you know, I see it as a more temporary thing, and it will converge, is what I expect. And there's probably three different reasons that I see that to be the case. You know, one of them is, you know, this emerging heterogeneity that you see in the compute environment where, okay, now I have specialized prefill versus decode. Well, you know, and now people are saying, well, we really need a specialized tier for midfill, not just prefill, right? You know, so now I have early prefill and now I have midfill, which is, you know, differential. Maybe speculate and verify if we listen to OpenAI here, right? Yeah. You know, and, you know, all of a sudden my compute fleet becomes more and more granular across the workload. And I think anytime that you've seen that in history, it's not sustainable. And then all of a sudden, right, when, you know, as you look at that and now people are saying, well, as I go to reasoning models, I want something that looks more like a CPU again. So, you know, all of a sudden I didn't want that level, you know, that didn't become the predominant portion of my compute capacity. I needed more of it over here, right? And model and model workflows and, you know, so on, you know. So I really don't like, I'll say, at-scale specialization, right? Because I think the workloads are going to continue to moderate, you know, and migrate so significantly. First, you know, second is I think today's models are about to go through some evolutionary breakthroughs as well, right? You know, I think the LLM is sort of reaching the limits, and now as people look to how do I really model 3D, right? You know, where a flattish LLM doesn't work well when you go to molecules and chemicals and, you know, imaging, you know. So I think we'll see limits of that that'll cause different shifts in the algorithmic domain. I also think that some of the most interesting workloads become where you start, you know, I'll say, bringing HPC-like things back into AI-like things, where all of a sudden, you know, 64-bit precision counts again, right? So I see...
What would be an example of that?
Well, you know, imagine that my AI model is now inducing the five most interesting domains for a chemical analysis, right? Okay, those chemical algorithms are going to run with high precision, right? You know, which is going to be the dominant piece of the workload, right? Getting to the five algorithms I need to run or running the five algorithms, right, on different chemical or biological systems. So, you know, I do think that, you know, or as we optimize our AI systems, they will lead us back to some of the things that have been more traditional high-performance computing. So I see that aspect of workload as well. So, you know, all of that said, is, you know, this extreme specialization to me in the compute architecture, I sort of don't like it because I don't think it's going to be what the workloads will look like two, three, four years from now. So that's one reason I don't see all of these AI chips, as they get more and more specialized for portions of the compute workload, to necessarily be right. You know, second is, right, there's like a hundred of them now, right? You know, there's multiple optical ones, there's probabilistic ones, and so on. And, you know, they're not all going to win, and they're not all going to win because at the end of the day, you have to get scale on these things, and scale requires, you know, you have to win, you have to get capital, you have to get workloads onto it. So I think it sort of defies logic that you're going to have a hundred of these things. So I see them, you know, coming back as a result of, you know, simple, you know, capital, market share, et cetera, you know, which are the winning teams, you know, winning designs, winning architecture. So I see the workload driving that, you know, I see the natural, you know, industry consolidations, you know, that will occur. And then I also expect that the winners will pick some winners, right? You know, where, you know, an OpenAI, an NVIDIA, Anthropic will say, I like this one because it isn't just hardware, it's how do the hardware and software code evolve. Yeah, right. You know, for it. And I do expect that there's going to be, you know, here's this ten I could pick for this. I like that one. And it does require investment in the software and the workload evolution to take advantage of those platforms. So for those three reasons, yeah, we're going to see, you know, a narrowing of the field in that sense.
And the deployment complexity as well. Sorry, go ahead.
I mean, makes total sense, and I think I agree in most points. Let me still try to stir a little bit of controversy here, just to stir a little bit of controversy. So you guys have changed it, right? So obviously 100 companies won't exist, consolidate, right? And completely agree with your argument that at the end of the day, the large consumers will pick winners to some degree, right? Today, I think to be a first-tier chip startup, to some degree, whether you're that or not is defined by how much traction you have with the really big ones. That said, and there's also an argument to say, like, look, in any kind of transition period, you probably have a bit more of an explosion of different approaches over time that'll shake out. Now, all that said, I think there's an argument you can make that basically says, look, historically, we standardized on, you know, one or two architectures because, you know, different instruction sets were hard. We had to build multiple compilers, you know, like this basically, the software complexity dictated that you can only have a certain small number of hardware platforms. That part seems to be changing with AI, right? Today, you know, writing an optimized kernel for, I mean, I think it was a great example where they basically said, look, humans can't really program this thing anymore, but an agent swarm overnight can do it. That's all we need, right? So where basically you can say, like, if you can build a chip that has a certain capability, it's now much, much easier to build the software layers on top that fully leverages that. Could a result of that be that we have a little bit more heterogeneity just because, you know, it's easier to program that heterogeneity going forward?
And I do think it changes some of, you know, where the bottleneck lands, right? Changes in that regard. You know, the first bottleneck is the one I'd point to that we've already talked about. Still takes a year and a half to build these things, right? You know, and then the second bottleneck is not only does it take a year and a half to get these things into racks, the scales and so on like that, now to make them useful at full scale. Okay, well, could I borrow 50 billion, right? I mean, you're not talking about small capital expenditures. You're talking about very large capital expenditures and data centers, you know, power, you know, commitments and so on like that. Well, I got to build this thing at scale, right, associated with it. So, and that's going to defy a lot of heterogeneity as a result. Now, you know, clearly big players, and we saw this with what, you know, NVIDIA did with Groq, okay? You know, we're going to sort of own some of that heterogeneity, right? And, you know, two years from now, you're not going to tell what Groq is inside of NVIDIA, right? It's just going to be like another, right? You know, something like that. So you'll see the architectures, you know, sort of hide the heterogeneity, right, inside of it, which to me is just fine, right? You know, I mean, that's what layering, that's what abstraction has always done associated with it, you know, but I think those forces are powerful ones to, I'll say, expose a lot of the heterogeneity. No, right? I think it will get abstracted underneath, you know, some of these, but at the end of the day, it's still hard to build these things at scale.
Earlier you talked about HBM being a hideous memory.
By far the best one we have today.
Exactly. It's like the thing about democracy. We can't recall when there was the last memory innovation. It might have been Optane. I think you probably had something to do with it.
Yeah, that one died. So...
Yeah, we killed it. We killed it. So, do you think memory innovation is around the corner, or what needs to happen for... Obviously, we understand the scale problem because it's even more larger there. Yeah. But if you keep the scale aside, even from a technology point of view.
Yeah. And let's, you know, look at that history. I've probably personally been associated with at least five different new memory architectures. Optane just being one of them, right, you know, that did not see the light of day, right, as well. I'm probably familiar with close to a hundred that have happened in the industry over the last 30 years. And exactly how many major new memories have you had over the last 30 years?
Zero. Right. You know, which, you know, okay, DRAM, SRAM, flash. Okay, what else is there? DRAM, SRAM, flash. Yeah. You know, so I do think...
Flash stacking counts a little bit.
Barely. Yeah. Right. You know, but anyway. Right. So, you know, in that, it, you know, it has been, right, you know, a disappointing... that we haven't created, you know, the next physics that enable, you know, low-cost, high-performance, resilient, manufacturable memory, right, you know, for it. So I do think, and now, and over the last 30, 40 years, you know, memory was a terrible industry, right, because you would make money one year and then you lose money for the next four, right? So let's go do R&D in an industry that makes money one out of five years, right? You know, it's just always, you know, it's just been so terribly difficult because it's been driven by such violent commoditization cycles.
That has changed a little bit, though, recently.
Over the last five years.
So all three big memory vendors are in the top 20 most valuable companies on the planet.
Yeah, that's a crazy change, isn't it?
Oh, it's just so, right. You know, who would have thought, right, associated with it. And anybody who looks at the AI workloads, and it's always, you know, it's much harder to, you know, it's easy to predict the past, but it's a little bit harder to predict the future, right? And, you know, in that, you look at it and say AI is a memory-bound workload, right, you know, inside of it. So I think there is the combination of capital and technology needs that will justify memory innovation. Now, there's two domains, things that I'm excited about, you know, for it. You know, one is, you know, this idea, you know, I do think that the shoreline bandwidth limitations of HBM is a fundamental issue, right? So with that, the idea, you know, and obviously, you know, we have a company, D-Matrix, that's doing this, but I think others will be, you know, whether it's, you know, Cerebras, you know, and their techniques...
The future is stacked, yeah.
Somehow you got to bring memory and...
You know, you got to bring memory and compute together, right, you know, in a fundamental, into new structures. I do think that given the workload now has such capital being deployed against it, you know, in the AI space, that I do think we'll actually be able to break through some of those physics challenges as well. So I do believe, and, you know, I've just invested in a new memory company as well that's still in stealth, but, you know, I'm quite excited about some of the innovations that will occur in new memories. And you're going to see new materials, you know, for memory. People, you know, a lot of people looking at things like ferroelectrics, you know, finding non-capacitive, you know, high-density memories, finding memories that can, you know, be high-performance, high-density, and stackable, right? But, you know, people are looking at low-cost flash, you know, for that. I'm not particularly enthusiastic about that idea myself. I think, you know, speed and performance of the underlying cell are somewhat problematic with a flash or an MRAM structure, you know, but I do think there's memory innovations are coming for the first time in 30 years, and I do think that's going to be an exciting space, and, you know, we'll have a couple of companies there, I'm sure.
Hopefully we'll do a few of them together. What do you think? Should we do... Okay. You know, but we'll find some things to do here. But I think memory innovation for the first time in 30 years is nigh upon us.
Yeah. Yeah. It's amazing what a couple of $10 billion of market cap... I don't even know. You say a trillion. Okay. Sorry.
You know, I think the memory industry has now increased by market cap by $2.5 trillion dollars. That's great. You know, over the last four years.
I think that's about right. That is... Yeah.
We could have just invested there and not done any foundational stuff.
Yeah, exactly. Use your money. Um, how long... what's your view? Sorry, go ahead.
Yes. To expand a little bit on the stacking part, if you have to predict, how quickly will our chips get taller? Is this going to be like, you know, we're going to go to two and then sit there for a while, or will we have, you know, the 16-layer sandwich, you know, happily mixing logic, memory... pulled down this HPM is going down at Nvidia, but how do you see this evolve?
