One of the first people I ever interviewed for this channel was AMD CTO Mark Papermaster. He has been a mainstay at AMD through thick and thin and has had an illustrious career as well. So, it's my pleasure to bring him back to the channel. Mark, welcome. Great to see you again.
If you like this content, there are multiple ways to support the channel. You can like and subscribe this video, and many thanks for doing so. There's also Patreon, which gives you access to our Discord. There's a merchandise store and a newsletter. Links in the description. For all of you who do contribute, thank you. You are keeping me well fed.
If I go back through my memory, and thankfully Theresa corrected me as well, we first met in 2016 at EPHA at a time when AMD wasn't the big behemoth it is now and we were discussing the ins and outs of the market at the time. It was very much a more casual chat. What would you say inside the company given that you now have Ryzen, you now have EPYC, you now have Instinct, you now have these major product lines driving the company forward. What's changed the most?
Well, I love that you go back to when we met in Berlin, right? At EPHA, so almost 10 years ago. And it's a perfect capture of our journey because you think about it, the work up to that point was that we had done. So I joined late 2011. Two or three months later, Lisa came on. I ran engineering. She came on to run the business and then she became CEO two and a half years later. And so people didn't realize in that point up to when you and I spoke of all of the building that had to go on to build, I'll call it, a chip infrastructure that could be there for the AI era. And so when you and I spoke, I don't think most anyone believed that we could be a player frankly in the industry. And what I loved about when you and I chatted is you were very receptive, you follow the industry and so you were asking like, well why do you think this new CPU Zen could compete and we chatted about that and we sort of talked about the different segments of the market and where it could go and in fact it did play out that way and we've had a chance to stay in touch over the years. So when you say what has changed in that period of time is visions turned to reality both of the industry and certainly what the game plan that we set out to do at AMD. We set out to be a very agile competitor offering not only CPUs and GPUs but other key hardware and software IPs to be able to be a solution provider that could listen to customers, take their input, react quickly and help provide very optimized and therefore provide customers value and that's what we've done and now have an incredibly broad portfolio. And then what's happened in the industry since you and I spoke that vision played out a little bit different. I mean in 2016 we probably all thought that the meta world would be more advanced than it is.
Oh, I'd always forgotten about that.
Yes. So 2016 we thought that was probably a bigger segment than it is but we were very much, if you think about my talk then, I talked about how disruptive AI would be. It's been all of that and more. So I think when you look back in that period of time, for the most part, our vision of the industry, our vision for AMD has played out as we thought but moved even more quickly than we imagined.
We've obviously had lots of discussions and I speak with the architects and every time we talk about designing a chip we have this two, three-year cycle or at least you have to think that far out but then you have five years and seven years of pathfinding and working with foundry partners for example.
Just going back to then, how much of a smoke screen do you think that the success of the first couple of generations of Zen was to where the projected nature of the business was going to go?
Well, if you think about what we did with the first generations of Zen, so we launched Zen in 2017, came out in EPYC servers, it came out in the line of Ryzen PCs, and the first two generations, you call it a smoke screen. I don't use that analogy. I call that our proving ground years was the first two generations. Is AMD for real? Is the value they're providing real? And will they be a sustained innovator? Will they show up every cycle? Right? Are they that true competitor bringing leadership performance or are they a flash in the pan?
But doesn't the before the success happened so to speak, you obviously have to project where you're going, right? And you've said with AI and everything going now everything's hyper accelerated but back then that this is why I say smoke screen right it's because at some point you have to put pedal to the floor. Was there any of that or were you able to sort of like accurately work with customers and partners at the time to model where you expected to be?
Well, we did have our pedal to the floor the entire time because you can't do anything in this industry but move at your absolute best pace. And so what you realize is when we launched Zen, we had banked the turnaround of AMD on having not just a competitive but a leadership processor. So it took three generations to hit the knee of the curve on our Zen processor capabilities and so we were moving as fast as we can. So we took what had been a core, if you go back prior to Zen, it had big gaps of competitiveness. So first generation added 42% instruction per clock. I remember that presentation, it's one of my key memories. So it brought competition back to x86 CPUs and then we've had double-digit IPC gains, significant, you know, 15 to 20 percent IPCs every generation. So again no slowing down and by the time we hit third generation everyone realized wait a minute AMD is a different AMD. They're here every generation they're delivering on exactly what they promised when they promised. And so that's when the knee of the curve, that's when our market gains started to really grow. So it was Milan was the third generation in EPYC and likewise with the equivalent generation of Ryzen and PCs. That's a turning point for AMD because that's when in the industry not only of course the technology capabilities but the financial results that came with it. Everyone saw the knee of the curve and that's when we had the credibility and of course that led to the rest of the story because we had always planned to start with CPUs because there's no barrier to adoption x86 is x86 if we delivered leadership it would sell and people didn't see that we had a GPU effort that was running in tandem with that to grow both gaming graphics with Radeon and then of course we'd always been very focused on AI and bringing GPU compute to market.
So you mentioned double digit gen-on-gen performance consistently and it's the question that gets asked every five years. But given Moore's law, given Dennard scaling, do you see a limit anytime soon in that raw single core performance or are we going to end up being workload dependent?
I love that you asked that question because you've asked me that in every interview and I expect everybody asks you that in every interview. It's a common question. And I'll tell you I'm so, I love the question because I never cease to be amazed at the innovation of our engineers. Our CPU team is now a story team. It's under the technical leadership of Mike Clark. He started the definition of Zen. He coined the name Zen. Actually, it's called the Zen Daddy. So when you look at, but it's not just Mike, it's the whole team around it are incredibly committed and innovative. And so I could not be more excited with where our CPU roadmap's going. Yes, we're getting less on every technology node. But it's changing. I mean we're already leveraging every node we have an incredibly deep co-optimization with TSMC. So when you think about 2nm, we were the very first customer of TSMC, the first tape out that they received in 2nm, we highly optimized around that and so although we're not getting the same energy efficiency gains we are getting density gains and so that's very very important to drive TCO forward so we're leveraging that. So you're going to continue to see yet more density of cores. But you can't just bank on density of cores. You really have to think about how you're architecting to keep that balance of CPU computation of memory and IO. How do you keep that balanced engine moving forward with yet another swizzle? Now, how do you make sure when you did that you have really efficient multiply accumulate because guess what AI is everywhere and when everyone has to have fleets of CPUs because you have all your legacy workloads. It's not like they all go to a GPU, they're going to be around for ages and we are very very excited about how we have modified our AVX engine. So your listeners know we have a dedicated AVX engine. So we already run native 512 bit wide and it's already a good inferencing engine with what we do. It supports VNNI. Then going forward you've seen we said publicly in our work with the x86 advisory group that we have what we call the ACE engine. It's an advanced inferencing capability very very flexible that we're building in Zen 7 and beyond. So super exciting future of CPU computation not just of performance per watt per millimeter squared but also of how you blend in inferencing capability and how do you make sure it works seamlessly with the GPUs for heterogeneous computing.
So going back a little bit to the beginning and you've been at AMD now 15 years and going through that Zen, EPYC, Instinct ramp. Has there ever been any situations where you've had, especially at your level, to choose between an A or a B and perhaps regretted the decision or actually realized it was really good we did this over that.
Well, yeah. I mean, of the latter, many decisions that we looked back and said were the right ones. There's a few that were wrong for sure. But the biggest thing that we did right, and we've talked about it in the past, was that investment even before we started Zen core, it was that investment in what became publicly we call it our Infinity Fabric, our network on chip, our link technology. And it was game-changing for AMD because when you think about what we do as a chip company, you could build the best CPU, you could build the best GPU, you could build the best audio and video acceleration and fixed function engines. You could do all of those things, but if you can't connect them to where they can scale seamlessly, you can hook them up so that you don't create bottlenecks. You can hook them up with flexibility because customers need widely different configurations. And that was an investment that we started in 2012. We've grown it. We've had multiple generations. We're on our fifth generation of that fabric. That's been massively enabling. And at the time it was hugely disruptive. People quit the company. They felt it was the wrong way to go. You know, we had top leaders just get up and leave.
I remember Mike saying that. Yeah.
Mhm. So that was an absolute decision that was game-changing. And I don't know if we could have made it had we not been in a position where we had to make big bets. And so when Lisa and I were brought into the company, there was a number of big bets we had to make. That was one of them. Chiplet technology was a huge huge corporate bet. It's paid off incredibly handsomely. In 3D stacking, we're still the only one in really mass production of 3D stacking. And think about look at a gaming chip that has our SRAM or 3D V-Cache. I mean for four years it's gone unchallenged because of bringing that memory locality. So big decisions like that. Not everything works. But we're very, I'll say fortunate but you make your own fortune. We have very thoughtful high-level design. One of the things I've done from an engineering culture is support what I call healthy contention. I'd rather have healthy contention and people throw rocks at different ideas and we can debate it and we can yell and shout a little bit but we keep it professional and we make better decisions. So we haven't had huge misses. We've had where we've found things we could do better and we get it the next train that leaves the station, the next generation. But largely we hold that culture sacred and it's kept us in good stead. So if you notice we haven't had any, you think about our microarchitectural visions, we don't have any that we've had to say oh we were just kidding we're not supporting that right and which a lot of companies do.
So, no, I think we've done a good job. The misses we've had, we learn from. If we had something that caused a spin of silicon, because a late breaking bug, we try and absolutely minimize that. But we've had issues that we discovered late. We run a deep lessons learned on it and how do we fundamentally change our methodology so that we don't repeat that. So, again, that's built in our culture.
I remember the story Mike was saying about how the first spin that came back from Zen one could only work under sub-zero cooling of some of the learnings had to be made at that point.
Yeah, we absolutely had some issues on first pass but guess what we worked around each one of them. That's another thing is you have to build in resiliency. So of course we have reliability, availability, serviceability architected in for field robust operation. But what people don't realize is we have a ton of diagnostics and fungibility built in so that if there is something that got through that we can work around it. Either we work around it as we're testing and bringing up our product or if it's found late and it's in silicon that's the production level silicon that we can have a very robust patch that's production patches so it's not, you know, from a customer there's no loss of functionality.
How important is it that a Zen has to work ready for commercial use and well I guess a follow on to that is how often do you hit that target.
So our GPUs most every cycle the A0 is production target because GPUs are imminently programmable. So they're forgiven that way. CPUs we've typically always designed with two passes in mind. But that's changed going forward for two reasons. One, it's immense cost to build a mask set. I mean when you have so many millions of dollars going into every mask set that's only part one. Part two is building the test samples because you need to really test across thousands of samples also immensely expensive so we need across our whole portfolio to have first time right. Great news AI has now come along and so what we're finding is that we're making great strides with AI of course across physical design but where most excited is actually design verification. We're getting great agentic processes that improving our coverage. We're finding bugs earlier. We've invested years ago in deep emulation technology. Very important for us to get our chip designs right, but also our FPGAs are used. So we're eating our own cooking with the FPGA side of the portfolio. So huge focus going forward. First time right.
So how many of those machine learning tools in design and verification are coming directly from your EDA partners versus being built internally?
Yeah, it's a great question. It's a mix roughly half and half. So, what do I mean by that? If you look at the synthesis flows and the place and route flows, those we absolutely deploy our EDA partner solutions because they're heavily investing there. But what we find the other half where do we have to really invest? It's our knowledge. It's the agentic flow. So, it's how you take an end-to-end flow of chip design and how you take the smarts that you have. We're a 55-year-old chip design company and we have the most advanced packaging capabilities and some of the most advanced thermal cooling power delivery. Those things impact the design itself. How are we able to 3D stack? You don't just 3D stack, you architect for 3D stack, you have to ensure you can. The multi-physics wasn't there when you did it. That's right. So we had to create capabilities to make sure we didn't have hotspots. And things that ANSYS and other tools can handle today could not be, were not there when we started that. So where we're cutting edge, we'll develop capability that we need and then going forward deploying AI we're going to use the best of what the industry provides but it's our agentic flows that we're running with the knowledge capture that we have over many years.
So what some of the audience may not realize is that you actually spent 26 years at IBM working on a variety of things including IBM Power and all the tools we've just spoken about are obviously very new. You know place and route whether you call that AI or not has changed over the years. But are there any sort of common threads you can draw from not only those 26 years of experience but also that have come through your career to today?
Well, I was so lucky to start when I did. So, you know, date myself here. It was early 80s when I graduated and started with IBM. And I started with a small renegade team that was tagged to create IBM's first NMOS and then two years later their first CMOS AS/400. Well, there was no EDA industry and our chips were... Not quite. We've gone beyond Mylar tracing. It was a true CAD/CAM design but our customer at that time was IBM mainframe which was a behemoth in the industry and just drove so much of the company's revenue and so we had to innovate very very quickly and what folks don't realize at that time mainframe drove tremendous advances in technology. So I learned creating multi-chip designs back then. Having to do advanced circuits. One of the, I started as a circuit designer and we had to do advanced circuits.
So you're an analog person.
I started as an analog person then quickly went into digital and microprocessor design and led Power microprocessor design.
Is that because you realized analog was a weird magic?
It is a weird magic. It's a black box of magic for sure. But what it taught me is that you can never lose sight of the physics. And so even though you have all the advanced analysis tools and things like that, it always came back to does the physics make sense? What problem are you solving? Is the science behind it sound? It starts with that root in circuits designs. They were very, I'll call them high-speed interfaces back then but you wouldn't, they were huge block delays that we used to do. I mean this is 180, 1.8 nanometer when we got there it was like an incredibly advanced design at that point and so we learned a lot but what it really taught me from early on is the importance of cross-disciplinary innovation. So we were tackling back then that was the dawn of the EDA industry so it was really bringing advanced tooling and automation into chip design and because I told you it was a small renegade group we had to work closely with the fab lab. So co-design with the foundry to me that was innate. I learned that in my earliest years I was probes down on wafers taking measurements. So the lessons from the 80s and 90s couldn't be more applicable than today. Think about why did bipolar change over to CMOS because bipolar CMOS drove too much power. Here we are running up against power limits again and having to innovate through those walls. But the lessons I took away, Ian, is anytime someone tells you we're dead. There's a barrier. Moore's law is dead. This is dead, that's dead. Guess what? Innovation rules. We find ways to move around those barriers and to provide customer value.
So, Moore's law is not dead.
It's interesting you bring up your experience like that because one person we both know, Philip Wong at Stanford, he's often lamented the fact that students today don't have the same opportunities or the jobs in the industry aren't being pivoted more towards chip design and physical design on that side. Is there anything, is there any opinion you have on ways that we can change that?
Well, actually I just saw Philip just in the last couple weeks and we chatted briefly about this. There actually is starting to be a return to, it was everything had shifted to software. We are now seeing more students interested in hardware but Ian it's completely different now. What we're seeing and we are promoting this hugely at AMD is a different type of chip design approach. It's an AI native approach. It's thinking about chip design with AI being your right hand. I mean just like your phone, you wouldn't go anywhere without your phone. It's an indispensable tool. AI is a tool. It's a productivity tool. And to reimagine chip design AI native, AI from the outset I think is going to be incredibly disruptive over the next five plus years I think how we practice chip design today is going to completely change. First time right of course everything will be first time right. We're going to have with the kind of computation that we're building up to be able to do AI training that same computation can do AI for science. Very high precision simulation, digital twin design and as we leverage AI and AI for science into our design processes, it's going to disrupt the industry, we're going to be able to get a much tighter marriage of silicon to algorithms, going forward.
Does that not create a risk of if the algorithms change the silicon is worthless?
Well, that's why we continue our investments on general purpose CPUs and GPUs. People don't, I think people always jump to the conclusion that hey, there's a new approach. Hey, look at this custom design approach and has this merit. It gets 40% more performance on a given kind of workload. And so they think, well, everything will shift there. No. For the very reason that you stated, there's a constant change of algorithms. So, you're always going to need general purpose computing.
