Mark Papermaster, AMD CTO / Ian Interviews #47

Yeah, it's a pleasure.

Well, I love that you go back to when we met in Berlin, right? At EPHA, so almost 10 years ago. And it's a perfect capture of our journey because you think about it, the work up to that point was that we had done. So I joined late 2011. Two or three months later, Lisa came on. I ran engineering. She came on to run the business and then she became CEO two and a half years later. And so people didn't realize in that point up to when you and I spoke of all of the building that had to go on to build, I'll call it, a chip infrastructure that could be there for the AI era. And so when you and I spoke, I don't think most anyone believed that we could be a player frankly in the industry. And what I loved about when you and I chatted is you were very receptive, you follow the industry and so you were asking like, well why do you think this new CPU Zen could compete and we chatted about that and we sort of talked about the different segments of the market and where it could go and in fact it did play out that way and we've had a chance to stay in touch over the years. So when you say what has changed in that period of time is visions turned to reality both of the industry and certainly what the game plan that we set out to do at AMD. We set out to be a very agile competitor offering not only CPUs and GPUs but other key hardware and software IPs to be able to be a solution provider that could listen to customers, take their input, react quickly and help provide very optimized and therefore provide customers value and that's what we've done and now have an incredibly broad portfolio. And then what's happened in the industry since you and I spoke that vision played out a little bit different. I mean in 2016 we probably all thought that the meta world would be more advanced than it is.

Yes. So 2016 we thought that was probably a bigger segment than it is but we were very much, if you think about my talk then, I talked about how disruptive AI would be. It's been all of that and more. So I think when you look back in that period of time, for the most part, our vision of the industry, our vision for AMD has played out as we thought but moved even more quickly than we imagined.

Mhm.

Well, if you think about what we did with the first generations of Zen, so we launched Zen in 2017, came out in EPYC servers, it came out in the line of Ryzen PCs, and the first two generations, you call it a smoke screen. I don't use that analogy. I call that our proving ground years was the first two generations. Is AMD for real? Is the value they're providing real? And will they be a sustained innovator? Will they show up every cycle? Right? Are they that true competitor bringing leadership performance or are they a flash in the pan?

Well, we did have our pedal to the floor the entire time because you can't do anything in this industry but move at your absolute best pace. And so what you realize is when we launched Zen, we had banked the turnaround of AMD on having not just a competitive but a leadership processor. So it took three generations to hit the knee of the curve on our Zen processor capabilities and so we were moving as fast as we can. So we took what had been a core, if you go back prior to Zen, it had big gaps of competitiveness. So first generation added 42% instruction per clock. I remember that presentation, it's one of my key memories. So it brought competition back to x86 CPUs and then we've had double-digit IPC gains, significant, you know, 15 to 20 percent IPCs every generation. So again no slowing down and by the time we hit third generation everyone realized wait a minute AMD is a different AMD. They're here every generation they're delivering on exactly what they promised when they promised. And so that's when the knee of the curve, that's when our market gains started to really grow. So it was Milan was the third generation in EPYC and likewise with the equivalent generation of Ryzen and PCs. That's a turning point for AMD because that's when in the industry not only of course the technology capabilities but the financial results that came with it. Everyone saw the knee of the curve and that's when we had the credibility and of course that led to the rest of the story because we had always planned to start with CPUs because there's no barrier to adoption x86 is x86 if we delivered leadership it would sell and people didn't see that we had a GPU effort that was running in tandem with that to grow both gaming graphics with Radeon and then of course we'd always been very focused on AI and bringing GPU compute to market.

I love that you asked that question because you've asked me that in every interview and I expect everybody asks you that in every interview. It's a common question. And I'll tell you I'm so, I love the question because I never cease to be amazed at the innovation of our engineers. Our CPU team is now a story team. It's under the technical leadership of Mike Clark. He started the definition of Zen. He coined the name Zen. Actually, it's called the Zen Daddy. So when you look at, but it's not just Mike, it's the whole team around it are incredibly committed and innovative. And so I could not be more excited with where our CPU roadmap's going. Yes, we're getting less on every technology node. But it's changing. I mean we're already leveraging every node we have an incredibly deep co-optimization with TSMC. So when you think about 2nm, we were the very first customer of TSMC, the first tape out that they received in 2nm, we highly optimized around that and so although we're not getting the same energy efficiency gains we are getting density gains and so that's very very important to drive TCO forward so we're leveraging that. So you're going to continue to see yet more density of cores. But you can't just bank on density of cores. You really have to think about how you're architecting to keep that balance of CPU computation of memory and IO. How do you keep that balanced engine moving forward with yet another swizzle? Now, how do you make sure when you did that you have really efficient multiply accumulate because guess what AI is everywhere and when everyone has to have fleets of CPUs because you have all your legacy workloads. It's not like they all go to a GPU, they're going to be around for ages and we are very very excited about how we have modified our AVX engine. So your listeners know we have a dedicated AVX engine. So we already run native 512 bit wide and it's already a good inferencing engine with what we do. It supports VNNI. Then going forward you've seen we said publicly in our work with the x86 advisory group that we have what we call the ACE engine. It's an advanced inferencing capability very very flexible that we're building in Zen 7 and beyond. So super exciting future of CPU computation not just of performance per watt per millimeter squared but also of how you blend in inferencing capability and how do you make sure it works seamlessly with the GPUs for heterogeneous computing.

Well, yeah. I mean, of the latter, many decisions that we looked back and said were the right ones. There's a few that were wrong for sure. But the biggest thing that we did right, and we've talked about it in the past, was that investment even before we started Zen core, it was that investment in what became publicly we call it our Infinity Fabric, our network on chip, our link technology. And it was game-changing for AMD because when you think about what we do as a chip company, you could build the best CPU, you could build the best GPU, you could build the best audio and video acceleration and fixed function engines. You could do all of those things, but if you can't connect them to where they can scale seamlessly, you can hook them up so that you don't create bottlenecks. You can hook them up with flexibility because customers need widely different configurations. And that was an investment that we started in 2012. We've grown it. We've had multiple generations. We're on our fifth generation of that fabric. That's been massively enabling. And at the time it was hugely disruptive. People quit the company. They felt it was the wrong way to go. You know, we had top leaders just get up and leave.

Mhm. So that was an absolute decision that was game-changing. And I don't know if we could have made it had we not been in a position where we had to make big bets. And so when Lisa and I were brought into the company, there was a number of big bets we had to make. That was one of them. Chiplet technology was a huge huge corporate bet. It's paid off incredibly handsomely. In 3D stacking, we're still the only one in really mass production of 3D stacking. And think about look at a gaming chip that has our SRAM or 3D V-Cache. I mean for four years it's gone unchallenged because of bringing that memory locality. So big decisions like that. Not everything works. But we're very, I'll say fortunate but you make your own fortune. We have very thoughtful high-level design. One of the things I've done from an engineering culture is support what I call healthy contention. I'd rather have healthy contention and people throw rocks at different ideas and we can debate it and we can yell and shout a little bit but we keep it professional and we make better decisions. So we haven't had huge misses. We've had where we've found things we could do better and we get it the next train that leaves the station, the next generation. But largely we hold that culture sacred and it's kept us in good stead. So if you notice we haven't had any, you think about our microarchitectural visions, we don't have any that we've had to say oh we were just kidding we're not supporting that right and which a lot of companies do.

So, no, I think we've done a good job. The misses we've had, we learn from. If we had something that caused a spin of silicon, because a late breaking bug, we try and absolutely minimize that. But we've had issues that we discovered late. We run a deep lessons learned on it and how do we fundamentally change our methodology so that we don't repeat that. So, again, that's built in our culture.

Yeah, we absolutely had some issues on first pass but guess what we worked around each one of them. That's another thing is you have to build in resiliency. So of course we have reliability, availability, serviceability architected in for field robust operation. But what people don't realize is we have a ton of diagnostics and fungibility built in so that if there is something that got through that we can work around it. Either we work around it as we're testing and bringing up our product or if it's found late and it's in silicon that's the production level silicon that we can have a very robust patch that's production patches so it's not, you know, from a customer there's no loss of functionality.

So our GPUs most every cycle the A0 is production target because GPUs are imminently programmable. So they're forgiven that way. CPUs we've typically always designed with two passes in mind. But that's changed going forward for two reasons. One, it's immense cost to build a mask set. I mean when you have so many millions of dollars going into every mask set that's only part one. Part two is building the test samples because you need to really test across thousands of samples also immensely expensive so we need across our whole portfolio to have first time right. Great news AI has now come along and so what we're finding is that we're making great strides with AI of course across physical design but where most excited is actually design verification. We're getting great agentic processes that improving our coverage. We're finding bugs earlier. We've invested years ago in deep emulation technology. Very important for us to get our chip designs right, but also our FPGAs are used. So we're eating our own cooking with the FPGA side of the portfolio. So huge focus going forward. First time right.

Yeah, it's a great question. It's a mix roughly half and half. So, what do I mean by that? If you look at the synthesis flows and the place and route flows, those we absolutely deploy our EDA partner solutions because they're heavily investing there. But what we find the other half where do we have to really invest? It's our knowledge. It's the agentic flow. So, it's how you take an end-to-end flow of chip design and how you take the smarts that you have. We're a 55-year-old chip design company and we have the most advanced packaging capabilities and some of the most advanced thermal cooling power delivery. Those things impact the design itself. How are we able to 3D stack? You don't just 3D stack, you architect for 3D stack, you have to ensure you can. The multi-physics wasn't there when you did it. That's right. So we had to create capabilities to make sure we didn't have hotspots. And things that ANSYS and other tools can handle today could not be, were not there when we started that. So where we're cutting edge, we'll develop capability that we need and then going forward deploying AI we're going to use the best of what the industry provides but it's our agentic flows that we're running with the knowledge capture that we have over many years.

Well, I was so lucky to start when I did. So, you know, date myself here. It was early 80s when I graduated and started with IBM. And I started with a small renegade team that was tagged to create IBM's first NMOS and then two years later their first CMOS AS/400. Well, there was no EDA industry and our chips were... Not quite. We've gone beyond Mylar tracing. It was a true CAD/CAM design but our customer at that time was IBM mainframe which was a behemoth in the industry and just drove so much of the company's revenue and so we had to innovate very very quickly and what folks don't realize at that time mainframe drove tremendous advances in technology. So I learned creating multi-chip designs back then. Having to do advanced circuits. One of the, I started as a circuit designer and we had to do advanced circuits.

I started as an analog person then quickly went into digital and microprocessor design and led Power microprocessor design.

It is a weird magic. It's a black box of magic for sure. But what it taught me is that you can never lose sight of the physics. And so even though you have all the advanced analysis tools and things like that, it always came back to does the physics make sense? What problem are you solving? Is the science behind it sound? It starts with that root in circuits designs. They were very, I'll call them high-speed interfaces back then but you wouldn't, they were huge block delays that we used to do. I mean this is 180, 1.8 nanometer when we got there it was like an incredibly advanced design at that point and so we learned a lot but what it really taught me from early on is the importance of cross-disciplinary innovation. So we were tackling back then that was the dawn of the EDA industry so it was really bringing advanced tooling and automation into chip design and because I told you it was a small renegade group we had to work closely with the fab lab. So co-design with the foundry to me that was innate. I learned that in my earliest years I was probes down on wafers taking measurements. So the lessons from the 80s and 90s couldn't be more applicable than today. Think about why did bipolar change over to CMOS because bipolar CMOS drove too much power. Here we are running up against power limits again and having to innovate through those walls. But the lessons I took away, Ian, is anytime someone tells you we're dead. There's a barrier. Moore's law is dead. This is dead, that's dead. Guess what? Innovation rules. We find ways to move around those barriers and to provide customer value.

Exactly.

Well, actually I just saw Philip just in the last couple weeks and we chatted briefly about this. There actually is starting to be a return to, it was everything had shifted to software. We are now seeing more students interested in hardware but Ian it's completely different now. What we're seeing and we are promoting this hugely at AMD is a different type of chip design approach. It's an AI native approach. It's thinking about chip design with AI being your right hand. I mean just like your phone, you wouldn't go anywhere without your phone. It's an indispensable tool. AI is a tool. It's a productivity tool. And to reimagine chip design AI native, AI from the outset I think is going to be incredibly disruptive over the next five plus years I think how we practice chip design today is going to completely change. First time right of course everything will be first time right. We're going to have with the kind of computation that we're building up to be able to do AI training that same computation can do AI for science. Very high precision simulation, digital twin design and as we leverage AI and AI for science into our design processes, it's going to disrupt the industry, we're going to be able to get a much tighter marriage of silicon to algorithms, going forward.

Well, that's why we continue our investments on general purpose CPUs and GPUs. People don't, I think people always jump to the conclusion that hey, there's a new approach. Hey, look at this custom design approach and has this merit. It gets 40% more performance on a given kind of workload. And so they think, well, everything will shift there. No. For the very reason that you stated, there's a constant change of algorithms. So, you're always going to need general purpose computing.

Well it's a constant process right so if it stabilizes even for a two to three year period you're going to more than make up the tailoring that you could do that's why at AMD we believe that the tent is so large in terms of opportunity to provide compute solutions there's room for tailored specialized solutions even while you need the general programmability going forward. And so that's what we're thrilled with our portfolio. So we provide the general purpose CPU and GPU. We have FPGA so you can prototype and adapt to the latest algorithms and for select, we're not in the general ASIC business but for our big customers that need to do a tailored version of silicon we're happy to do that and leverage the advanced packaging and foundry partnerships we have and bring that to bear.

Well, it's changing. So first of all as CTO I work with all the teams. So it's very fortunate to have that role and when you look at the skill acquisitions we've done, we're very focused. So most of it has been AI, AI model development. We did a set of skills that we needed with photonic and optical development. But the biggest acquisition this last year was ZT Systems which gave us true rack scale design and you don't have to look beyond the next year's Helios rack. We've showed it off at OCP this year and you can just see the benefit of that tight partnership. The compliments we got from our customers, our competitors that came in and looked at it said, wow, that is a true co-design. That's what it takes going forward. So already for years you have to have very very tight co-design of us as chip designers with the foundry. Now you need it for true AI training and large model inferencing you have to have that same partnership with the top hyperscalers and model developers. We've earned that seat at the table. And then to optimize you have to be skilled all the way through to the rack design. And so for the biggest AI compute capabilities, we now provide rack reference designs. We're not going to compete with the folks that build all those racks, but we provide fully optimized rack reference designs.

They were. So what we did, we were on a path to acquire. It takes about a year to close these things. So what we did was created a set of agreements. The way it works, they have to be able to survive. If the acquisition doesn't go through, you have to be able to part ways and you had contracts that handle all that. So we put all that in place, created that deep co-design partnership. And we did successfully close. So now...

No, we had already been working. So go back in time. And we already had a relationship with ZT for years. We got a strong appreciation for their design capabilities and their manufacturing capabilities. So we already had agreements in place. Once we set ourselves on a path to acquire, we put deep collaboration agreements in place. Again, survivable if it hadn't gone through. But so yeah, what you see in Helios is a result of true co-design.

You can say money but what is money? Money is having a leadership AI cluster design. I mean everyone knows that the industry is growing so quickly. So the carrot out there is huge.

Well, it drives just like I said earlier where you think about some people will look at and say, well, it's going to be a barrier like we're clearly going to hit a power wall and innovation will stop. No, it drives different innovation. So, I mean, here we are shipping 750 watt GPUs today on our way to kilowatt and 2 kilowatt designs and beyond going forward. So it's spurring tremendous innovation. I've met just in the last two weeks with startups with university research going on on heat spreading heat removal technology. Look at IEEE Spectrum. The lead article last month was on a diamond heat spreading technology. That's pretty amazing. So there's a lot in the development pipelines that will help here. On the immediate term what you're going to see those that need dense clusters will all adopt a tightly integrated liquid cooling.

So that's becoming a de facto standard for dense racks.

It's a problem to overcome. But again, it's not a bolt-on. You have to co-architect it.

Yeah. Oh my god. Those mainframes that I referred to earlier. They each shipped sat on a pogo stick which was a heat removal thermal conduit. So I mean now can you afford that? I mean if you take inflation into account that's a pretty expensive design approach. But it just tells you again engineers innovate, engineers solve problems. And we're going to and the new approaches will be yet much more economical than back in the 90s to do equivalent highly efficient heat removal.

Today it's much more modular than it was at that time. We're committed to modularity at AMD because it expands our addressable market. If we think about if we tailored the GPU and the CPU compute core only for data center computation and we didn't think about how you could derive it in a facile way to support enterprise needs, edge computing all the way down to PCs, we're running 100 billion parameter models on AI PCs today and those modification of those devices are out there for embedded edge applications. And so from our standpoint thinking end to end about the range of applications as we architect from the beginning. So it's another, think about an optimization matrix. You have a vertical optimization you're doing and guess what for the highest end it is rack scale. So we have people that wake up every day, it's ZT Systems engineers, it's our data center engineers all the way down to our chip design and IP designers. You create one part of the matrix that's vertically oriented and you want those folks just thinking about how do I get the best rack scale. Then we have other people who are thinking horizontal and they create that healthy contention that I talked about and said, hey, I love what you're doing for rack, but if you did a slight change, we could derive that in a very facile way. It's a hidden gem that people don't realize. Again, it goes back to when I talked to you back in 2016 and talked to you about the importance of modularity even then in our approach. It's really allowed us to address a much broader market and it's been a huge part of our history today and it's an even bigger part of our opportunity going forward.

The way to think about optics is we've been all talking about it for many years. Why is it only prevalent in sort of scaling out massive clusters in data center not creating scale up the densest cluster? The reason has been cost. Copper has done the trick. You can build, you know Helios is a 72 GPU design, copper, cheap, reliable, works great. It's all that you have to architect in the reliability and we have. But when you do that it's a fine solution but as you go beyond 72 node clusters and you start really getting the scalability to thousands when you think of mixture of experts type approaches which would be prevalent for some time until the next huge innovation, you want bigger and bigger cluster sizes. Good news is photonics in the next several years will tip over to be economically viable. The supply chain is coming up. You have multiple competitors providing solutions. So we're very excited about offering again where you need very very dense nodes. You will see photonic solutions phase in but it doesn't mean copper goes away because you have such a range of solution. Think about all the enterprises out there that do not need thousand node plus foundational model training, they're running largely inferencing. So think of photonics is coming on board in the next three years. You think about the supply chain whether it be laser development, the fiber attach units all the pieces are coming together in the next several years. And you should think about it as starting at just the biggest clusters but then over time becoming more prevalent.

No, you can, it's all about how you architect. You can architect for both. I talked about those fiber attach units just a moment ago. And so there's a lot of innovation going on and so it's just how do you architect your photonic solution for co-packaged optics? Do you have it, you know, and there's a lot of decisions I'm not going to get into all the details but there's things you can do to architect in flexibility. It's most of the discussion today is chip to chip, but we're obviously seeing some solutions talking about well why not do optics to more banks of HBM sort of chip to memory expansion.

We follow it all. I mean Celestial.ai has just announced that Marvell will be acquiring them and they...

Yeah. So if you're talking to Matt and team, you'll learn more and we think that's a fine solution and congratulations to David and team for being acquired by Marvell and I think that will get them more speed and they're working on solutions like that. So again AMD is all about an open ecosystem. We encourage all of these type of innovations because it gives our customers more choice and at AMD we're not about a solution that's proprietary and it's all AMD top to bottom. For us it's about doing what we do best getting the best compute elements out, a diversity and having there be multiple partners that can piece that solution together. So to us that's a great advancement. But you can also look at several other optics providers out there. So our customers are going to have choice. The cost will come down, the supply chain will come up. Next three to five years are going to be very exciting around photonics.

Yeah, great question. Well, we were already on an annual cadence of PCs. Yes. And commercial and consumer gaming and things of that nature. Our EPYC CPUs have been and really remain on 18 months. So but we knew what it takes to go to an annual cadence and so it is torid but the pace of AI is torid and so it's driven by customers who just need that type of improvement and adaptation whether it be new math approximation formats and things of that nature. It just needs to be on that torid pace and that's what we've done. So what we found we had to do though Ian is really understand the pinch points and so we've beefed up staffing in certain areas and it's where we're focused on applying AI into our chip design practices on the pinch points that would facilitate that annual cadence.

Yeah, it's a great question. How do you still innovate when you're on the treadmill of getting designs out so quickly? And I'll be candid. I mean, I've been engineers. I sit down and have roundtables with engineers all the time. What's on your mind? What's not working? And that's a common concern like we're working at such a fast and torid pace. How do we still innovate? And so everyone finds their way to do that. Some companies have said, well, we're going to have a half a day. Every Friday is like innovation. We haven't done that. I haven't found that to be effective. What we have found is you have to have innovation as built into the design process. So rather than rush that high-level design up front, we have pathfinding teams. First it starts in research. So we have a very strong research team and it doubled in size more than doubled in size when we acquired Xilinx. Xilinx has just had a rich tradition of having a strong research, strong university collaboration. Xilinx was also committed to open ecosystems and open source just like AMD. So that came together beautifully. So the pathfinding starts there. So if you think about the most advanced work on CPU, GPU, AI algorithms, it's in a five-year plus time frame from a research team, but we don't have just an ivory tower research team that's just looking at five years and beyond. Part of that research team is in a three to five year and they're actually co-mingled and working with the development teams are getting out the next generation Zen CPU, the next generation Instinct or GPU or next FPGA. And so that's a very very tight partnership. So it starts there with a well-oiled research and development pathfinding integration and then beyond that it's that high-level design phase where we want to give the teams the time to make sure that they have the right innovations to be a competitive and leadership design and we have a well-oiled roadmap process. It's one of the things that Lisa and I put in place back around the time when you and I talked. Around 2016, actually even prior to that it was probably 2014 that we put in a very very rigid, when I say rigid I mean rigid in terms of design groups bringing in the defined thought process of what is it we're putting this design that will really matter to customers and then is there a business team that paired with them and made sure that in fact that matches a market need so that's what I mean by rigid we give our teams the agility to innovate and how they put things together. But when we approve a project and it goes through, it's got to have that rigor behind it. And it's a necessity for us because we can't afford to be doing projects where you cancel x% of them. We need our projects to all deliver and return what they promised to the company.

2026 I think is going to be really exciting from a couple fronts. At AMD, we couldn't be more excited about what our next generation Instinct GPU brings because with that Helios rack, it's rack scale. It brings us smack dab into full scale training and inference. We support training today but it's to the thousands of node not to the hundreds of thousands of GPU that will have the capability to support. So we extend our portfolio but also 2026 from an industry standpoint to me is incredibly exciting because just like we're seeing with our chip design with AI really starting to have high impact agentic processes coming online. I'm talking to more and more customers on enterprise, broader enterprise that are seeing that impact and I'm hoping we start to see more and more consumer applications. So, of course, we all use our favorite large language model and ChatGPT or its competitors in our day-to-day lives. But I think in 2026 we start to see yet more use that we all have in our day-to-day lives where AI becomes absolutely indispensable.

Thank you, Ian.

Thank you and you too.