GAEA Talks Live from HumanX - How AMD Plans to Win The AI Era with AMD CTO Mark Papermaster

If we can't lower the cost to make AI more accessible, we're going to create tremendous divisions in society because that's how powerful this capability is. It's going to be the halves and halves nots. You have to have trust in a new technology. You know, Apple spent a lot of focus on the security of those transmissions and it actually went through a very controlled network so it couldn't be hacked or tapped into. I remember going back and forth with Steve Jobs on the angle of the camera. How are people even going to hold the phone to have the right experience? And we aligned on what that was and it worked pretty well. It's been widely adopted. It's just a dramatic change of increase of capability of AI. And you know, one of the things that I'm urging our engineers at AMD and our non-engineers at AMD is to embrace it because a change of that nature in a short time, it can be scary. It is scary.

You do. Excellent. I'm really looking forward to this one. You have a legendary background, but for very specific reasons that I think the audience will absolutely know. So, give us a little background who you are and what your journey has been to where you are today.

It's a pleasure.

That positions you in a remarkable place at this moment in time where the AI revolution as it is mutually understood across the world, compute is such an important dependency which perhaps things change for a while but it's back there at the forefront, the idea that hardware will drive a lot forward and I think it's fascinating with your experience with mobile devices as well. I actually interviewed one of the co-founders of Symbian OS, David Wood, learned a huge amount of what that moment in time looked like and there's definite similarities to where we are now and the idea that compute will end up being distributed again, centralized then redistributed through the innovation of new models.

I love your observation. What goes around comes around because we have cycled through centralization to distributing computing over the years. The mainframe was completely centralized. What happened with PCs and then workstations is it became decentralized. And you look at now in the AI era, not only is it going to be distributed, you're going to see how we run AI disaggregated where portions are run in the cloud, portions are run on your PC, portions in your mobile device, and frankly embedded all around us because almost everything we deal with is going to have a small AI processor embedded in it. So it tells you how AI truly will make technology pervasive. We thought it already was but now this AI revolution fundamentally changes how we interface with computing. We thought it was revolutionary. Think about the iPhone. Oh my god, it's so easy. It's just intuitive. I can just, you know, pull up the screen, touch and select and it's just very intuitive. Well, now that's going to be obsoleted because now it's the dialogue simply like you and I are having today is how we're going to interface with computing. I will be describing what do I want to do? It'll be an agentic process. It's going to translate those goals and suggest a set of actions. It's going to start running it. I'm going to make sure it's what I really want it to do. And away we go.

Well, FaceTime came out during my time overseeing the iPhone and iPod development at Apple and we knew it could be a real game changer of how people interface because think about it when you add the communication element of our expression of our eyes, of our reaction, then the bandwidth of communication goes up dramatically. It changes the whole experience. So, there was tremendous focus on how to have the right experience. I remember going back and forth with Steve Jobs on the angle of the camera. How are people even going to hold the phone to have the right experience? And we aligned on what that was and it worked pretty well. It's been widely adopted. But tremendous focus on the experience and the trust. You have to have trust in a new technology. And so it was important on FaceTime that Apple spent a lot of focus on the security of those transmissions and it actually went through a very controlled network so it couldn't be hacked or tapped into. And the reason I highlight that is it goes right back to the point you made in this AI era. Security and having trust in using these devices is paramount. So you said, hey, people are going to probably certain things they might only want to do on a phone that they haven't possessed. By the way, it's the same thing on the PC because we use our PCs for our content creation. It's bigger screen. It's just that interface. And likewise when you're running AI in the PC, it's just like the phone. You can decide when it's connected and what information you choose to transmit or not transmit. Those are in your control. And for us at AMD because we span AI computation and processing from supercomputing in the cloud to the PC to the edge and embedded devices. This is very, very important for us. So, we think that that kind of trust is sacrosanct. We have built-in encryption capabilities. So, it can all be encrypted. Someone walked away with your laptop and they don't have your codes and your passwords, it's useless to them. We actually do the same on the cloud. We have encryption capabilities that can be run there. It's called confidential compute. So you might be running some, you're a business and you have the same concern that you said consumers would have, a business might have that same concern and they can run their AI processing on the cloud yet they control the keys that unlock that encryption. The cloud provider can't possibly even see the contents. So you really hit on an element that will become more and more discussed because we're just at the very beginnings of people starting to have extensive use of AI in their personal lives or in business really banking their business, their crown jewels, their secrets being trained in AI models. So this theme of how do you trust what you're exposing to AI will take off and we are trying to get way ahead of that by ensuring everybody has the opportunity to control their information. They are the owner.

Well, here's the challenge. The challenge is that AI as you create autonomous flows, they're trying to help. They're trying to get the task done. And so if you didn't, as you suggest, if you didn't think through what is accessible, what information is allowed to fill in and what in fact must remain private, you could indeed have disastrous consequences. So it changes completely how we have to think about putting solutions together. We have to solve problems in a different way than we have before. And what do I mean by that? And how do we think about it at AMD? Go back in AMD's history. It's an almost 57-year-old company and it was about creating compute devices and you didn't have to think too hard about all the uses. It could be used in a plethora of applications and it would be used for its purpose. It'd be computing. Data would be provided to it, programs would run on it, the results would come out. It was our job to make sure those calculations were correct. You know that it was all performed properly. And of course we had to provide security on the device itself to make sure that device couldn't be hacked. But if you look at most of our history, we didn't have to think about the rest of the solution stack. We could focus on just that device. Well, that's different now, right? Because of the complexity, because AI opens up the solution space, because you can probe down to the lowest levels of computing, hardware, networking, storage, whether you're running on your mobile device, your PC, it's likely connected to the cloud and accessing data there. And so you have to actually as an industry come together and you do indeed need standards of how are we managing this. And I do see good progress, Graham, on managing the security of that information as it flows in that path. And so there are encryption techniques and other standards that allow you to engineer that security, but it's only as good as it's applied. And these new agentic processes, they're new and marrying the kind of capabilities of securing that data and controlling access to the actual AI flows that are being created. There are not standards yet at this time. So you're hitting, I'd say, an area that needs a lot of focus. It's in a lot of discussion I'm having with my peers where I talk to CTOs up and down the stack, the solution stack, we're all realizing that we're going to have to collaborate and to come up with those standards and marry the different pieces that we have in there to provide the necessary security. It's all there, but it's got to be put together. Those taking responsibility for leadership, which I think is fantastic that you're out there, you're talking about this, you're raising the concerns, you're promoting collaboration in so many aspects of the world of AI as it moves forward. If we all see life from one particular lens, to have very many views of the same thing from different angles can only enrich, help protect, help see the opportunity, help get full coverage of what the questions should be. Not everyone thinks that way. At AMD, we really do think about the ecosystem. It's why we have an open ecosystem. We build our compute solutions typically with open standards. Our software stack to run AI is open. People can make modifications. They can create a fork of that code. They can help improve that code and we find a number of like-minded companies and we partner with them because we do feel that it's an onus and the accountability has to be on us as an industry to do that collaboration because no one company can solve it on their own and it doesn't do any good when there's some, you know, imagine if there's a huge disaster that takes down your bank, that takes down your source of energy. What good does it do if one company's pointing at another like I did my part right? It fell in the cracks between where my part of the solution, my computer chip was being used with the software that ran on it or being run with how the total solution was being applied in potentially a less secure way. That's bad for the industry because again at the end of the day our broad customer set loses faith in technology if we're not providing a collaborative solution to keep that sacred trust. So we've done initiatives, we've got work that we've done at AMD that proposed a set of standards such that the security that we do on our chips can be communicated and authenticated as it moves down the food chain, the supply chain as it's being used. I'll give one example. We're working with Microsoft who is working on a set of safeguards around the software and those applications and we're marrying it with the safeguards and authentications that we do at chip level and it's not just a Microsoft AMD thing. It'll be a standard others can do the same. So, there are good efforts going on here and I'd urge any of our compatriots in the industry who are listening in on your show to if they're not already, please join in because it's our trust with users of the IT industry that's at stake.

Well, Graham, there won't just be niches anymore. I mean, agentic processes can reach out and pull in any aspect of a problem solving that you want to chain together. I mean, that's what it does. So again I go to the fact that no one can just operate in their silo. You don't get to just sit in your silo and solve your problem. It's not independent anymore. It's going to be chained together with other pieces of problem solving. The world's changed period. What we try and do at AMD is frankly not allow anyone to be in just a silo. I mean what we do, we apply AI every day to what we're doing. We're advancing our chip design. We're accelerating our chip design process by applying AI. We're a leader with our IT is how we support that. And so we're practicing what we preach. We're using our own cooking. We're looking for those kind of problems every day. Model development. We're not a model company. We're a compute solution, everything from small devices up to huge massive data center GPU racks, but we create our own model. Not to sell it or not to create a competition with OpenAI or Anthropic or the rest, but to make sure that we know that we have that expertise of where that needs to be optimized, where there could be risks in developing those models. We're not allowing ourselves to operate in the silo. We're making ourselves a user and a full-fledged member of optimizing an AI solution stack for the industry.

Well, at AMD, first of all, this year is a super exciting year because we've been advancing our AI portfolio. We were a leader in high performance computing, HPC. So, you look at the top number one supercomputer in the world, the number two supercomputer, many of the top 500 are based on AMD. But it wasn't until December 2023 that we took on optimizing not just for high performance computing things like modeling weather systems and weather forecasting or enzyme solutions, fluid flow dynamics. That's all HPC. In December 23 we brought competition to AI with our Instinct line of products and we've been improving it every year. Our software stack, our open software stack has improved to where it's now very robust. And this year we launched what will be a just a top-flight AI computer which can mesh together 72 GPUs and then be connected horizontally through Ethernet connectivity to just massive clusters. So we've extended our competitive reach to be from thousands of GPUs to CPU, these are all heterogeneous CPU and GPU systems but from the tens to you can think of even 100,000 GPU clusters but also what we're very modular in how we develop and so you can scale that right back to what does enterprise need. Enterprise have liquid cooled kilowatt processors, they need air cooled that drop in existing data centers. Well, we have that with our modularity. We have options that play to the needs of enterprise and again all the way down we think you're going to see more and more distributed compute and PCs. So into this year we launch, extend the range of AMD to the upper echelons of AI computation. And we really bring our open ecosystem and open software stack to a fully competitive level and it's getting very broad acceptance.

That's right. And it's almost like people forget the trajectory of how things happen over time. But we can look at the past and it's relatively predictable in that instance. Whether it's mechanized technology in the industrial revolution, whether it's the internet, whether it's this particular circumstance, there is that understanding of what that quantum will look like. And it's a huge need to have accessible modular compute.

It's essential because if we can't lower the cost and make AI more accessible, we're going to create tremendous divisions in society. Because that's how powerful this capability is. It's going to be the halves and halves nots. I mean already most tech companies are making sure, I mean anyone they hire, are you AI literate? It's one of the, it's not the number one question it's number two. So it's very important for society that we increase the access. So that's why all of our PCs are AI enabled and run the same software stack. ROCm is our name of our software stack, our open software stack that you run on our huge GPUs and CPUs in the cloud. And you can get our highest-end PC, it's called Strix Halo that you can buy at your retail store today. You can run hundreds of billions of parameter models on it. It's actually very, very capable and it's a huge step to make AI accessible. We're going to be driving that capability down into even more accessible notebooks. And I think that is point number one is it's a societal need to get AI computation out to the masses. So that's point one. Point two, you talked about the PhDs. Well, they can't practice their craft without access to compute. And so universities are needing to invest, they are investing making sure they have the kind of computation to train their students and in industry for tech industry, these top-notch PhDs that we hire they want to make sure they have access to the AI computing so we're showing them as we interview them they're interviewing us making sure they have access to the kind of computation they need so it hits exactly at the point that is underlying your comment that AI computation is becoming a huge enabler to what people can do and we need to make it more and more accessible.

Well, again, we talked about my journey and I'm very fortunate because my journey was years of IBM on how to create high performance computation and then I went to Apple and in the time I was there and the ability, you know, if you ask me what's the number one thing I learned having the opportunity to work with Steve Jobs is his maniacal focus on the experience. How do you make technology where it doesn't even, it's not some foreign technology? It's just an extension. It's simply an extension of how you're running your life and it brings joy and productivity to you. And that deeply affected me and you might think, well, how does that apply to AMD? You guys are creating computational solutions. We would create CPUs, GPUs, adaptable computing. But again, times have changed. We don't just create that device. These devices, we work all the way through the stack to enable the applications. Think about gaming on an AMD GPU. We're working with every game title developer so that it's a beautiful experience. It's optimized. You feel immersed in the experience. We want, if you're running on a PC, we work very closely with Microsoft on that whole Windows experience and it's a co-engineering of that solution to deliver that experience. Likewise, game consoles, we're the chip underneath the Microsoft Xbox, the PlayStation from Sony where we deeply partner to deliver that experience and now AI where it's going to touch everything. So we are co-optimizing across the stack to maximize the compute while we're minimizing the energy consumed and that we can't do on our own. We've got massive efforts that we've had across our engineering to optimize energy. We've made great gains. We have an initiative in play right now to take by 2030 to deliver a 20x improvement in AI compute efficiency in the data center. And that's the third initiative of that ilk that we've had. We do it on a rolling basis. So we play our part but when you look at the gains we get it's not just our silo. I go back to that collaboration theme we went early in the discussion. We drive that energy optimization up and down the stack. We drive the experience optimization up and down the stack. So it's been always part of my journey. But I will say I was fortunate at Apple to learn that aspect from, I have to say history's best at delivering unique experiences.

That's right.

It does matter. So what's another lesson I learned in the relatively short time at Apple, but it was on Apple speed, so it was incredibly accelerated pace and you had to make holiday. I mean, the iPod and iPhone had to come out every fall. You can't miss that release window. It has to be hit yet. Absolutely. As you said, the attention to detail, the nuance had to be there and that creates a maniacal focus on how you perform the engineering tasks to get that product out. And I had always, if you look at my history prior to Apple, I was always very focused on those details of engineering. But I will say it's the nuance that I did indeed learn at Apple and I will tell you it carried through at AMD. AMD has such a storied history of innovation, but when Lisa Su and I were recruited to join AMD, one of the things that we first really focused on was getting that culture of detail, of really executing so our customers would be delighted with the product that we had and not just good enough, but delighted. And so, it's, you know, again, I don't think your listeners may think, well, how does that really apply to a semiconductor company like AMD, but it applies very much so in terms of how you create a culture of execution and sweating the details because it matters. And so, I'd say it's our industry. Frankly, that approach applies to most every industry.

Yes.

That's right. If you round the corners, if you cut out steps that need to be there, it's going to be imminently obvious because it's not the end quality that your end users are looking for. It really matters and that's what I love about our culture at AMD. It's collaborative. It's innovative, but it's focused on executing and getting that quality. It's a pride. It's like a personal pride across our engineering team, our executive team. Lisa's a phenomenal leader that drives this culture at the company. It makes a huge difference.

Well, first of all, I couldn't agree with you more in terms of the last four months. I see it vividly. It's just a dramatic change of increase of capability of AI. And you know, one of the things that I'm urging our engineers at AMD and our non-engineers at AMD is to embrace it because a change of that nature in a short time, it can be scary. It is scary. And what do you do when something comes at you that's new? Well, one thing's for sure, none of us can bury our head in the sand and ignore it. It's not going away. And so, what we are doing is embracing it. We're driving 100% education of the ability to apply agentic flows and improve our productivity. You think about it, we're a growing company. We can't possibly hire the number of people we need at the rate, I mean, because of sensational demand for new computing. So, we need to be more productive. We need to embrace these capabilities and that's what we're doing. So, we're in that process, but it's hard for some people to deal with that kind of change. It's different than they've seen in their entire careers.

And there are different speeds. I described it in a podcast previously is there's many, many cogs and they're all moving at different speeds and they're all coming together. The RPM of one is 100, the RPM of another is 10,000.

Because AI is a productivity aid, you know, it can accelerate what we're trying to do. But if it's not understood and by the way if the industry doesn't put guard rails in place to make sure it's used correctly which all of us need to do, we're very focused on that at AMD to make sure that we are responsible in the way that we're applying AI. But if we do it the right way and we really communicate and people can understand, I think we can have great outcomes. If not you're going to see people rejecting it. And the problem is if certain countries reject it and others don't, it's going to create an imbalance of capability and frankly that leads to instability. It's the friction point.

Yes.

I've lived my life in technology. I couldn't be more excited. I'm an optimist. I see all the good that can come. I see discovery, you know, new forms of energy, fusion energy. I see discovering new drugs that can solve disease. All these huge problems which we couldn't reach, we couldn't tackle was beyond our capability. I see them all accelerating into the realm of doability. So my final thought, this is an incredible opportunity for us if we embrace this technology and use it for good.

Thank you very much, Graham.