Thank you Arjit. Well I'm also really excited to be here and thanks for having me. I would like to extend my thanks as well to Jim and Helen Carricker for sponsoring this speaker series and I couldn't be more honored to kick off and to be able to share with you some thoughts. It's pretty amazing what's going on in the industry right now. And so that's what we're going to talk about the next hour. What's going on, how there's an incredible transition of technology, and I actually won't even use the word transition. I have to use the word disruption of technology. We all know it, but I'd like to give you a perspective of what does that mean for us as a community of technology developers and how does it impact the way that we think about developing technology and how does AI change the way frankly that we need to be engineers? It's that disruptive. So, I'll give you a few perspectives and hopefully you're even more encouraged at the end of the talk as to the opportunity for AI and both our technology endeavors and your opportunity to leave a lasting mark. On that last point, I have to say I'm jealous of the opportunity that many of you have who are in your studies now. You're an undergraduate, you're a graduate, you're completing your studies, you're entering either academia or industry. I just have to tell you, I wish I was right there with you because now and by the way, that's someone speaking to you that graduated when the first PC came out and the internet came out and mobile devices came out. So I have been through major transitions in technology. And so it's that voice of experience is telling you that I'm wishing I was starting now because this is going to be so exciting.
So let's jump in a little bit. If you don't believe me, pick up any periodical as you're surfing the web or your favorite portal that you're getting all your information on and you just are getting a constant barrage of what's going on. What you're seeing is articles in every single sector, healthcare, transportation, industrial automation, and you're just seeing example day after day of where AI is being deployed to make a difference. But it's beyond those applications. Now it's coming down to governments realizing that AI and technology is as fundamental as water and electricity and how they've thought about it. And so you're seeing sovereign governments learning and coming up to speed on the area of engineering and technology and computing at a depth that you've never seen before. In fact, I just flew in last night from DC. I spent the day yesterday with officials at the White House, with officials that are running the policy and investments and there's no question in the United States and I've had the same discussion with other governments, this is top of mind. They get it in terms of what this disruption means. It's a once in a generation challenge and it's a once in a generation opportunity.
So just a few numbers. Look at that number of 378 million users this year of AI and almost 800 billion in AI spending, broad AI spending. We had New York this week, our AMD financial analyst day, and what we put out is a number that we see very clearly from the trends that run right now. And that's a $1 trillion TAM for AI chip development. Not AI spend. The first stat I give you is overall AI spending. All of the hardware, software, data center buildouts, every aspect that's already growing rapidly. Just in our AMD world of chip design, CPU, GPUs, network chips, all the peripheral devices that it takes and that we play in and cross embedded, that is growing to a $1 trillion market opportunity by 2030. It's just huge.
And so, the key takeaway that I'd like you to think about when you hear these numbers is why is it that this isn't just hyperbole? Haven't we heard this before? Oh, there's a new trend. It's going to take over and then we see it die down. I mean, IoT of course is everywhere, but IoT was everything. The headmount displays were going to be everything. And yet it didn't turn out to be that way. Fiber optics, you go back to when Cisco exploded like a rocket. It's not that all that technology is not there, but some of those technologies take longer to deliver value. The way to think about what's different with AI is companies are not investing this money if they're not getting value out of it. The biggest hyperscalers in the world started building out these massive data centers and some of them went on pause building that out. They're not now. They're redoubling their capital investments because they're just seeing the value that their customer either there's first-party applications like Meta recommendation engines. Every time you're on Facebook, every time you're on WhatsApp, behind the scenes there's engines at work looking at what are they recommending, what is the content they're putting back in front of you. And if Meta wasn't getting return on that, they wouldn't be leaning in to the extent they are with their capital expenditures. And that same thing holds to other major all the other major hyperscalers in the US and globally. So your skills and compute and data and architecture are more important than ever.
So the other impact that AI is having that I want to just spend just a minute on is AI for science. I grew up with high performance computing. My career started actually in IBM. I ran microprocessor and then server technology development at IBM and IBM was an early leader in providing the supercomputers for high performance computing and I was always so enthused with the impact that myself and my team were able to have on supercomputing and couldn't have been more thrilled. I've been the last 14 years at AMD we've been involved we are the CPU and the Titan supercomputer. We're now the CPU and GPU behind the number one and number two supercomputers in the world with Frontier with LCAP 10 number one and Frontier number two. Why am I so enthused about that is that high performance computing has been driving science forward and this is with the ability to simulate much more accurately. We've had already not with I'll get to AI for science in just a minute but just think about traditional HPC think about the impact of lives when you have more accurate weather prediction, the molecular modeling that we've had, the advances we've made. AI for science when you come in and you combine the ability to run high performance computing at high precision double precision FP64 when you need it, but marry it with lower precision and AI acceleration where you can create models where you can predict based on other data that you've collected. It turns out it's speeding time in a dramatic way to discovery. And so, here's this area that I've had a passion in for four decades that now is going to accelerate time to invention. And I hear it every day. In drug discovery, I mentioned in the slide here a company Absi. I know Sean Mlan well the CEO of Absi. And as I've gotten to know him, he now texts me and he shares with me the advances they're getting along the way. They create new enzymes to treat disease. And he's cut in half the time it takes to create these new enzymes. So new drugs, new treatments are going to be coming to society at a much faster rate leveraging AI for science and you're going to see it across every technology endeavor. I look at as we start to address some of the biggest problems, fusion energy, quantum computing where you need quantum simulation before we can even get the error correction to where we need it to run pure quantum. They're all critical to accelerate time to discovery.
Okay. But here's the sobering part of my talk. There's a complexity that comes with AI and AI for science and that is when you're creating these massive large language models of course it comes with an explosion of compute demand. So you look at the rate that we've been on as we've started seeing starting with OpenAI and ChatGPT but now a plethora of large language models you see an exponentially rising demand compounded growth rate demand of computation. So that's one of the themes I want to spend the next few minutes talking about. How do we address as engineers, as technologists, this exponential demand for more computing? Oh, by the way, it gets harder yet again because it turns out computation, there's historical work we've all done to drive CPUs, GPUs, dedicated devices, ASICs to be more and more compute performance. But you have to feed the engines and memory is not keeping pace. Memory is running at about a doubling of bandwidth every two years. So it is not keeping pace with the demand curve, the compute demand curve. And so what that means is we'd simply tail off. So you need more high bandwidth memory for the AI computation. You need to address the power that comes with more memory and so it's yet an additional challenge. By the way, I can give three other examples, four other examples besides memory. Power delivery is not keeping pace. We need innovations to be able to create compute density and I literally could spend a half day with you on the challenges and how we think about going out about it but I wanted to just pick a couple and know that you have about five or six challenges behind it and what those challenges all lead to is how do you address that insatiable demand for more compute capacity when every trend we have is fighting against it to be able to deliver that kind of computation at energy efficiency. It creates a need for phenomenal innovation because we're hitting the energy wall. When we at AMD talk to anybody about selling our GPUs in the next data center, that conversation now comes with the discussion on power. Well, we've already been talking to this company that has this amount of power. We have this data center area that we've helped negotiate and secure. You can't sell computer chips today without understanding the source of energy that you're going to provide. Who would have thought that we would be needing this type of a gigawatt scale? And that is where we're at. And so we need tremendous innovations in energy sources. We have right here in Georgia down by Savannah, I understood from Arjit just before major investment in nuclear power. You see SMRs, the highly optimized and localized nuclear energy that can be provided. You have more efforts as well that need to continue in this area and eventually we need that AI for science I talked about earlier. It needs to come through and we need to accelerate achievement of fusion at economic scale. I had the chance a couple of times actually I've visited the National Ignition Facility at Lawrence Livermore National Lab. And when you realize what it takes to achieve a fusion today, you realize we've got tremendous innovation required to get that down to a scale that can be deployed in any economic way. But it's going to happen and AI for science can dramatically accelerate. That example I gave you, the National Ignition Facility, massive facility, and they do one shot per day to actually fire and gather data and simulate their progression on achieving national fusion with AI for science and really getting a digital twin. Imagine when that goes up many orders of magnitude. You're again accelerating the time to discovery.
Well, I want to talk to you about what I mentioned at the start that here's all these challenges. It really drives different thinking about how you solve the challenges and because I live this every day at AMD working across our technology teams and talking to our customers. I'll use AMD as an example of how are we thinking about really attacking these challenges? How do we attack the physical challenges to build these systems? How do we attack the economic challenges to make them more compact? Make them more energy efficient. And what I'll tell you is we found that you have to address every level of a computing system from the silicon, the foundry, the chip, the compute engines, the memory, the IO, all of the interconnect, the software, and to the application level. I've been around a while so I was mentioning Arjit reminds me of when I started the industry it was with IBM years ago and we were developing a chip designer developing the technology for mainframes and it was energy gated. It was relying on bipolar technology and it took all kinds of innovation to continue mainframe development. But now we're in a gigawatt scale and so it's even further challenges than we faced historically in the industry. And what it means is although people that run data centers always love to run a homogeneous data center. I mean isn't it nice if you could just get all your computing needs done and buy off-the-shelf commodity x86 servers and you got it. And there's many CIOs I talked to not in the last 5 years but go back 7 years ago 10 years ago that's what every IT administrator wanted to do. It's just easier. You can't do that anymore to address these challenges. You need different devices. You need heterogeneous devices. You need domain specific optimizations based on the specific algorithm specific problem you have because you have to go all the way through using the right device and the right workload optimization. And the way that we think about it is by segment because again it's not homogeneous anymore. So what do we do best as engineers? We solve problems. We all of us in the room or virtually together here, we're all problem solvers. And so we think about problems by breaking it out to the kind of area that we're applying technology to. Is it a massive data center and is it building those foundational models or the needs of huge enterprises? Is it also adding not only a CPU with a GPU accelerator is it adding a quantum accelerator because in the next five years you'll see quantum bolted on as an effective accelerator? Edge devices have unique demands, physical AI is taking off, you see base robotic implementations but now broader implementations taking off as we speak and of course the endpoint devices can be transformed if you can put more and more computing that you're holding in your hands and have sitting on your lap. So we'll start with the first two.
I'll start at the cloud level and what I will tell you what we found is it's back to the future. You have to design for the cloud at rack level. We actually did an entire acquisition. We acquired ZT Systems, one of the leading rack designers and manufacturers to get this capability in house. We did a massive acquisition over billion dollar several billion dollar acquisition and we kept the design team and we divested the manufacturing team because what we're doing is we want to optimize at rack level and then we'll put out reference architectures. AMD is committed to open software and open hardware ecosystems. So we're not going to sell the systems, but we're going to put out a reference architecture for our partners so that they can speed to market. I'll talk in a little bit about the Helios design, our next generation rack, and you'll see it being as I use it an example of that rack scale design. But moreover, it's all about co-design at the rack level. So let's look at what that looks like. There we go. So I first coined this term holistic design at an event in Germany that I've been invited to by IMEC. IMEC is one of the consortium that drives semiconductor development. And I put a challenge out there to all of my peers in the industry that what we had to think about to accelerate that compute capability meet that insatiable demand is fundamentally change our approach to this holistic design where as I said earlier we think about all the dimensions as they come together hardware and software co-optimization. It's really easy to say but I can tell you from the AMD example it's hard because people have for frankly their careers they've been they're great engineers and they're problem solving but they're problem solving in their domain and now you just come in one day and you say sorry that's not good enough you can't problem solve just in your domain so it turns out we've had to really change the way that our engineers work together. We changed the way before you can even start a project like this Helios rack I'm about to show you the example of we literally had to create a team a force a team across all these disciplines to get together and we didn't proceed past that initial high-level design until we saw that we had the co-optimization being done. So it again it's easy to look at a slide like yeah of course it's a lot harder to actually put this to practice because people are people and we're set in our ways of how we've done things. But I couldn't be more excited about that Helios design. It is open standard for our commitment. We did this in cooperation with Meta. It's an open rack-wide design out there in the OCP foundation and it's very much leverages open standards not just the rack design but you'll see in a moment even the interconnects that we use we want it opened up to the industry and really it's for us at AMD it was a blueprint of that co-optimization but it's also a blueprint for an open ecosystem how multiple vendors can come together not locking anyone in with a given design as some of our competitors do, but making a vibrant ecosystem that drives more innovation. It's a win for all.
So, I'm going to just pull out the compute tray virtually here and just show you the example. There's compute trays, there's also switch trays I don't have time to take you through, but I want to show you the compute tray because it just shows you what happens when you truly co-design. You know, you think about what we're trying to attack, which is much more efficient computation. What you're seeing here is a very very tight design of our GPU complex connected with our coherently connected to the Epic CPU. The Epic CPU, this is shipping, by the way, in Q3 of next year. So the Epic CPU is with our Zen 6 CPU core. It's built in leading edge TSMC 2nm. We're already sampling with customers. It's doing great, but it's integrated in this heterogeneous cluster with our MI 400 series and in this case it's the MI450 and it brings together that very tight connectivity of the CPU and GPU, but then it uses open standards. It's using ultra acceleration link where we actually donated some of our link technology to an open standard many companies in there. So we're using ultra accelerator link to connect to the network interface chip and then as you scale up we take that ultra acceleration link standard and we packetize it and we run it over Ethernet. So again it's an open ecosystem is how you scale but it's co-designed co-optimized across the disciplines including the software which I'll get to a little bit later in the presentation.
And then I talked about quantum again innovation just doesn't happen in silos so this is again a great example of cross company cooperation. Our FPGAs, AMD had acquired Xilinx in 2022. It turns out our FPGAs are perfect for managing quantum states and our VU19P our latest is used by IBM for instance in their cryo quantum state creation to control and to create those. It's also another FPGA has been used for error correction. IBM just published a paper a couple weeks ago talking about excellent advances in error correction by leveraging that FPGA. So it starts there. How do we partner with others to enable them? Again, we're an ecosystem player. How do we enable them to advance quantum and we're working with pretty much every quantum company in that regard. But that's not enough. And so we've invested more. We're starting with IBM. We're going to do it with all the other quantum companies on hybrid computing because the quantum is an accelerator. Not all problems really suit themselves to that type of multi-state and statistical approach and error management that quantum brings. It will be an absolute breakthrough on a certain class of problems but the vast majority of the other calculations and problem solving you're doing is in classical CPU and GPU. So we're working close with IBM and will with others on efficiently marrying quantum as an accelerator and how it works seamlessly with quantum computing.
Switching to embedded, it's a fact that if all of us are needing to leverage AI and expand its use and if we all needed those massive foundational models like Meta and OpenAI and Google and the rest that's just not feasible. AI has to come to us where we are. It has to be much more efficient. And so it needs to be if you're running a business, it's got to be on your factory floor. It's got to be in the medical device. My Lisa Sue and I met with the CEO of Medtronics in Boston just three weeks ago. Amazing what they're doing, embedding in these edge devices, edge applications for robotics, surgery, and what they're able to do now with much more precision than they could in the past. And it's physical AI which needs a combination not just of the integrated CPU and GPUs and MPUs that you see in both edge devices and AIPCs I'll talk about in a minute. You need much more than that for physical AI. You need all the sensor technology, you need the RF, it's an entire complement of technology before we see a mass utilization and realization of applications like humanoid robotics.
And then you know I just talked about this earlier this whole range of solutions. Just to give you a little point of fact of what does that mean in terms of the depth of what it as you start tailoring these technology solutions we have at AMD over 7,000 embedded customers, 7,000 unique applications that you see spread across all the applications here. So, think about particularly those of you that are early, you're in your studies, you're early in your career, where are your academic interests? I mean, you think about every one of these areas, they're all ripe for disruption. The word I used in my opening comments, every one of these applications is wide open for disruption. And so, it should start your thinking of how are you going to help disrupt it to make it more productive? How do we make medicine save more lives? How do we really leave an impact? And I have some comments at the end about what that means to those of us engineers. Do we fear AI? Do we adopt AI? I'll get to that in just a bit.
And then I again talked about physical AI. We're just absolutely at a turning point now where we're starting to see applications. Of course, robotics is widely deployed. You go to the you see the images of AWS it's robotic loading and the technical people they have are spending their time much more productively and they're able to handle the swings in workload that they have on the loading docks much more productively as they have task specific automation but physical AI is well beyond task specific. It really needs much more capabilities in terms of tactical controls, in terms of reasoning capabilities. So really it's going to need much more efficient computing to be able to run in a device that has to be at the edge and in a humanoid robotic application it's mobile. So you can just imagine the innovation required before it can come to full fruition. But it's estimated to be like a $30 trillion market in years out 2035, 2040 but it's just going to be incredibly massive.
So from an AMD standpoint what have we done? We've targeted our technology in these areas. We've targeted our work that we've done with our high precision controls with our FPGAs, our adaptive compute. We've got our products that used our gaming products, Radeon used to be used just for gaming and graphics and which is phenomenal. And I'll show you an example of that application in a minute. But guess what? Going forward, those Radeon cards, they're going to be edge AI devices because we already have our AI software stack ROCm running on both Radeon and on the right side you see the AIPCs. So, this is really very exciting.
And lastly, I mentioned those AIPCs. Pretty amazing what you can do with an AIPC. And in fact, we enable this to be not only a use of applications Windows you can run Copilot Plus and all those applications but also think about the AIPC as an incredibly affordable AI development vehicle. And so we have supported we have an SDK that we have out there and we'll share more with you about this where you can do your AI development right on an AIPC very low energy very very efficiently. And believe it or not, with our most capable AIPCs, we call it the Ryzen AI Max, you can run massive models, LLMs many, many billion parameter models. It has 128 gig of memory. It's got the latest CPU, GPU, NPU. It's incredibly capable. And our team Andrew is here. So those of you in the room, catch Andrew later for a detail of how we enable that. But what you'll see now as you leverage those platforms, they're widely available. Almost every PC is becoming an AI AIPC. And again, let us help you figure out how to let that be an inexpensive way to speed your development of AI applications.
I mentioned gaming because gaming's been completely transformed. You know, when I'll go back in time, but I took this role as CTO at AMD 14 years ago. And I remember as we were working on the gaming, we had these physics engines and what were we trying to do? It was rote deployment of physics engine. So we could model how the wind blew the hair in the individuals in let's say a first party gaming application. Back then if you played a game all the images of the folks in the game they're all wearing hats or you just couldn't even see hair because you couldn't model it. So we went to rote math engines. Well no more. What we're doing now is we're using tensor wave engines. We're using AI and ML to much more efficiently upscale. And so what we actually can do is save tremendous energy by rendering at a lower precision. So you render lower precision, use AI to upscale. What do you end up with? You end up with a super resolution. We call it FidelityFX FSR and we'll see if we have enough bandwidth here so that I could show you just a little image of this. So it's a little bandwidth limited here so it's a little choppy. But see if we can let me just try it one more time. There we go. So watch now. This is as we add the 4K FSR you'll see it in just a second as we add the 4K FSR you see incredibly more detail. So we lowered the energy so you saw the resolution go up much more clarity of detail here we just broke it out left versus right and that was done with a significant reduction of energy. So again this is applying engineering that makes a difference. Trust me, if you're a gamer, that type of resolution makes a big difference. But the same is used in medical imaging devices and on and on. I mean, the applications are just immense.
And then again, our commitment to co-design hardware and software. Our commitment to open is our ROCm stack. That's our software enablement. And of course, it's for the massive data center GPUs but it's also across the entire portfolio. So we call it train once infer everywhere. So if you haven't yet kicked the tires on ROCm, please do because we have now battle tested this software. If you tried it a few years ago, you said, 'Okay, come on AMD, you're getting there, but you've got to you're missing this, you're missing that.' We have a complete solution out there. And please give it a try. I think you'll be quite pleased. And I won't go through this in detail. I want to speed up a little bit so we have some time in Q&A. But we have support across the entire stack. Most people now are coding at layers of abstraction. So you're coding at a Python PyTorch level. You're coding at Triton which is vendor agnostic. But if not and if you've coded at a low level a CUDA level we have a very very efficient porting. It takes a day maybe two to port and then there's usually some tuning but it's very very efficient and of course we have all the libraries to enable.
Okay, I said I wanted to wrap my talk just talking about not the technology and the applications but what does it mean to us as an engineering community to those in electrical and computer and data science and again what did I say earlier is I think you're coming upon the most exciting time in the history of mankind and so I want to hit a bit of a controversial thought. And I have to give credit to Andrew Ng. Andrew Ng's a professor at Stanford and I've worked with him on multiple facets across the industry. And this is several years ago. He said, 'Mark, I know your engineers are worried. Is AI going to take their job? Is this just an engineer replacement?' And he said, 'Watch as this develops.' He goes, 'The engineers that thrive will embrace AI because it's going to make every technologist give them the opportunity to be 10x the technologist that they could have been.' And you think about the demands upon us in technology. I said earlier it's insatiable that the race for us to innovate faster to get products out faster. So rather than think about does this mean is this a peril to those of us in the technology industry or others frankly it's my comments apply if you're in a finance community if you're in a supply chain application if you embrace AI and let it make you 10x better the contributor that you were before you're going to apply your expertise your ability as engineers to solve problems just got massively accelerated. So, we urge our engineers to make sure they are applying AI every single day. And we've really ramped up the usage and I very much urge you all to do the same and mentor others. Bring them with you on the journey.
So I want to wrap up again giving us time for questions but you are the future. The world has become technological beyond what I ever could have managed ever could have imagined when I started my career decades ago. So it needs us to engage. It's up to you. Are you embracing the technology? Are you engaging? Are you also remembering that in the era of AI, your human skills, your soft skills now became much more important. You're getting an assist on things that used to take a ton of your time. Rote elements that used to chew up half your day. They're gone. You're going to be leveraging AI for those. So now you're part of that holistic design. You need to be able to communicate, collaborate, work across discipline, and now you're that 10x engineer in this AI first era. I will tell you at AMD, we're looking to hire AI first engineers. That's they don't even think about like, of course, AI is like a calculator. No one uses calculators anymore. It's like your analytics program. It's just a tool that you don't even think about. You're using it every day. And lastly, please engage. Please take on the toughest problems, accelerate your deployment of leading edge technology, and have an impact. And when you look back, you're at a point of career I'm at, there's just nothing better than being able to have examples you feel good about where you have helped by solving problems and progressing technology forward. And wow, Georgia Tech, what a place to drive that learning and that application of technology. Arjit, I'm so impressed with what you all have done here, what you will do in the future, and at AMD, we look forward to growing our partnership with Georgia Tech. Thanks so much.
