AWS CEO Matt Garman Talks AI Roadmap | Bloomberg Talks

Thanks for having me on. It's nice to be here again. It's been a fantastic year of innovation. It's really been incredible. And as I look out there, one of the things that I've been most excited about is how fast our customers are innovating and adopting many of the new technologies that we have. And as you think about customers that are on this cloud migration journey, many of them have been doing that for over the last several years, but this year in particular that we've really seen an explosion of AI technologies, of agentic technologies, and increasingly we're seeing more and more customers move their entire estates into the cloud and AWS. So, it's been really fun to see. It's been an incredible pace of technology. And it's been a really fun first year.

Yeah, that is AWS, right? And so the key is that's a mix of customers running their own models. Some of that is on Amazon Bedrock, which is our own hosted models where we have first-party models like Amazon Nova as well as many of the third-party models like Anthropic's models. And some of those are applications, things like Amazon Q which helps people do automated software development as well as a host of other capabilities. And so there's a mix of that and I think part of the most interesting thing about being at a multi-billion dollar run rate is we're at the very earliest stages of how AI is going to completely transform for every single customer out there. And we talk to customers and we look at where the technology landscape is and we firmly believe that every single business, every single industry and really every single job is going to be fundamentally transformed by AI. And I think we're starting to see the early stages of that but again we're just at the very earliest stages of what's going to be possible. And so that multi-billion dollar business that we have today is really just the start.

Yeah, like I said, we are in multiple billions of dollars. And that's for customers using AWS. We also use lots of generative AI inside of Amazon for a wide range of things. We use it to optimize our fulfillment centers. We use it when you go to the retail site to summarize reviews or to help customers find products in a faster and more interesting way. We use AI in Alexa in our new Alexa Plus offering where we conversationally talk to customers through the Alexa interface and help them accomplish things through voice that they were never able to do before. So every single aspect of what Amazon does leverages AI and our customers are exactly the same. Customers are looking to AWS to completely change whether it's their contact centers through something like Amazon Connect where it shows AI capabilities so that you don't have to go program it all the way down to our custom chips or NVIDIA processors or anything where customers at the metal are building their own models. We have the whole range of people that are building AI on top of AWS as well as Amazon themselves.

Sure. And that changes over time. I think look, as we progress over time, more and more of the AI workloads are being inferenced. I'd say in the early stages of AI and generative AI, a lot of the usage was dominated by training as people were building these very large models with small amounts of usage. Now, the models are getting bigger and bigger, but the usage is exploding at a rapid rate. And so I expect that over the fullness of time 80%, 90%, the vast majority of usage is going to be inference out there. And really, for all those out there, inference it really is how AI is embedded in the applications that everybody uses. And so as we think about our customers building, there's a small number of people who are going to be building these models but everyone out there is going to use inference as a core building block in everything they do. And every application is going to have inference and already is starting to see inference built in to every application. And we think about it as just the new building block. It's just like compute. It's just like storage. It's just like a database. Inference is a core building block. And so as you talk to people who are building new applications, they don't think about it as AI is over here and my application is over here. They really think about AI is embedded in the experience. And so it's increasingly I think it's going to be difficult for people to say what part of your revenue is going to be driven by AI. It's just part of the application that you're building. And it's going to be a core part of that experience and it's going to deliver lots of benefits from efficiency, from capabilities and from user experience for all sorts of applications and industries.

No, I think that at this point more, definitely more usage is inference than training.

I don't have any metrics to share on that front, but I think it's one of the measures that we can look at is the numbers of tokens that are being served out there. But it's not the only one. And I increasingly think that people are going to be thinking about these things differently. Tokens are a particularly interesting thing to look at when you're thinking about text generation, but not all things are created equal. I think particularly as you think about AI reasoning models. The input and output tokens don't necessarily talk about the work that's being done. And increasingly you're seeing models that can do work for a really long period of time before they output tokens. And so you're having these models that can sometimes think for hours at a time, right? They might, you ask these things to go and actually do research on your behalf. They can go out to the internet. They can pull information back. They can synthesize. They can redo things. If you think about coding and Q Developer, we're seeing lots of coding where it goes and actually reasons and does iterations and iterations and improves on itself, looks at what it's done and then eventually outputs the end result. And so at some point the final output token is not really the best measure of how much work is being done. If you think about images, if you think about videos, there's a lot of content that's being created and a lot of thought that's being done. And so tokens are one aspect of it and it's an interesting measure, but I don't think it's the only measure to look at. Although they are rapidly increasing.