Hello, welcome to theCUBE. I'm John Furrier, your host here in Seattle, the AWS re:Invent building with Matt Garman, the CEO of AWS, for an exclusive interview for re:Invent and all the activity happening. Matt, great to see you. Thanks for coming on theCUBE in your studio.
And home game for you, away game for me. Absolutely. Happy to do it. Thank you for coming in and chatting.
So obviously, re:Invent, the biggest conference you guys have. It's been, I think our 14th year, we've seen the transformation and you as CEO now. We talked in August about your vision. Folks can look at that YouTube video, but we kind of teased out this Gen 2 kind of cloud play with agentic AI and all this gen of AI and agents are coming. The game has changed, but it's still back down to the infrastructure. And so I want to get into that with you, but I also want to talk about your growth. And you guys had great earnings and you had great performance. I think it's a 10-year-high for the stock price. The growth is continuing to go and we're in this post-ZIRP era and there's a real emphasis on profitability. You guys are doing well as well as your customers. What's your view on the current market right now?
Yeah, we're seeing a lot of people that are really leaning into the cloud. I would say as we talk to customers out there, they are increasingly interested in how they drive revenue growth, how they drive innovation, how they get their businesses growing. And a lot of customers are seeing that gen AI is one of the things that's driving that. And as you increasingly talk to customers about how they're really going to get value and how they're going to get enterprise value and drive their businesses forward, they realize that they've got to get those workloads into the cloud so they can have the agility to take advantage of all the latest technologies that are coming out. And that means getting their data to the cloud. That means getting their applications to the cloud so they can apply all these new models, whether they're agentic workflows. All of these new benefits that they see coming down the road, they realize that that cloud piece is there. And so we see that as a real tailwind to the business and something that customers are really excited about moving.
And what's your focus for re:Invent this year? Because obviously, every year, there's a slew of announcements. It's always the classic stack. You've got infrastructure, you've got the data layer and the higher level services, and obviously the applications are changing radically. What's the focus?
Some of the focus is we're also going to get back to some of the basics on just how do we go and help customers innovate in all the different areas that they get to. And I think look, we'll have a lot of announcements in things about AI and generative AI, but we're also going to have a lot of really cool innovations and announcements around the core of our infrastructure and thinking about compute and storage and databases. That is an area that we have been hard at work innovating on. And I think it's one of those things that we don't want to forget. It's kind of the core of what we do and I think it's something that our customers are hungry for as they look for continued innovation across that whole stack, not just in the shiny new objects, which we'll have a lot of announcements there also.
Yeah, I think our last interview, we talked about inference as the killer app, and you said you have good training, you're going to need inference to go with that. What are some of the performance gains we're seeing this year? Because again, the hype curve of generative AI has kind of been gone. Now we're in the reality curve where it's show me the proof, but there's still a lot of work to be done at the infrastructure layer because everyone wants performance and they don't want to pay out the nose for it. They want to have good cost structures and visibility into the unit economics. What's the update there?
Look, I think that there's two things you think about as you think about inference. And this is really is how do people really integrate it in. I think when we saw early on when generative AI hit the scene two years ago, everybody ran and they did a quick proof of concept and they put a chatbot on their website and they were very excited about it and they got some early wins. Now though, I talked to customers and they have 100, 200 proof of concepts out there in the enterprise and they're trying to figure out how they actually now figure out, as you point out, how do you get value from that? How do you actually drive enterprise value, not just have this cool factory, go test? And so the couple things that customers are looking at are, one, what is the cost and how do I bring down the cost of running that inference? And two, how do I actually integrate it into my production environment? And those are two real challenges, which are not there when you do a proof of concept. And so on the cost side, we're driving down a number of areas where we think about new models, we think about new efficiencies, we think about new infrastructure, whether that's new instance types, new networking, et cetera. And so we are pushing really hard to continue to lower the cost of inference. And if you've seen over the last year, the cost of inference has been coming down and down and down. And we think that there's still a long way to go there. And we think that hardware has a lot to do with that, the system or the service that you run there. And we think that Bedrock is a great place for people to go to get that absolute lowest cost as you think about new models and new infrastructure. But just as important is really how do you integrate it into that data? Because customers really are hungry for how do you take those 100 proof of concepts, turn them into the three or four or five real production applications where they're going to drive real enterprise value that's going to drive revenue, lower costs, et cetera. And that means how do you have a framework that it gets tied into your data? How do you have a framework that thinks about security, thinks about permissioning in your enterprise and your organization, thinks about how do you have an actual usable ability to go launch and build agentic workflows? How do you think about that whole piece that you really think about a production? And that's where I think customers see that AWS is quite good at.
So you're going to have some IaaS, some infrastructure announcements around performance and silicon, like always, pretty much going to be the same.
Everybody's going to need to tune in for re:Invent.
But we're very excited about many of the announcements.
Okay, so check the box on that one, and needed by the way. Okay, so that's going to power this next wave. I think we talked about our last interview. You got to pedal as fast as you can on the infrastructure, get the chips going and the speed. And that's going to enable this next layer up in the stack, we talked about the gen stack. So we're seeing growth in the PaaS side. Our research has shown your PaaS business is growing significantly fast. I would say the infrastructure has been the bread and butter core services, but as gen AI hits, we're seeing a lot of activity being enabled in database and other services around workflows end to end. And I talked to Werner about the Lambda 10 year anniversary and what came out of that conversation I want to ask you is Lambda enabled serverless and there was no blueprint. It was solving a problem from core services that I think WeTransfer had and some of your other customers. And that opened up and became a legendary and historic product. And what that enabled was just better everything. Is there a gen AI version of that? Because gen AI has a similar pattern. There's no real blueprint other than you got to get the learnings. And so this is a PaaS layer, it involves database, it involves a lot of these services that used to be run a certain way but might run differently. What's your view on this? And also because agents are going to come out of this.
Yeah, look, almost for sure that the entire way that you go build generative AI applications is going to change and be reinvented. And because we're so early in that space, I think we're all finding how to go build that. And our job is to make that easier for customers. If you take a half a step back, you talk about the core building blocks that we have in compute and storage and databases. I actually have this view that customers ask about generative AI applications. I actually think they're just applications. And so I actually think inference is the next core building block. And it is, if you think about it that way, if you think about inference as part of every application that you go build, it's not just a separate generative AI application and then all my other applications, it's just an integral part just like you would think about... It's not a database application and a non-database application, it's just a part of what you might use. And so if you think about it that way, then you got to think about how does it get integrated into the fabric of what you're doing? And that means you think about data workflows, you think about how these models might interact really well with S3 and EC2 and your database and how do they all kind of work together? And that's the job that we're trying to make easier for customers so that when you're thinking about startups out there or enterprises trying to solve a problem and they're going and building a new application, it's just a tool that they can get really powerful and exciting capabilities deeply embedded into their application.
So serverless came out of Lambda, that basically still ran on servers. It really wasn't serverless, it felt like serverless. Is there a gen AI version of that coming that you see? Because if you look at what you guys are doing and you can see as you look at Bedrock and SageMaker under the hood there, you're looking at essentially stitching together the mosaic of services that used to be standalone or configured either manually or the old school way. Old school way, it's funny to say, that's the old cloud way. But the new cloud that you guys have with gen AI, that's interesting because this is what agents need. They need to have the agility. What are you learning from customers that's going to feed into this new direction?
Yeah, I think there's a couple of things that I would say is one of a funny anecdotes. I remember when I was running EC2 and Charlie Bell was super excited about serverless and he was telling all of his directs, 'Everybody has to move their services to a serverless offering.' And I was like, 'What about me?' He's like, 'Okay, not you. EC2 is actually servers at the end of the day.'
You do need to have an actual server at the end of the day, but you fast-forward, and that serverless model is still something that I actually think is super important in AI. And you think about Bedrock, Bedrock is serverless, right? You send it tokens, you get tokens back. And as you think about agentic workflows, again, Lambda is a key part of that. So you can have this bit of compute that helps the models know where they're going to go. So I do think that as we look at those new models and you think about that new programming paradigm, serverless, people don't want to have to go run large clusters of models. They may use EC2 and instances to do things like pre-training and fine-tuning and other things like that. But once you really get to the application, you don't want to have to worry about scaling and managing that infrastructure. And so that vision that we had about serverless 10 years ago absolutely still applies to where we are today. And I think the AI models are easily applicable to that. And we kind of take that same mentality that just because it's a new technology doesn't mean that customers all of a sudden want to manage infrastructure. They actually just want to get the capabilities and actually go think about the logic and the workflows and the-
I mean the magic of Amazon was abstracting away the complexities and just the functionality doesn't go away. Talk about agents because agentic is the hot wave, I won't say shiny new toy because there's legitimacy to what's going to happen. You're already seeing MQ doing it for developers, you're seeing it in business. This is going to be the application feature, whether it's retrofitted or net new applications. What's your vision on how you guys are going to create those agents? How do customers have an architecture and platform that's going to be successful?
Yeah, and why people are excited about it's not surprising, I mean it's why we launched agents as part of Bedrock probably six months ago. And the key thing you want to think about is the value that you get out of many of these AI capabilities. Number one, there's consumption, creating content, summarizing content. That is super powerful and it is a thing that I think seen people get really excited about in Bed and a lot of different things. The next step is how do you get many of these models to actually start executing tasks and doing the next wave? And that is what agents are all about. At the end of the day, they need to take that next step. If you can summarize a bunch of data into a crisp enough thing that you can make a decision on, how do you then automate that step? And it really is just process flow automation. It's thinking about how do you have these things really go? And the cool part, and I think the part of the problem that we want to help customers solve too, is that we're at the part where that's possible now. The interesting thing though is when you really start to get leverage as you have thousands and thousands and thousands of agents doing things, and then that pretty soon gets super complicated and a thing that is really hard to manage and wrangle and figure out how you do. We want to make it possible for people to actually go do agents at scale, think about more complicated processes at scale, and what are the frameworks and areas that we can help them actually drive that so that it's a manageable process, because if you have 1,000 or 10,000 or 100,000 agents out doing things, pretty soon that process gets unmanageable. And so we're trying to think about how do we help customers solve those problems?
Well, that's why I brought the Lambda conversations. I think that enables serverless to do a lot of these things of distributed computing. But you're bringing up a great point. I think this is the hot topic that we want to dig into is multi-agent and distributed AI workloads. Because what you're basically saying is it's not an agent, it's this collection of agents, it's multiple agents, it's distributed AI with agents, they have to talk to each other. How do you handle resilience, routing? These are technical but important things. How do you roll back and recover from a security standpoint? How does resilience fit into agents? Because things are happening all the time.
And look, the industry is moving fast and we're trying to help people get their hands around it so that they can move fast with it.
Anything in re:Invent you think is going to pop out that's notable, people should pay attention to as they think about distributed computing going agentic, what are the key things that you guys advise and hear from customers that it's on their mind around setting the table for this future?
