Very good afternoon, everyone. Thanks for joining us for this fireside chat where for the next 45 minutes, I'm going to have my esteemed boss here, Matt Garman, and we'll have a bit of a dialogue and a conversation across several topics, several perspectives that I think leaders here in the room would love to hear about. Come January 1, even though the title says VP of Global Services, come January 1, I will lead AWS Industries globally, and part of that responsibility will also include the decision on where we build new data centers and infrastructure regions, as well as everything around sovereignty. So, I will dive into some of these topics with Matt here as we go into that discussion. So, Matt, pleasure to have you here with us. So, Matt, first of all, I want to start at the macro level. You know, a lot of political dynamics going on in the world today, tariffs, trade discussions, but even more importantly, discussions around sovereignty, digital residency, and when we think about all of these dynamics that are at play here, from an AWS perspective, a CEO of AWS, what comes to your mind and how do you see this as an opportunity or something that we can overcome as an organization there?
Yeah. I really do see it as an opportunity. I see that as an extension of things that we're able to take on for customers. Today, many of you all run global businesses and if you have to think about how you handle data sovereignty, how you think about localization, how you think about operating in different locations and changing regulations out there, and by the way, they're changing pretty rapidly. If you had to do that not in a cloud world, it's like borderline impossible, right? You'd be out there procuring ColoSpace left and right. You'd be having to switch, you'd have to be... In the cloud, one of the reasons you can be in the cloud is to easily adjust to a changing landscape. It is one of the things that, one of the benefits you get, that agility. And I think business agility is one of the things that is the defining, it's one of the defining characteristics of a successful business today. It's can you adjust to the changing landscape, whether it's customer dynamics, government dynamics, other geopolitical changes. And if you can change, that's kind of our job is to help you with that. If you need to have local infrastructure, you can launch in another region where we have infrastructure. If you have regulations that you need to hit, you can lean on our team to help you hit those regulations. If there are details that you need to change around managing tracking of data, we can help and make it easier to track that data. And so, the more you can get into a cloud world, it doesn't make these problems go away. These are hard, right? As business leaders like these are, I'm not suggesting you're like, 'Great, all right, I'm out. I don't have to worry about any of those problems anymore.' But we can help with that. And just like we help with infrastructure, I think that there are opportunities for us to help around data sovereignty.
No, very good. And probably I'll ask you to dive deeper into the European Sovereign Cloud.
Yeah, that's a good one. Thank you. I meant to bring that up. Quite excited about what we're going to be launching here in just a couple of weeks, which is the European Sovereign Cloud. And so, stepping back, there's been a lot of push in the EU for a sovereign solution for both governments to use as well as regulated industries. But I will tell you, if you ask, people ask me, 'What is your solution to sovereignty?' There is no one solution. It turns out every country has a different view. Every region has a different view. Every kind of person you talk to, even inside of those places have different views. And so when Europe said, 'Hey, we need to have a cloud solution that protects our data sovereignty.' What we did is not unlike what we do at AWS a lot. We tried to start with the customer first. We went to those governments and we said, 'What are you trying to accomplish?' Actually, we went really deep with the BSI in Germany, and as in one particular organization that was particularly helpful in really diving deep. And we were like, 'What are you trying to accomplish?' Because if it's protectionism, right, you're just trying to prop up a European cloud that doesn't really exist for whatever it's worth. We don't have a solution for that and good luck to you. But if you have real concerns, which they did by the way, right? And when you get down to it, most of these government agencies are actually quite rational and they say, 'Look, I need to be able to protect these workloads that if the United States government comes to you and says, I'm going to throw you in prison if you don't turn over this data or shut off this workload. How do I know you won't do it?' And so I was like, 'Well, that's a good question.' And so we were... That's a problem we can work with. And so we built the European Sovereign Cloud with this idea that to try to build in as many of those protections as possible where it's entirely housed inside of Europe, right? So, even the authentication stack, even your credentials, everything will be in Europe, right? So, if your regular AWS credentials that you use globally won't work in this region, you'll have to create a new one. It's a different partition. So everything stays there locally. So if there's a fiber cut, there's no issue globally around the world for a really long time, right? Because these countries have to think about war scenarios and things like that. All data centers and everything there is operated by EU citizens. It's actually, we built the whole, put that whole region actually in a wholly owned subsidiary that has an independent governing board that when they get kind of lawful requests, they can choose to say if that follows EU laws or not, not just US laws or other place laws. And so, we built a ton of protections in place such that we've had now many of these regulations in countries, actually the vast majority of them in Europe have come to say, 'Great, yep, that's actually the solution we want. And that's the model that we want going forward.' Really excited about that. We have a number of global partners, many of them are here this week launching with us and we're quite excited about that.
Very good there. Thank you, Matt there. I'll switch to the next one. Since we talked about regions infrastructure, infrastructure investment. 125 billion in CapEx, we see a lot of speculative investments in data centers around the world. And even in the US there is the talk about the AI bubble. From that perspective, what would your response be?
Well, so number one is yes, turns out running a cloud infrastructure is capital intensive. As we grow faster, we spend more capital on it. So that's kind of us for us to do so that you don't have to. So, the CapEx required to kind of build these data centers to build AI servers to acquire all the power and land is a lot but it's a good business for us. And obviously that's kind of the business that we are in. One of the things that's going around a lot is this thing about people concerned that AI is a bubble. I'm actually curious. We have a lot of really great business leaders in here. How many of you either today already believe it or think that within the next six months, show of hands, how many of you think that you have a path to positive ROI for your AWS or for your AI investments over the next kind of six months or so if not already today? To me, that is a lot of value that you're not going to stop getting, right? That is a lot of people, like that's fully 2/3 of this room already you're getting positive returns or have a path to in six months. You're not going to just turn that off. You're going to be like, 'I love that those returns that I'm getting, I don't want to get them anymore.' And the potential is only going to go up for better returns on that. And so to me, for us, where you all are our customers, that bubble is, I think, I'm not particularly worried about that at all. I think all of us are going to be using way more AI a year, two years, three years from now than we're using today for sure. Just the value is too much. Now, if you talk about billion-dollar startups that haven't written a line of code, maybe, like there may be bubbles there and that's different. Sorry, if there's any of you in the audience, maybe you're different, but that bubble could exist. But I think from the overall point of view, there's just way too much value there.
No, very good there. And Silicon, 1 million Trainium keeps deployed so far and... It'll be interesting for the audience to hear your point of view where we've got NVIDIA GPUs, we've got Trainium. How did that decision come about to build our own chips and how does that change the economics then there for our customers as you think moving forward?
Yeah. Well, if you allow me to take half a step back, AWS has been building custom silicon for over 10 years now. It started with our own nitro chips, which allowed us to virtualize all of our networking and storage and compute into a much more secure environment, a much more cost-performant environment, a much more high performance environment. And Nitro has been such a rousing, a huge success for us. And it is part of the reason why I think many of you trust AWS and build your businesses on AWS today. And it's why we've been able to innovate at such a rapid rate around some of our infrastructure. We then took that and said, 'You know what? We can build, we think that we can take what we've learned there and build a general purpose compute chip.' And that's where we launched Graviton. And now, for three years in a row, we've launched more Graviton processors inside of AWS than any other processor, right? And so, more than 50% actually is Graviton today of what we land in AWS for the last three years in a row. So, huge success. It means, by the way, there's still a huge chunk of that that's Intel and AMD and other processors. But we delivered really differentiated cost-performance benefits with Graviton. If any you're not using Graviton yet, you get a 20% absolute performance gain from Graviton compared to X86 at 20% lower pricing. It is such a no-brainer. And so, what we learned is that when we can increase performance, because we know the exact environment our chips are going into, we know exactly what our customers are looking for and we know how we can drive cost out of that equation. It's a win for everybody. And so we did that with general purpose compute and about five years ago we saw that more and more workloads were moving to AI. And we said, 'This is by far a more complicated chip for us to go build but we think that we can do it.' And we knew this one was going to take us a little bit longer to really start to ramp up. Trainium 1 was a good first chip, but it was kind of a learning chip, as was Graviton 1. It's one of these things where you have to kind of work some of the kinks out. And so Trainium 2, we knew that or we had the, I knew, but we had good data that more and more compute was going to need accelerated compute and was going to be in the AI space. And we thought we saw the benefits that we gave with Graviton and we said, 'If we can do the same in the AI space, we're going to be in a good spot.' Now, this was before generative AI completely exploded and kind of now it's an obvious thing to say, but fortunately we were already well down that path. And so, with Trainium 2 was really what I would consider our first real high-powered AI chip that was kind of at the leading edge of performance and significantly better for some workloads on price performance. And it's been fantastic. I'll say when we launched Trainium 2 the biggest deficiency was the software ecosystem around it, where NVIDIA's spent the last 20 years building up their CUDA ecosystem and the software around it. And so, we've been running really fast over the last couple of years to get the software ecosystem around Trainium to be great. And that's part of why I'm incredibly excited for Trainium 3. Trainium 2 honestly was a fantastic chip that kind of took an expert team to use, right? And so, we partnered deeply with Anthropic, we partnered deeply with a couple of other big AI labs. We partnered deeply internally on our own teams. But it took kind of experts. You had to do kernel programming and other to get the performance where you needed to. The software stack has now evolved where you can actually just write to libraries to get it to work. And that's a much better place. And so, I'm very excited on Trainium 3. I think that we're going to see a lot more people being able to use it and take advantage of those performance benefits. Lots of people by the way, get to take advantage of the performance benefits of Trainium today because it powers Bedrock because we could do the hard work under the covers to optimize the models. And so you get better price performance when you run some of the models on Trainium through Bedrock, where over 50% of the traffic is already running there. But I really think Trainium 3 is going to be a game changer. And what I'm seeing with Trainium 4, I think Trainium has the potential to lead in absolute performance for the next couple of years, forget price performance.
You know, and it's very interesting. I've had some customers, especially in the oil and gas sector, deep high performance computing who are super excited about Trainium and saying they can even optimize their code now that they understand the roadmaps of the successive Trainium versions that are coming out. So a lot of that collaboration is going to be very key.
That's interesting. I would think they would need FP64 specificity, which we don't have in some of the training chips but I'll go talk... For traditional HPC it's an interesting use case. So it's...
Very good. Matt, let me switch to data. In your keynote you talked about the importance of data and as we think through this agentic world and what we're driving here, the data corpus becomes the key aspect of what we're trying to drive here. So talk a bit about that and probably for the benefit of the audience here. Share your perspective.
Yeah. I had a funny... I'll share where I was talking to some analysts a little earlier today because we have an analyst summit across the street and one of them was saying, 'Hey, you know what, we love that you guys are talking about all this AI and other things like that. But do you really think that...' They're saying like, 'When I talk to customers, I don't know if all of them are going to be ready right away to adopt agentic, an agent or whatever.' And I was like, 'No, they're not.' If customers don't get their data to the cloud first, as a first step, they are not going to get the value out of AI or agents or whatever. So, it's like if they're not at that, it's kind of like you have to pause and you'd be like, 'Whoa, whoa, sorry.' If we're talking about how you're transforming with AI and you haven't moved any of your data to the cloud yet, we have to pause that discussion, put it on hold, or do a sidebar and go figure out how we help you modernize and get your data to the cloud really rapidly. And you can do some of these in parallel by the way. So you don't have to be serial, but if you don't go do that, if right now you're like, 'I have no cloud footprint, everything is on-prem, but I'm hoping to transform with AI.' You're probably not going to. Like, you're just, like it probably isn't because you're not going to have differentiated abilities. You're not going to be able to tie agents deeply into your data and your workloads. And I'm not saying you're not going to be able to any, but you're going to get significantly less value from that. And so, that data is incredibly important. Getting your applications moved to the cloud, your data moved to the cloud. By the way, not just your applications moved but also modernized is a key step too. Because if your applications are both moved and modernized, that's when you really start to see the huge impact. And so, as leaders, you can't get distracted by the overly excited shiny object and forget to do the baseline level of work. You've got to get those workloads and the key ones. And by the way, I understand it'll take a while if you haven't done it. Think about what those strategic ones are. Think about the ones that are going to make the biggest impact because you can do these things in parallel. I'm actually not suggesting you stop investing in your AI workflows, but if you haven't done the investment and getting the cloud landscape put together, you just won't get the value out of it.
Absolutely and in the last four years, in the transformation work we've done with me leading professional services and helping customers in that migration and modernization, working with partners, this has been a key discussion we've been having with them. But agentic AI or GenAI has even made it more imperative for us to make that move with customers.
Yeah, yeah. And it's like, just think about it this way. If you're not getting your data such that it can be ingested and acted upon by these AI systems, then you're just implementing generic technology. You might do it in a cool and interesting or somewhat novel way, but if it's really not specific to the workflows of your company and your specific data and your specific customers, you're just not going to get the value.
Absolutely. That's good. Let me connect the dots to Nova Forge where you are completely looking to transform the way customers can build their own foundational model at a much lower price point. So, love to hear your perspective on this and what you think this future holds for us from this perspective.
Sure and you know, let me, I'll talk a little bit of how we got to Nova Forge. Hopefully some of you heard this from the keynote, but if not, I'll do a quick outline of what it is. But what I saw and what the team saw is that as we were talking to customers over the last year and a half, we kept hearing over and over people saying, 'You know what, I think I'm going to build my own model.' And be like, okay, well you know, it's expensive, it's hard. Like building your own model is tricky, particularly if you need to get kind of frontier scale. But say more about why. And they would say, 'Well, because the existing model doesn't really understand my business.' Like I have a bunch of proprietary data and I think I can do a better job of building a model that understands my business. And I use Amazon as an example, okay? We have a, no surprise, we have a lot of really good retail data, okay? The generic models don't have that level of retail data, right? We have click-through data, we have what buys, what products are, how you think about products, whatever, et cetera. Lots of really good retail data. And so, we were going to go build Rufus, which is the Amazon shopping assistant that gives you suggestions about what to buy and things like that. And they tested generic models and they didn't do a good job because they didn't understand our products, they didn't understand our ASINs, they didn't understand the relationships between those two things or how people wanted. And so, they built their own model, right? And their scrappy team and they had some really, we obviously have some very talented engineers internally and they built their own model. It was expensive by the way. They used millions of dollars worth of GPUs and took a long time. And where they struggled was because they still didn't build a frontier model, right? They struggled because their model did a much better job of understanding retail data and retail shopping patterns and wasn't as good at the core logic and reasoning as the frontier models. And they're like, 'Ugh, I wish I could have the great reasoning of Anthropic's Claude and have it know my data.' Or they're like internally to our own teams, they're like, 'We love your Nova models, but we really want it to know shopping.' But I don't want to put that into the Nova models because then everybody has my shopping data and sorry, a few retailers out there, but Amazon's like, I want that proprietary to me, right? I don't want that to be out there for everybody. And it was a story we heard over and over again. We talked to banks, they would say the same thing and insurance folks, they'd say the same thing and healthcare companies would say the same thing. And they were all thinking about how to get, how do you have this problem where I can't spend a billion dollars and go build a frontier model with all my data in it and tuning and reinforcement learning only goes so far and I can't quite get that model. And so, the idea of Nova Forge was born from that, which is, well, we're training a frontier model. Amazon has the resources to go do this. If we're going to go spend a couple billion dollars or however much it is to go build a frontier model in Nova, and Nova 2 is fantastic by the way, like you should look at some of the benchmarks and go play with it, I encourage you to. It is just as performant as many of the other and the other leading models out there on many of the benchmarks at a much better price performance number, which is fantastic. And so we've got this, now we have a frontier model that gives you great performance, it gives you great reasoning, it gives you great tool use, it gives you good agentic capabilities, it gives you good ability to code. And what we said is, how about instead of, and we're like, we're kind of brainstorming and we're like, 'Okay, how about if we do open-weights?' We're like, well, the problem with open-weights is it's like it's okay but you still can only customize so much. You really want to be in the pre-training stages so that you really have it, understand it from the early stages of training the model. And so, then we thought 'Okay, what if we give people a checkpoint and then let them train from there?' The problem is you still need Amazon data to finish that training, right? If you start at the 70% checkpoint, you still need our data that we use for that last 30% to finish all the reasoning pieces. And then this is when we eventually got to this place where we said we could come up with this curated data set to finish the training, put it behind an API so you can't, because you're not, we don't have license access to give you the data, but we can let you use the data. And so you can have access to this data, you can inject your own data in there, mix it together, and finish training the model. And now you have a frontier model that understands your company, your data, your domain, right? If you're in a specialized domain that is not covered by a generic model, you can feed that data, I was just talking to someone yesterday. They're focusing on computational biology. They're like, 'Most models don't understand computational biology.' Because that's just not what the generic models are trained on. They want to feed that in there so that it does and then the resulting model they can actually do reinforcement learning on and get much better results. I actually think it's going to be transformative to a number of your businesses. As I talk to customers and many of you out there, like I can see a world where most companies are going to want their own trained version of their model. Even then after you distill it later, after you do reinforcement learning, the model actually turns out that if you've inserted your data early on, it responds better to the reinforcement learning you do. It responds better to the fine tuning you do because it kind of knows your domain already. And so, it actually gets smarter already. It's kind of like teaching at physics when you've already taught at calculus, like if you haven't taught a calculus first, the physics doesn't really work. It's like directionally, I don't know if that's a perfect analogy, but directionally it's like that kind of thing. And so the models actually learn the things that you want them to better because you've done at the beginning. So, we're quite excited about it. I actually think many of you are going to want to do this. And by the way, we have some teams that are ready to help jump in. And if you don't have a team that knows how to kind of mix data and think through things like that, we're happy to jump in and help you. But it's an area we're quite excited about.
