Bloomberg Audio Studios podcasts, radio, news. It is a joy to be joined by Matt. Thank you. Matt, welcome from Las Vegas. re:Invent upon you, chip ultra clusters, ultra servers, chip development, LLM, then we've got to talk about developer tools. There's a lot to get through. Let's start with the ultra cluster. Can you just talk to us a little bit about this more than 100,000 chips that will be put in one single area for AI hardware? Why do it? What will it offer?
Yeah, this is something that we're building together with our partners in Anthropic. And it's actually going to be several hundred thousand chips. And they're focused on, and it's our new Trainium2 chips, and so we expect that this should deliver them five times more compute than they've used in their last model training. And so we're really excited about what they're able to accomplish. And they're expecting to build much bigger and much more capable AI models from that large compute cluster.
Do we have a US location? Do we have an exact time frame other than early 2025?
No location to share yet.
Matt, what then of the technical challenges? Paint the picture for our audience how difficult this is in terms of cooling, in terms of energy use. What has been the things that you have to overcome with this?
Yeah, well, the first core innovation is that we built our own chip, actually. And so we've built our own chip, it's called Trainium2, and we're quite excited about the performance that we get out of that. And that's combined all in these really large UltraClusters that combine 64 Trainium chips all to deliver 83 petaflops from a single node. And so that is the first innovation, is our Amazon-designed custom silicon that gives us really unparalleled performance for generative AI capabilities. And then we build these together with high-performance networking that we also build in-house. And then, of course, there'll be cooling and heating and power that we need to go build the data centers. But it really starts with that silicon down at the level, and we innovate on the entire stack of AI to make sure that we can control everything that goes into those clusters.
Can you talk to us a little bit, compare and contrast here? This is about less dependency on Nvidia GPUs to many ways doing an alternative for your clients. What sort of cost saving will clients get? What sort of energy and efficiency will they get?
Yeah, well, first of all, I think we like to think of it as a supplement to Nvidia GPUs. Nvidia has a fantastic product. That team has done an outstanding job executing, and we think that the vast majority of workloads are going to continue to run on Nvidia processors for a long time. But customers want choice, and they want choice that can give them some lower-cost options. And we think that for certain workloads, for many workloads, Trainium2 can give customers 30 to 40% cost-performance benefits over today's GPU-powered instances. And so we think that that's a huge win for customers, particularly as they're looking to lower the cost of generative AI workloads. But we'll be great partners with Nvidia and continue to lean in on building great technologies together with them for a long time.
Trainium2 available, Trainium3 in the works. And all of this is as Nvidia at the moment, it's estimated, has about 95% market share. Do you agree with that sort of number they have? And what sort of area do you think that that will come down to?
Yeah, I mean, I think it's probably higher than that. I think that the vast majority of workloads in generative AI today run on Nvidia technology, and they've absolutely been the leaders in that space. But we do hear from customers that they want choice. And just with our processors where we're tight partners with Intel and with AMD, but we decided to go build a general-purpose processor called Graviton, and it's been hugely successful with our customers. But we also provide lots of Intel and lots of AMD processors in our cloud today, and those businesses continue to grow. So I expect that our usage of Nvidia will continue to grow from our customers, and that choice is really going to be powerful. And as you see this explosion of generative AI usage, I think there's going to be plenty of business for multiple different people to be successful.
A key partner is Intel. How concerned have you been about the change at the top?
Oh, that's okay. I wish Pat the best. I know Pat, and he's been a good partner of ours. But we've been partners with Intel for a long time. It's been 18 years since we launched our first Intel instance when AWS and EC2 first launched. And we'll continue to be great partners with Intel, and they have a great technology team there. And we look forward to continuing to roll out the latest technologies from Intel for our customers to be able to use.
And you've been continuing to roll out, as you say, Nvidia offerings. Last time you were on, we were talking about Blackwell. Of course, many have been frustrated perhaps by some slowness to market there. When do you anticipate that Blackwell will be unfolding, and how difficult has it been to ensure the supply side is there?
Well, they've obviously had some manufacturing things that they're going through, but we're very excited about it. I think the early returns and the early looks at Blackwell look fantastic. We expect almost a two-and-a-half times in the compute power that you get from Blackwell that we saw from H100s. And so I think it'll be a really material jump for customers once we get those out. And I think those will be early next year, and we're excited about putting them in customers' hands. And we'll get them out there as soon as they're available.
