We'll start back from there into the vertical integration needed, knowing that as long as you're serving the market well with the product, you can always cost-optimize. There's a question going forward. Right now, we have models that have this distinction between training and inference. One could argue that there's a smaller and smaller difference between the different models. Going forward, if you're really expecting something like human-level intelligence, humans learn on the job. If you think about your last 30 years, what makes Satya tokens so valuable? It's the last 30 years of wisdom and experience you've gained in Microsoft. We will eventually have models, if they get to human level, which will have this ability to continuously learn on the job. That will drive so much value to the model company that is ahead, at least in my view, because you have copies of one model broadly deployed through the economy learning how to do every single job. And unlike humans, they can amalgamate their learnings to that model. So there's this sort of continuous learning exponential feedback loop, which almost looks like a sort of intelligence explosion. If that happens and Microsoft isn't the leading model company by that time… You're saying that well, we substitute one model for another, et cetera. Doesn't that then matter less? Because it's like this one model knows how to do every single job in the economy, the others in the long tail don't. Your point, if there's one model that is the only model that's most broadly deployed in the world and it sees all the data and it does continuous learning, that's game set match and you stop shop. The reality that at least I see is that in the world today, for all the dominance of any one model, that is not the case. Take coding, there are multiple models. In fact, everyday it's less the case. There is not one model that is getting deployed broadly. There are multiple models that are getting deployed. It's like databases. It's always the thing, 'Can one database be the one that is just used everywhere?' Except it's not. There are multiple types of databases that are getting deployed for different use cases. I think that there are going to be some network effects of continual learning—I call it data liquidity—that any one model has. Is it going to happen in all domains? I don't think so. Is it going to happen in all geos? I don't think so. Is it going to happen in all segments? I don't think so. It'll happen in all categories at the same time? I don't think so. So therefore I feel like the design space is so large that there's plenty of opportunity. But your fundamental point is having a capability which is at the infrastructure layer, model layer, and at the scaffolding layer, and then being able to compose these things not just as a vertical stack, but to be able to compose each thing for what its purpose is. You can't build an infrastructure that's optimized for one model. If you do that, what if you fall behind? In fact, all the infrastructure you built will be a waste. You kind of need to build an infrastructure that's capable of supporting multiple families and lineages of models. Otherwise the capital you put in, which is optimized for one model architecture, means you're one tweak away, some MoE-like breakthrough that happens, and your entire network topology goes out of the window. That's a scary thing. Therefore you kind of want the infrastructure to support whatever may come in your own model family and other model families. You've got to be open. If you're serious about the hyperscale business, you've got to be serious about that. If you're serious about being a model company, you have to basically say, 'What are the ways people can do things on top of the model so that I can have an ISV ecosystem?' Unless I'm thinking I'll own every category, that just can't be that. Then you won't have an API business and that, by definition, will mean you'll never be a platform company that's successfully deployed everywhere. Therefore the industry structure is such that it will really force people to specialize. In that specialization, a company like Microsoft should compete in each layer by its merits, but not think that this is all about the road to game set match, where I just compose vertically all these layers. That just doesn't happen.
So last year Microsoft was on path to be the largest infrastructure provider by far. You were the earliest in 2023, so you went out there, you acquired all the resources in terms of leasing data centers, starting construction, securing power, everything. You guys were on pace to beat Amazon in 2026 or 2027. Certainly by 2028 you were going to beat them. Since then, in the second half of last year, Microsoft did this big pause, where they let go of a bunch of leasing sites that they were going to take, which then Google, Meta, Amazon in some cases, Oracle, took these sites. We're sitting in one of the largest data centers in the world, so obviously it's not everything, you guys are expanding like crazy. But there are sites that you just stopped working on. Why did you do this?
This goes back a little bit to, what is the hyperscale business all about? One of the key decisions we made was that if we're going to build out Azure to be fantastic for all stages of AI—from training to mid-training to data gen to inference—we just need fungibility of the fleet. So that entire thing caused us basically not to go build a whole lot of capacity with a particular set of generations. Because the other thing you have to realize is that having up to now 10x'ed every 18 months enough training capacity for the various OpenAI models, we realized that the key is to stay on that path. But the more important thing is to have a balance, to not just train, but to be able to serve these models all around the world. Because at the end of the day, the rate of monetization is what will then allow us to keep funding. And then the infrastructure was going to need us to support multiple models. So once we said that that's the case, we just course-corrected to the path we're on. If I look at the path we're on, we are doing a lot more starts now. We are also buying up as much managed capacity as we can, whether it's to build, whether it's to lease, or even GPUs as a service. But we're building it for where we see the demand and the serving needs and our training needs. We didn't want to just be a hoster for one company and have just a massive book of business with one customer. That's not a business, you should be vertically integrated with that company. Given that OpenAI was going to be a successful independent company, which is fantastic. It makes sense. And even Meta may use third-party capacity, but ultimately they're all going to be first-party. For anyone who has large scale, they'll be a hyperscaler on their own. To me, it was to build out a hyperscale fleet and our own research compute. That's what the adjustment was. So I feel very, very good. By the way, the other thing is that I didn't want to get stuck with massive scale of one generation. We just saw the GB200s, the GB300s are coming. By the time I get to Vera Rubin, Vera Rubin Ultra, the data center is going to look very different because the power per rack, power per row, is going to be so different. The cooling requirements are going to be so different. That means I don't want to just go build out a whole number of gigawatts that are only for a one-generation, one family. So I think the pacing matters, the fungibility and the location matters, the workload diversity matters, customer diversity matters and that's what we're building towards. The other thing that we've learned a lot is that every AI workload requires not only the AI accelerator, but it requires a whole lot of other things. In fact, a lot of the margin structure for us will be in those other things. Therefore, we want to build out Azure as being fantastic for the long tail of the workloads, because that's the hyperscale business, while knowing that we've got to be super competitive starting with the bare-metal for the highest end training. But that can't crowd out the rest of the business, because we're not in the business of just doing five contracts with five customers being their bare-metal service. That's not a Microsoft business. That may be a business for someone else, and that's a good thing. What we have said is that we're in the hyperscale business, which is at the end of the day a long tail business for AI workloads. And in order to do that, we will have some leading bare-metal-as-a-service capabilities for a set of models, including our own. And that, I think, is the balance you see.
Another question that comes around this whole fungibility topic. Okay, it's not where you want it, you would rather have it in a good population center, like Atlanta. We're here. There's also the question of, how much does that matter as the horizon of AI tasks grows? 30 seconds for a reasoning prompt, or 30 minutes for a deep research, or it's going to be hours for software agents at some point and days and so on and so forth, the time to human interaction. Why does it matter if it's location A, B, or C?
It's a great question. That's exactly it. In fact, that's one of the other reasons why we want to think about what an Azure region looks like and what is the networking between Azure regions. This is where I think as the model capabilities evolve and the usage of these tokens evolves, whether it's synchronously or asynchronously, you don't want to be out of position. Then on top of that, by the way, what are the data residency laws? There's the entire EU thing, where we literally had to create an EU Data Boundary. That basically meant that you can't just roundtrip a call to wherever, even if it's asynchronous. Therefore you need to have maybe regional things that are high density, and then the power costs and so on. But you're 100% right in bringing up that the topology as we build out will have to evolve. One, for tokens per dollar per watt. What are the economics? Overlay that with, what is the usage pattern? Usage pattern in terms of synchronous, asynchronous. But also what is the compute storage? Because the latencies may matter for certain things. The storage better be there. If I have a Cosmos DB close to this for session data or even for an autonomous thing, then that also has to be somewhere close to it, and so on. All of those considerations are what will shape the hyperscale business.
Prior to the pause, what we had forecasted for you, by 2028 you were going to be 12–13 gigawatts. Now we're at nine and a half or so. But something that's even more relevant—and I just want you to more concretely state that this is the business you don't want to be in—is that Oracle's going from one fifth your size to bigger than you by the end of 2027. While it's not a Microsoft-level quality of return on invested capital, they're still making 35% gross margins. So the question is, maybe it's not Microsoft's business to do this, but you've created a hyperscaler now by refusing this business, by giving away the right of first refusal, et cetera.
First of all, I don't want to take away anything from the success Oracle has had in building their business and I wish them well. The thing that I think I've answered for you is that it didn't make sense for us to go be a hoster for one model company with limited time horizon RPO. Let's just put it that way. The thing that you have to think through is not what you do in the next five years, but what you do for the next 50. We made our set of decisions. I feel very good about our OpenAI partnership and what we're doing. We have a decent book of business. We wish them a lot of success. In fact, we are buyers of Oracle capacity. We wish them success. But at this point, I think the industrial logic for what we are trying to do is pretty clear, which is that it's not about chasing… First of all, I track, by the way, your things whether it's AWS or Google and ours, which I think is super useful. But it doesn't mean I have to chase those. I have to chase them for not just the gross margin that they may represent in a period of time. What is this book of business that Microsoft uniquely can go clear, which makes sense for us to clear? That's what we'll do.
I have a question even stepping back from this, I take your point that it's a better business to be in, all else equal, to have a long tail of customers you can have higher margin from rather than serving bare metal to a few labs. But then there's a question of, which way is the industry evolving? If we believe we're on the path to smarter and smarter AIs, then why isn't the shape of the industry that the OpenAIs and Anthropics and DeepMinds are the platform on which the long tail of enterprises are actually doing business? They need bare metal, but they are the platform. What is the long tail that is directly using Azure? Because you want to use the general cognitive core.
But those models are all going to be available on Azure, so any workload that says, 'Hey, I want to use some open source model and an OpenAI model,' if you go to Azure Foundry today, you have all these models that you can provision, buy PTUs, get a Cosmos DB, get a SQL DB, get some storage, get some compute. That's what a real workload looks like. A real workload is not just an API call to a model. A real workload needs all of these things to go build an app or instantiate an application. In fact, the model companies need that to build anything. It's not just like, 'I have a token factory.' I have to have all of these things. That's the hyperscale business. And it's not on any one model, but all of these models. So if you want Grok plus, say, OpenAI plus an open source model, come to Azure Foundry, provision them, build your application. Here is a database. That's kind of what the business is. There is a separate business called just selling raw bare-metal services to model companies. And that's the argument about how much of that business you want to be in and not be in and what that is. It's a very different segment of the business, which we are in, and we also have limits to how much of it is going to crowd out the rest of it. But that's kind of at least the way I look at it.
There are sort of two questions here. One is, why couldn't you just do both? The other one is, given our estimates on what your capacity is in 2028, it's three and a half gigawatts lower. Sure, you could have dedicated that to OpenAI training and inference capacity, but you could have also dedicated that to actually just running Azure, running Microsoft 365, running GitHub Copilot.
I could have just built it and not given it to OpenAI. Or I may want to build it in a different location. I may want to build it in the UAE, I may want to build it in India, I may want to build it in Europe. One of the things is, as I said, where we have real capacity constraints right now, given the regulatory needs and the data sovereignty needs, we've got to build all over the world. First of all, stateside capacity is super important, and we want to build everything. But when I look out to 2030, I have a global view of what is Microsoft's shape of business by first-party and third-party. Third-party segmented by the frontier labs and how much they want versus the inference capacity we want to build for multiple models, and our own research compute needs. That's all going into my calculus. You're rightfully pointing out the pause, but the pause was not done because we said, 'Oh my God, we don't want to build that.' We realized that we want to build what we want to build slightly differently by both workload type as well as geo-type and timing as well. We'll keep ramping up our gigawatts, and the question is at what pace and in what location. And how do I ride Moore's law on it, which is, do I really want to overbuild three and a half in 2027 or do I want to spread that in 2027-28 knowing even… One of the biggest learnings we had even with Nvidia is that their pace increased in terms of their migrations. That was a big factor. I didn't want to go get stuck for four or five years of depreciation on one generation. In fact, Jensen's advice to me was two things. One is, get on the speed-of-light execution. That's why the execution in this Atlanta data center… I mean, it's like 90 days between when we get it and to hand off to a real workload. That's real speed-of-light execution on that front. I wanted to get good on that. And then that way I'm building each generation in scaling. And then every five years, you have something much more balanced. So it becomes literally like a flow for a large-scale industrial operation like this where you're suddenly not lopsided, where you've built up a lot in one time and then you take a massive hiatus because you're stuck with all this, to your point, in one location which may be great for training, or it may not be great for inference because I can't serve, even if it's all asynchronous, because Europe won't let me round-trip to Texas. So that's all of the things.
How do I rationalize this statement with what you've done over the last few weeks? You've announced deals with Iris Energy, with Nebius, and Lambda Labs, and there's a few more coming as well. You're going out there and securing capacity that you're renting from the neoclouds rather than having built it yourself.
It's fine for us because now when you have line of sight to demand, which can be served where people are building, it's great. In fact we will take leases, we will take build-to-suit, we'll even take GPUs-as-a-service where we don't have capacity but we need capacity and someone else has that. And by the way, I would even sort of welcome every neocloud to just be part of our marketplace. Because guess what? If they go bring their capacity into our marketplace, that customer who comes through Azure will use the neocloud, which is a great win for them, and will use compute, storage, databases, all the rest from Azure. So I'm not at all thinking of this as, 'Hey, I should just go gobble up all of that myself.'
You mentioned how this depreciating asset, in five or six years, is 75% of the TCO of a data center. And Jensen is taking a 75% margin on that. So what all the hyperscalers are trying to do is develop their own accelerator so that they can reduce this overwhelming cost for equipment, to increase their margins. And when you look at where they are, Google's way ahead of everyone else. They've been doing it for the longest. They're going to make something like five to seven million chips of their own TPUs. You look at Amazon and they're trying to make three to five million lifetime shipment units. But when we look at what Microsoft is ordering of their own chips, it's way below that number. You've had a program for just as long. What's going on with your internal chips?
It's a good question. A couple of things. One is that the thing that is the biggest competitor for any new accelerator is kind of even the previous generation of Nvidia. In a fleet, what I'm going to look at is the overall TCO. The bar I have, even for our own… By the way, I was just looking at the data for Maia 200 which looks great, except that one of the things that we learned even on the compute side… We had a lot of Intel, then we introduced AMD, and then we introduced Cobalt. That's how we scaled it. We have good existence proof of, at least in core compute, how to build your own silicon and then manage a fleet where all three are at play in some balance. Because by the way, even Google's buying Nvidia, and so is Amazon. It makes sense because Nvidia is innovating and it's the general-purpose thing. All models run on it and customer demand is there. Because if you build your own vertical thing, you better have your own model, which is either going to use it for training or inference, and you have to generate your own demand for it or subsidize the demand for it. So therefore you want to make sure you scale it appropriately. The way we are going to do it is to have a close loop between our own MAI models and our silicon, because I feel like that's what gives you the birthright to do your own silicon, where you literally have designed the microarchitecture with what you're doing, and then you keep pace with your own models. In our case, the good news here is that OpenAI has a program which we have access to.
So therefore to think that Microsoft is not going to have something that's— What level of access do you have to that? All of it. You just get the IP for all of that? So the only IP you don't have is consumer hardware?
Oh, okay. Interesting. By the way, we gave them a bunch of IP as well to bootstrap them. This is one of the reasons why they… Because we built all these supercomputers together. We built it for them and they benefited from it, rightfully so. And now as they innovate, even at the system level, we get access to all of it. And we first want to instantiate what they build, for them, but then we'll extend it. So if anything, the way I think about your question is, Microsoft wants to be a fantastic, I'll call it, speed-of-light execution partner for Nvidia. Because quite frankly that fleet is life itself. Obviously Jensen's doing super well with his margins, but the TCO has many dimensions to it and I want to be great at that TCO. On top of that, I want to be able to really work with the OpenAI lineage and the MAI lineage and the system design, knowing that we have the IP rights on both ends. Speaking of rights, you had an interview a couple days ago where you said that in the new agreement you made with OpenAI you have rights, the exclusivity, to the stateless API calls that OpenAI makes. We were sort of confused about if there's any state whatsoever. You were just mentioning a second ago that all these complicated workloads that are coming up are going to require memory and databases and storage and so forth. Is that now not stateless if ChatGPT is storing stuff on sessions?
That's the reason why. The strategic decision we made, and also accommodating for the flexibility OpenAI needed in order to be able to procure compute for… Essentially think of OpenAI having a PaaS business and a SaaS business. The SaaS business is ChatGPT. Their PaaS business is their API. That API is Azure-exclusive. The SaaS business, they can run it anywhere.
And they can partner with anyone they want to to build SaaS products?
If they want a partner and that partner wants to use a stateless API, then Azure is the place where they can get the stateless API.
