Sort of want to talk about compute capital and who delivers it. That's sort of the topic. But for those of you who don't know SemiAnalysis, we sort of cover the entire gamut of the industry. So we've got people who've built, like, worked on tools such as ASML tools, Lam Research tools for building chips. People worked on R&D and fabs. People who have worked on designing chips at places like Nvidia. People worked on infrastructure at hyperscalers like Microsoft, Amazon, Google. People have worked on compilers. People have trained models at places like Tesla and others. And you know, we cover a lot of different things. So the main business, besides, you know, shitposting, is mostly selling data. So we sell a lot of data across the industry. You know, we track every data center in the world. We track every power plant, we track all the fabs, we track all the equipment, we track every cable, etc. In addition, we do a lot of testing. So ClusterMax, which you know was talked about a little bit, we benchmark all the clouds, test them for security, lifecycle, network performance, storage performance, etc. In addition, we have InferenceX, completely open source, open benchmarking of various chips, including seven kinds of GPUs from Nvidia, three kinds from AMD, TPUs, and even OpenAI's internal chip. So we benchmark those on various open source models across sweeping, you know, sorts of all sorts of, you know, parallelism configs, prefill-decode disaggregation, spec, all these sorts of things. So we sweep that. It runs every day across 2,000 chips worldwide. We're adding Trainium in the coming months as well. In addition, we started up a lab. So in Oregon, we have a lab where we reverse engineer chips. We delayer them layer by layer, take high-resolution photos of them, you know, sort of dissect into the architecture, look at the transistor architecture. In the top left is sort of Intel's 18A and Samsung's 2 nanometer, right? Everyone knows TSMC is the best, but how far behind are Samsung and Intel? And so, like, you know, we actually have, you know, tens of millions of dollars of equipment in terms of, you know, various electron microscopes and such to characterize and track the chip architecture itself. You know, we extend all the way up into like tokenomics, which is sort of tracking the supply chain of, you know, sort of who's selling tokens where, what are the buyers, what's the cost, what's the margin, etc. So yeah, started six years ago, just me, bootstrapped. Now it's 115 people. So pretty sick. But today going to talk about the compute race. It's no surprise to anyone who's tried to buy chips or rent chips over the last year that it's extremely tight. And the reason for that is it's OpenAI and Google. I mean, OpenAI and Anthropic primarily, they're just buying everything out there. And so when you look at the total industry in terms of gigawatts that exist, you know, this is including AI and non-AI, you know, in 2024 the entire world had 60 gigawatts of compute globally by the end of 2024, and by the end of this year we're at over 100 already. So it's nearly doubled in just two years, which is an incredible pace, right? It took us, you know, 50 years to get to 50, and then, you know, just a couple years to get to 100. And that demand is primarily being driven by the AI labs. So by the end of last year they had about four gigawatts total between Anthropic and OpenAI, and this year they'll have about 10, and then the year after that they'll have over 20. So they're taking an incremental huge amount of the compute, right? If you think about 2027, you know, there's about 12 gigawatts that both of them add, and yet, you know, globally we're only adding about 40. So they're taking like, you know, 25, 30%, 40% of the compute that's being added, and then you stack on the hyperscalers, and so this is a very sort of dire situation, right? End of last year, you know, Anthropic's revenue started to go ballistic with Claude Code, and so they started buying compute like crazy. Meta started going crazy buying compute, Open started going crazy buying compute. Now it's like impossible to buy compute for a lot of folks. And so, you know, in 2028, based on what we've seen signed and such, the AI labs are going to add on the order of 30 gigawatts in 2028 alone. So it's pretty crazy how much is being added. I think, you know, one interesting thing is that like the spend difference between the two, right? What amount is going to training and what amount is going to inference is actually very lopsided. You would think most of it's going to inference, but they're actually mostly spending on training. And most of their compute is, you know, all of Claude Code is served by a gigawatt of compute despite the fact that Anthropic has four gigawatts today. So it's pretty shocking how little compute they spend on inference relative to training. I think, like, you know, going back to the point of what compute costs, you know, in October was sort of the trough, right? You know, sort of $1.70 for H100, and you could find it as low as $1.45, and if the cloud sucked, you could find it even less. Price-wise, nowadays we see, you know, transactions, you know, as high as 240 to 320, and the midpoint being 280. So some of the business we do is we broker a lot of compute, and so that lets us get a lot of price accuracy because we know what stuff's actually transacting at, and then we become friends with a lot of companies and they tell us what they pay. And so the price has really, really spiked. In December of last year we said that prices of compute were going to spike, and we're surprised by how much they've continued to spike. And it's driven by, again, like the massive pace. And going to the pricing data, right? Even H100, which is $2.80 for a year, you know, no one, you know, you'd be crazy to sign that two years ago, but now it's like sort of like people are desperate to sign something in that range. And the price keeps going up, right? And if you look at one year, three year, five year, like it's very hard to buy any compute for a year, which sucks for any startup because it's like, well, look, I don't know if I'm going to be in business more than a year from now, so I should only buy compute for a year. So ultimately it's a really challenging market. And the reason for that is everyone wants to sell tokens. You know, it costs like $9 million to buy and run, rent. If you talk about like, okay, it costs like $40 million to build the cluster, but to operate it over five years, it's about 9 million a year, $40 million to build the cluster and then the operational cost. So $9 million a year for five years. And that's about the cost if you build your own B300 cluster. If you're renting it from the clouds, you know, you're lucky if you're paying $14 million a year for a megawatt. And then you look at what does it mean when you sell open source models. If you're serving DeepSeek V4.1 Flash and you have a 60% utilization, right? So you're, you know, only selling 60% of the tokens that you could generate. You're making $70 million a year. So the profitability is ridiculous. And this is on the open source InferenceX agent benchmarks that we've done. They're containers you can download and just run. So it's pretty based on top of vLLM, SGLang, etc. So it's pretty ridiculous that this is how, you know, this is how much money you could make. Even if you say, oh, utilization is going to be 30%, you're still going to make a ridiculous sum of money. And so running inference at small scale is like not that difficult. Once you start getting to like modal scale where you have 25 clouds and you're routing stuff between all of them and all that becomes much more challenging on an inference perspective. But literally there's like dozens of people that I know who their business is they've rented some GPUs and they're serving inference and they're making a ton of money. And so this is, this is like partially why like you could blame the labs, you could blame like the labs, you know, buying all the compute, but ultimately a lot of it is also just the fact that, you know, it's so profitable for even open source model running inference. And that's before you get into, you know, all these fancy speculative decoding techniques and all these fancy things that all the really good companies that run inference do. Just like the baseline what we've open sourced with, you know, four engineers and built on top of the vLLM and SGLang stacks. So that's sort of why pricing just continues to rise. You know, it's just if you can find B300s for $6 an hour, you should sign it and you should sell tokens, right? Like it's, and that's an insane price, but this is just like it is a correct decision to make. Of course, all the clouds want you to prepay, so then that makes it a lot harder. So, but that's basically, you know, a big reason why costs keep going up. Anthropic, you know, they're, you know, when they're serving Fable or when they're serving Opus 5.5, it's north of 100 million, and that's observed, and that's in sort of their, some of their materials that they've been releasing through their IPO process to investors. So it's a tremendous difference. Even the closed labs are not, you know, not the only reason why the pricing is spiking. And if you look at like who's capturing the value, right, it's pretty much all the token provider. Hence all the clouds can push price, and you know, frankly, they can push price a lot harder. I would not be surprised if H100s are north of $3 and B300s are north of, you know, $7 or what have you. Like, I would not be surprised because the open source models are just getting better and better, and the profitability of them is getting better and better. You know, with GLM 5.3 and DeepSeek V4.1, ones that are not even that huge, let alone the big models that are coming. So the key thing here is like who actually can deliver on compute. So we did ClusterMax, which is ranking all the clouds. There's 323 providers that we've tracked, 77 that we actually reviewed, and 19 that we gave medallions to. And what that means is sort of like, you know, CoreWeave and Nebius are the best providers in our view. You know, CoreWeave will not sell you compute for the next six months. They're sold out. Nebius will make you prepay a full year upfront or six months upfront at some ridiculous price, but that's the only way that you can get compute from those guys. Oracle, Google are really good as well. Google's been moving up. The first ClusterMax, they were bronze, and then they moved up and up and up, but a lot of downgrades, right? We've been trying to move the bar up, and so there have been a lot of folks who have been downgraded over time. And that's just because as ClusterMax 1, 2, 3 came out, we've added a lot more stuff, right? With ClusterMax v3, we asked some models to hack the neoclouds, and there were quite a few that just like, you know, there's some open vulnerability on the internet that, you know, has already been published, and the model can just find that and exploit that and access other customers' storage. There was even an instance where a cloud that we asked an open-source model, which is behind what OpenAI and Anthropic have, we were able to gain access to Slurm across the entire, and many other things across the entire cluster, and we were able to see a certain country's defense industry, including their national security sort of intelligence agency, were on that cluster and their job names. And we kind of like immediately closed out because, you know, while no one's in that country at SemiAnalysis, no one wants to go to jail. But we immediately closed out and we're like, guys, this is ridiculous. And we published an article called 'Most Neoclouds Suck at Security.' And then Ilya went on a whole rant about how AGI will take over neoclouds, and that's going to be impossible for us to, you know, sort of save ourselves from unless these neoclouds get good at security. But anyways, you know, a lot of the criteria is security, but it's also things like network performance, storage performance, lifecycle, you know, how is your managed Slurm on Kubernetes or managed Kubernetes, you know, all these different aspects. There's dozens of criteria. I welcome you to read the report. It's completely free and open. You know, the simple of it is that, you know, I mentioned some of the upgrades, downgrades-wise. Azure got downgraded to silver, Crusoe got downgraded to, together downgraded to bronze, and AWS got downgraded to bronze. And you wouldn't imagine like, what, how is AWS bronze? It's like, well, it turns out to like get a cluster, you have to talk to like 10 people, and it takes forever, and their networking is a little bit difficult to deal with. Like it's just like not an easy user experience. Their HyperPod offering is not that great. Some of the other stuff that we did was we would inject failures. So we'd inject XID 719, I think there's an Nvidia like sort of error. You inject that into the GPU and you see how long it takes the cloud to catch it. And many clouds would not catch it until you alerted them. And many clouds even once you alerted them would take days to respond. And so I think, you know, that's like a bare minimum test, right? Your GPUs are going to be flaky. They're going to mess up. You tell your cloud, they should fix it. In fact, they should catch that in their own monitoring and fix it themselves. So I think there's a lot of different details about that. Again, read the full report. The other question is sort of who pays for it. So one of the things that we've said is by end of 2029 we expect 11 trillion dollars of capex globally on AI, five of that being from cash and equity, cash equity, and seven of that being debt. And that's a tremendously ridiculous number to most people. And this has been our view for a while. And then now over the last sort of two months, if you watch debt markets, the US 10-year yield has been spiking like crazy. And it's caused a lot of pandemonium in the market. So, you know, the capex that's coming online is humongous. And most of that capex is for the chips themselves, but a large amount of it is also for the data centers, energy, etc. The power plants that you're building on site of the data center so you can get them up sooner. The other aspect of this is what lenders do, right? When you look at the price of debt, if you go back a year ago, it used to be much cheaper, but as you move forward, it's going, getting more and more expensive. So all these neoclouds, they would love to build more compute, they're tremendously profitable, but if they have 100 megawatts and now they're trying to scale to a gigawatt, who's going to loan them money? The only way they can get a loan is if they go sell that gigawatt to one of the hyperscalers, and then one of the hyperscalers, you can turn around to the debt market and be like, look, Microsoft's running this or Google's running this, okay, great, you get a loan, but then none of that compute gets onto the market for the general industry. So there's a lot of like creative stuff that people are trying to do to make it so all of the compute just doesn't go to the AI labs and the hyperscalers. Because, you know, next year, if you look at all the compute that's been signed, it's something on the order of 65, 70% of it is going to hyperscalers and the AI labs. And that situation will just only get more dire as interest rates rise and the only people who can, you know, afford to sort of have a balance sheet. And so when we look at Nvidia, what they've been doing is very creative. You know, this chart has a lot of stuff on there, so maybe if you want to look at it, like take a photo and zoom in. But ultimately, like Nvidia recognizes this is a big problem, right? You know, we're here to support startups. You know, everyone wants startups to innovate. Everyone wants people to actually be able to build their own AI solutions. If you're trying to rent a cluster as a startup, they're going to ask you to pay half of it down today. Like that's just the reality of it, and then they're going to tell you it's going to come in three months or six months or nine months. Go talk to any of the neolabs that have raised a bunch of money recently. It's like it's hell for them to get compute. So what Nvidia started doing is saying, we'll backstop your cluster, right? If you're a cloud, you're going to build a cluster. We'll either say, hey, you can give us a revenue share on that cluster, and then you can use our name as a price floor to go get the loan, or you can use our, you know, or some other ways of doing that. And so Nvidia is loading up their balance sheet with all these compute commitments because it's the only way that the open ecosystem can get compute. Otherwise, no one's going to build that compute for them because no one's going to loan you money to build a cluster unless you have someone who is a very strong balance sheet backing you.
Yeah. So that's the, you know, sort of dire situation, and Nvidia is sort of stepping in to try and help that, but there's got to be a lot more creative solutions. So you've seen a lot of companies are now building their own clusters, right? Anyone who's doing inference, Fireworks or Modal or any of these other companies are now trying to build their own clusters. You look at many of the neolabs, they're trying to build clusters. And that's because the only way they can do that is if they go get, you know, a loan themselves, get Nvidia to back it, and build the cluster themselves. Otherwise, they're just in such dire situations, you know? So what does a gigawatt cost, right? And I think that's very important. If you look at like the guarantees, there's a few different deals that they've done. In terms of how they back these clusters, there's three different types that they've done. One is what they did with SoftBank Energy, where they only guaranteed the land, the power, and the shell. And SoftBank Energy is building this like 9 gigawatt facility in Ohio. That'll be up in like 2030. The whole thing is earmarked for OpenAI though. So like no one can actually access that or touch it unless OpenAI goes bankrupt, which seems unlikely. And if they do, then we're all... But you know, anyways, the other one is like Fermis. So Fermis is a company who's Australian, Singaporean, and they're building a lot of data centers in Australia. And what they did is Nvidia guaranteed, hey, if you, we'll buy, we'll rent this cluster at this price. And that price is basically Fermis makes no money. But they don't lose money. But if they have a guarantee now that they're going to sell it, so anyone's willing to loan money to their cluster. And now every dollar that they, every dollar per hour that they sell over that price, Nvidia gets to take half of it. Which is a ridiculous, like, you know, profiteering move. I think I love it. It's pretty smart. Jensen's going to make a lot of money assuming AI keeps going, which I think it will. Then there's other stuff which is I probably shouldn't explain a residual value guarantee because we have 45 seconds left. So, but you can look that up. There's a whole residual value guarantee which is pretty interesting. And then if you look at like backstops in perspective, you know, and you look across how much compute is being added each year, Nvidia's currently done this with six and a half gigawatts of compute where they've backstopped it, wrapped it, and that's going to be sold to the open market, not just to, you know, the AI labs and the hyperscalers. The most they can possibly do is 45 gigawatts. So even then that's only a tiny, tiny percentage of the total compute that's being deployed through 2030 over the next four years. And so this is, you know, again, the situation is a little dire in terms of that. But yeah, anyways, concluding, tokens are how stuff moves. You know, there's 100 trillion plus tokens a day just across a few providers. The ARR of these companies is spiking. You know, all of them are raising tons of money and generating tons of revenue. So hopefully they can get compute because otherwise we are all slaves to Anthropic and OpenAI.
