Dylan Patel — The single biggest bottleneck to scaling AI compute

So when you talk about the CapEx of these hyperscalers being on the order of $600 billion, and you look across the rest of the supply chain, it gets you to the order of a trillion dollars. A portion of this is immediately for compute going online this year: the chips and the other parts of CapEx that get paid this year. But there's a lot of setup CapEx as well. When we're talking about 20 gigawatts of incremental added capacity this year in America, a portion of this is not spent this year. A portion of that CapEx was actually spent the prior year. When you look at Google having $180 billion, a big chunk of that is spent on turbine deposits for '28 and '29. A chunk of that is spent on data center construction for '27. A chunk of that is spent on power purchasing agreements, down payments, and all these other things they're doing further out into the future so they can set up this super fast scaling. This applies to all the hyperscalers and other people in the supply chain. So with roughly 20 gigawatts deployed this year, a big chunk is hyperscalers, and a chunk is not. For all of these companies, their biggest customers are Anthropic and OpenAI. Anthropic and OpenAI are at roughly two to two-and-a-half gigawatts right now, and they're trying to scale much larger. If you look at what Anthropic has done over the last few months, with $4 billion or $6 billion in revenue added, we can just draw a straight line and say they'll add another $6 billion of revenue a month. People would argue that's bearish, and that they should go faster. What that implies is they're going to add $60 billion of revenue across the next ten months. At the current gross margins Anthropic had, as last reported by media, that would imply they have roughly $40 billion of compute spend for that inference, for that $60 billion of revenue. That $40 billion of compute, at roughly $10 billion a gigawatt in rental costs, means they need to add four gigawatts of inference capacity just to grow revenue. That's assuming their research and development training fleet stays flat. In a sense, Anthropic needs to get to well above five gigawatts by the end of this year. It's going to be really tough for them to get there, but it's possible.

They have to go to lower-quality providers that they would not have gone to before. Anthropic historically had the best quality providers, like Google and Amazon, the biggest companies in the world. Now Microsoft is expanding across the supply chain, and they're going to other newer players. OpenAI has been a bit more aggressive on going to many players. Yes, they have tons of capacity from Microsoft, Google, and Amazon, but they also have tons with CoreWeave and Oracle. They've gone to random companies, or companies one would think are random, like SoftBank Energy, who has never built a data center in their life but is building data centers now for OpenAI. They've gone to many others, like NScale, to get capacity. There's this conundrum for Anthropic because they were so conservative on compute, because they didn't want to go crazy. In some sense, a lot of the financial freakouts in the second half of last year were because, "OpenAI signed all these deals but they didn't have the money to pay for them…" Okay, Oracle's stock is going to tank, CoreWeave's stock is going to tank. All these companies' stocks tanked, and credit markets went crazy because people thought the end buyer couldn't pay for this. Now it's like, "Oh wait, they raised a ton of money. Okay, fine, they can pay for it." Anthropic was a lot more conservative. They were like, "We'll sign contracts, but we'll be principled. We'll purposely undershoot what we think we can possibly do and be conservative because we don't want to potentially go bankrupt."

To acquire excess compute, yes, there is capacity at hyperscalers. Not all contracts for compute are long-term, five-year deals. There's compute from 2023 or 2024, or H100s from 2025, that were signed at shorter terms. The vast majority of OpenAI's compute is signed on five-year deals, but there were many other customers that had one-year, two-year, three-year, or six-month deals, on demand. As these contracts roll off, who is the participant in the market most willing to pay price? In this sense, we've seen H100 prices inflect a lot and go up. People are willing to sign long-term deals for above $2 even. I've seen deals where certain AI labs—I'm being a little bit vague here for a reason—have signed at as high as $2.40 for two to three years for H100s. If you think about the margin, it costs $1.40 to build Hopper, across five years. Now, two years in, you're signing deals for two to three years at $2.40? Those margins are way higher. Now you can crowd out all of these other suppliers, whether Amazon had these, or CoreWeave, or Together AI, or Nebius, or whoever it is. These neoclouds are the firms that had a higher percentage of Hopper in general because they were more aggressive on it. They also tended to sign shorter-term deals, not CoreWeave but the others. So if I want Hopper, there is some capacity out there. Also, while most of the capacity at an Oracle or a CoreWeave is signed for a long-term deal in terms of Blackwell, anything that's going online this quarter is already sold. In some cases, they're not even hitting all the numbers they promised they would sell because there are some data center delays, not just those two, but Nebius, Microsoft, Amazon, and Google. But there are a lot of neoclouds, as well as some of the hyperscalers, who have capacity they're building that they haven't sold yet, or capacity they were going to allocate to some internal use that is not necessarily super AGI-focused, that they may now turn around and sell. Or in the case of Anthropic, they don't have to have all the compute directly. Amazon can have the compute and serve Bedrock, or Google can have the compute and serve Vertex, or Microsoft can have the compute and serve Foundry, and then do a revenue share with Anthropic, or vice versa.

Right, there's a trade-off there. But at the same time, for a solid four months, everyone was saying to OpenAI, "We're not going to sign deals with you." That sounds crazy, but it was because, "you don't have the money." Now everyone's saying, "OpenAI, we believed you the whole time. We can sign any deal because you've raised all this money." Anthropic is constrained in that sense. There are not that many incremental buyers of compute yet, because Anthropic hit the capability tier first where their revenue is mooning. That's interesting. Otherwise you might think having the best model is an extremely depreciating asset, because three months later you don't have the best model. But the reason it's important is that you can sign these deals, lock in the compute in advance, and get better prices.

There's a few strings to pull on there. One is, what happens to depreciation of GPUs? I guess I didn't answer your prior question, which is that I think Anthropic will be able to get to five gigawatts-ish, maybe a little bit more by the end of the year through themselves as well as their product being served through Bedrock, Vertex, or Foundry. I think they'll be able to get to five or six gigawatts, which is way above their initial plans. OpenAI will be roughly the same, actually a little bit higher based on our numbers. But anyway, the depreciation cycle of a GPU. Michael Burry was saying it's three years or less. That's sort of his argument. There are two lenses to look at this. Mechanically, there's a TCO model, total cost of ownership of a GPU, where we project pricing out for GPUs and build up the total cost of a cluster. There are a number of costs: your data center cost, your networking cost, your smart hands and people in the data center swapping stuff out. There's your spare parts, your actual chip cost, your server cost. All these various costs get lumped together. There's some depreciation cycles on it, certain credit costs on it. You build up to, "Hey, an H100 costs $1.40/hour to deploy at volume across five years if your depreciation is five years." If you sign a deal at $2/hour for those five years, your gross margin is roughly 35%. It's a little bit above that. If you sign it for $1.90, it's 35% roughly. Then you assume at that fifth year, the GPU falls off a bus and is dead. In some cases, the argument people are making is if you didn't sign a long-term deal, because every two years NVIDIA is tripling or quadrupling the performance while only 2X-ing or 50% increasing the price… Then the price of an H100… Sure maybe the value in the market was $2 at 35% gross margins in 2024, but in 2026, when Blackwell is in super high volume and deploying millions a year, you're actually now worth $1/hour. And when Rubin in '27 is in super high volume—even though it starts shipping this year, it's super high volume next year—doing millions of chips a year deployed into clouds, you've got another 3X in performance, another 50% or 2X in price, then the Hopper is only worth $0.70/hour. So the price of a GPU would continue to fall. That's one lens. The other lens is, what is the utility you get out of the chip? If you could build infinite Rubin or infinite of the newest chip, then yes, that's exactly what would happen. The price of a Hopper would fall at a spot or short-term contract rate as the new chips come out and the price per performance goes up. But because you are so limited on semiconductors and deployment timelines, what actually prices these chips is not the comparative thing I can buy today, but rather what is the value I can derive out of this chip today. In that sense, let's take GPT-5.4. GPT-5.4 is both way cheaper to run than GPT-4 and has fewer active parameters. It's much smaller, in that sense of active parameter, because it's a sparser MoE versus GPT-4 being a coarser MoE. There's also been so many other advancements in training, RL, model architecture, and data qualities that have made GPT-5.4 way better than GPT-4. And it's cheaper to serve. When you look at an H100, it can serve more tokens per GPU of 5.4 than if you had ran GPT-4 on it. So it's producing more tokens of a model that is of higher quality. What is the maximum TAM for GPT-4 tokens? Maybe it was a few billion dollars, maybe it was tens of billions of dollars. Adoption takes time. For GPT-5.4, that number is probably north of a hundred billion. But there's an adoption lag, there's competition, and there's the constant improvements that everyone else is having. If improvements stopped here, the value of an H100 is now predicated on the value that GPT-5.4 can get out of it instead of the value that GPT-4 can get out of it. These labs are in a competitive environment, so their margins can't go to infinity. You sort of have this dynamic that is quite interesting in that an H100 is worth more today than it was three years ago.

Oh, yeah, you've got petabytes? Name a petabyte of ones and zeros, bro. Name me a string.

I think going back to the earlier view that if the models are so powerful, the value of a GPU goes up over time, right now only OpenAI and Anthropic have that viewpoint. But as we approach further out, everyone is going to be able to see that value skyrocket per GPU. So in that sense, you should commit now to compute. Interestingly, in Anthropic fashion, there's a bit of a meme that they have commitment issues and are sort of polyamorous. Not Dario, but this is a bit of a meme.

Interesting. I think that makes a ton of sense. We just see all of the volumes are on the best models today, all the revenue is on the best models today. In a compute-limited world, two things happen. One, companies that don't have commitment issues and have these five-year contracts for compute have locked in a humongous margin advantage. They've locked in compute for five years at the price it transacted at two, three, or five years ago. Whereas if you're three years into that five-year contract and someone else's two-year or three-year contract rolled off, and now they're trying to buy that at modern pricing, when it's priced to the value of models, the price is going to be up a lot more. So the person who committed early has better margins in general. The percentage of the market that is in long-term contracts is much larger than the percentage of the market in short-term contracts that can be this flex capacity you add at the last second. At the same time, where does the margin go? Because models get more valuable, how much can the cloud players flex their pricing? If you look at CoreWeave, their average term duration is over three years right now. For ninety-eight percent plus of their compute, it's over three years. They end up with this conundrum where they can't actually flex price. But every year they're adding incrementally way more capacity than they had previously. This year alone, Meta's adding as much capacity as they had in their entire fleet of compute and data centers for all purposes for serving WhatsApp, Instagram, and Facebook in 2022, and doing AI. They're adding that alone this year. In the same sense, you talk about Meta doing that, CoreWeave, Google, and Amazon, all these companies are adding insane amounts of compute year on year. That new compute gets transacted at the new price. In a sense, yes, you've locked in, as long as we're in a takeoff. "Oh, OpenAI went from six hundred megawatts to two gigawatts last year, and from two gigawatts to six plus this year, and six to twelve next year." The incremental added compute is where all the cost is, not the prior long-term contracts. Then who holds the cards is the infra providers for charging margin. Now the cloud players, the neoclouds, or the hyperscalers can charge the margin. They can to some extent, but then as you go upstream to who has access to all the memory and logic capacity, it's Nvidia for the most part. They've signed a lot of long-term contracts. They've got ninety billion dollars of long-term contracts today, and they're negotiating three-year deals today with the memory vendors. You've got Amazon and Google through Broadcom, Amazon directly, and AMD. These companies hold all the cards because they've secured the capacity. TSMC is not raising prices, but memory vendors are, to some extent, raising a lot of price. They're going to double or triple price again, but then they're also signing these long-term deals. Who is able to accrue all the margin dollars is potentially the cloud, potentially the chip vendors, and the memory vendors, until TSMC or ASML break out and say, "No, we're going to charge a lot more." But at the same time, do the model vendors get to charge crazy margins? At least this year, we're going to see margins for the model vendors go up a lot. Because they're so capacity constrained, they have to destroy demand. There's no way Anthropic can continue at the current pace without destroying demand.

There are a couple points here. On 3 nm, if we go back to last year, the vast majority of 3 nm was Apple. Apple is being moved to 2 nm. Memory prices are going up, so Apple's volumes may go down. As memory prices go up, either they cut margin or they move on. There's some time lag because they have long-term contracts, but Apple likely reduces demand or moves to 2 nm faster, where 2 nm is only capable of mobile chips today. In the future, AI chips will move there. So Apple has that. Apple is also talking to third-party vendors because they're getting squeezed out of TSMC a little bit. TSMC's margins on high-performance computing—HPC, AI chips, et cetera—are higher than they are for mobile, because they have a bigger advantage in HPC than they do in mobile. When you look at TSMC's running calculus here, they're actually providing really good allocations to companies that are doing CPUs. When you think about Amazon having Trainium and Graviton, both of those are on 3 nm, Graviton being their CPU, Trainium being their AI chip. TSMC is much more excited to give allocation to Graviton than they are to Trainium because they view the CPU business as more stable, long-term growth. As a company that is conservative and doesn't want to ride cycles of growth too hard, you actually want to allocate to the market that is more stable with a lower growth rate first before you allocate all the incremental capacity to the fast growth rate market. That is the case generally. Same for AMD. The allocations they get on their CPUs, TSMC is much more excited about those than they are for GPUs. Likewise for Amazon. Nvidia is a bit unique because yes, they have CPUs, they make switches, they make networking, NVLink, InfiniBand, Ethernet, NICs. By and large, most of these things will be on 3 nm by the end of this year with the Rubin launch and all the chips in that family, the GPU being the most important one. Yet Nvidia is getting the majority of supply. Part of this is because you look at the market and TSMC and others forecast market demand in many ways, but it's also the market signal. The market signaled, "Hey, we need this much capacity next year. We need this much. We'll sign non-cancelable, non-returnable. We may even pay deposits." Nvidia just did it way earlier than Google or Amazon. In some cases, Google and Amazon had stumbling blocks. One of the chips got delayed slightly by a couple quarters. Trainium and all these sorts of things happened. In that case, there was a huge sort of, "Well, these guys are delaying, but Nvidia is wanting more, more, more, more. And we are checking with the rest of the supply chain, is there enough capacity?" They're going to all the PCB vendors and saying, "Is there enough PCB?" Victory Giant is one of the largest suppliers of PCBs to Nvidia, and they're a Chinese company. All the PCBs come from China, or many of them. They're like, "Do you have enough PCB capacity? Great. Hey memory vendors, who has all the memory capacity? Okay, Nvidia does. Great." When you look at who is AGI-pilled enough to buy compute on long timelines at levels that seem ridiculous to people who aren't AGI-pilled—but nonetheless, they're willing to pay a pretty good margin and sign it now because they view in the future that ratio is screwed up—the same thing happens with the supply chain for semiconductors. I don't think Nvidia is quite AGI-pilled. Jensen doesn't believe software is going to be fully automated and all these things.

It's AI chips.

Yeah. I think it's a broader term, AI is within that, but also physics modeling and simulations.

I think he's embracing it, but I just don't think he's AGI-pilled like Dario or Sam. But he's still way, way more AGI-pilled than Google was in Q3 of last year, or Amazon was in Q3 of last year, and he saw way more demand. The reason is pretty simple. You can see all the data center construction. He's like, "Okay, I want to have this market share." We have all the data centers tracked, and there's a lot of data centers that could be one or the other. To some extent, Google and Amazon, Google especially, even though their TPU is just better for them to deploy, they have to deploy a crap load of GPUs because they don't have enough TPUs to fill up their data centers. They can't get them fabbed.

Yes.

This is again a problem of… DeepMind people were like, "This is insane. Why did we do this?" But Google Cloud people and Google executives saw a different thought process. You and I know the compute team at Anthropic. Both of the main people came from Google. They saw this dislocation, they negotiated a deal, and they were able to get access to this compute before Google realized. The chain of events, at least from our data that we found, was in early Q3, over the course of six weeks, we saw capacity on TPUs go up by a significant amount. It went up multiple times in those six weeks. There were multiple requests. Google even had to go to TSMC and explain to them why they needed this increase in capacity because it was so sudden. A lot of that capacity increase was for selling to Anthropic. Because Anthropic saw it before Google. And then Google had Nano Banana and Gemini 3 which caused their user metrics to skyrocket. Then leadership at Google was like, "Oh." Then they started making the statement that we have to double compute every six months, or whatever the exact number was. They really woke up a lot more, and then they went to TSMC and said, "We want more. We want more." TSMC replied, "Sorry guys, we're sold out. We can maybe get 5-10% more for 2026, but really we're going to work on 2027." There was this information asymmetry among the labs, in my mind. I don't know exactly. It's the narrative I've spun myself from seeing all the data in the supply chain on wafer orders and what's going on with the data centers that Anthropic and Fluidstack signed. It's pretty clear to me that Google screwed up. You can see this from Google's Gemini ARR. They had next to nothing in Q1 to Q3—in Q3 a little bit once they started inflecting. But in Q4 they reached $5 billion in revenue on an ARR basis. It's clear Google didn't see revenue skyrocket initially. In a sense, Anthropic had a little bit of commitment issues before their ARR exploded, even though they had far more information asymmetry and saw what was coming down the pipe. Google is going to be more conservative than Anthropic and Google had even less ARR. So they were just not willing to do it, and then they realized they should do it. Since then, Google has gotten absurdly AGI-pilled in terms of what they're doing. They bought an energy company. They're putting deposits down for turbines. They're buying a ridiculous percentage of powered land. They're going to utilities and negotiating long-term agreements. They're doing this on the data center and power side very aggressively. I think Google woke up towards the end of last year, but it took them some time.

Buy my data. You charge for that kind of information.

The biggest bottleneck is compute. For that, the longest lead time supply chains are not power or data centers. They're actually the semiconductor supply chains themselves. It switches back from power and data centers as a major bottleneck to chips. In the chip supply chain, there's a number of different bottlenecks. There's memory, logic wafers from TSMC, and the fabs themselves. Construction of the fabs takes two to three years, versus a data center which takes less than a year. We've seen Amazon build data centers in as fast as eight months. There's a big difference in lead times because of the complexity of building the fab that actually makes the chips. The tools also have really long lead times. The bottlenecks, as we've scaled, have shifted based on what the supply chain is currently not able to do. It was CoWoS, power, and data centers, but those were all shorter lead time items. CoWoS is a much simpler process of packaging chips together. Power and data centers are ultimately way simpler than the actual manufacturing of the chips. There's been some sliding of capacity across mobile or PC to data center chips, which has been somewhat fungible. Whereas CoWoS, power, and data centers have had to start anew as supply chains. But now there's no more capacity for the mobile and PC industries—which used to be the majority of the semiconductor industry—to shift over to AI. Nvidia is now the largest customer at TSMC and SK Hynix, the largest memory manufacturer. It's sort of impossible for the sliding of resources away from the common person's PCs and smartphones to shift any more towards the AI chips. So now the question is how do we scale AI chip production? That's the biggest bottleneck as we go to 2030.

To scale compute further, there are different bottlenecks this year and next year, but ultimately by 2028 or 2029, the bottleneck falls to the lowest rung on the supply chain, which is ASML. ASML makes the world's most complicated machine: an EUV tool. The selling price for those is $300-400 million. Currently, they can make about 70. Next year, they'll get to 80. Even under very aggressive supply chain expansion, they only get to a little bit over 100 by the end of the decade. What does that mean? They can make a hundred of these tools by the end of the decade, and 70 right now. How does that actually translate to AI compute? We see all these numbers from Sam Altman and many others across the supply chain: gigawatts, gigawatts, gigawatts. How many gigawatts are we adding? We see Elon saying a hundred gigawatts in space.

A year. The problem with any of these numbers, or the challenge to these numbers, is actually not the power or the data center. We can dive into that, but it's manufacturing the chips. Take a gigawatt of Nvidia's Rubin chips. Rubin is announced at GTC, I believe the week this podcast goes live. To make a gigawatt worth of data center capacity of Nvidia's latest chip that they're releasing towards the end of this year, you need a few different wafer technologies. You need about 55,000 wafers of 3 nm. You need about 6,000 wafers of 5 nm, and then you need about 170,000 wafers of DRAM memory. Across these three different buckets, each requires different amounts of EUV. When you manufacture a wafer, there are thousands and thousands of process steps where you're depositing material and removing them. But the key critical step—which at least in advanced logic is 30% of the cost of the chip—is something that doesn't actually put anything on the wafer. You take the wafer, you deposit photoresist, which is a chemical that chemically changes when you expose it to light. Then you stick it into the EUV tool, which shines light at it in a certain way. It patterns it. There's what's called a mask, which is effectively a stencil for the design. When you look at a leading-edge 3 nm wafer, it has 70 or so masks, 70 or so layers of lithography, but 20 of them are the most advanced EUV. If you need 55,000 wafers for a gigawatt, and you do 20 EUV passes per wafer, you can do the math. That's 1.1 million passes of EUV for a single gigawatt. It's pretty simple. Once you add the rest of the stuff, it ends up being 2 million, across 5 nm and all the memory. You're at roughly 2 million EUV passes for a single gigawatt. These tools are very complicated. When you think about what it's doing across a wafer, it's taking the wafer and scanning and stepping across. It does this dozens of times across the whole wafer. When you're talking about how many EUV passes, that's the entire wafer being exposed at a certain rate. An EUV tool can do roughly 75 wafers per hour, and the tool is up roughly 90% of the time. In the end, you need about three and a half EUV tools to do the 2 million EUV wafer passes for the gigawatt.

Dude, let's LBO that. Let's LBO it.

That's completely compatible, if you think about it. TSMC and the entire ecosystem have something like 250 to 300 EUV tools already. Then you stack on 70 this year, 80 next year, growing to 100 by 2030. You're at 700 EUV tools by the end of the decade. 700 EUV tools, at three and a half tools per gigawatt—assuming it's all allocated to AI, which it's not—gets you to 200 gigawatts worth of AI chips for the data centers to deploy. Sam wants 52 gigawatts a year. He's only taking 25% share then. Obviously, there's some share given to mobile and PC, assuming we're even allowed to have consumer goods still and we don't get priced out of them. But roughly, he's saying 25% market share of the total chips fabbed. That's very reasonable given that this year alone, I think he's going to have access to 25% of the Blackwell GPUs that are deployed. It's not that crazy.

ASML's been shipping EUV tools now for roughly a decade, but it only entered mass volume production around 2020. The tool's not the same. Back then, the tools were even lower throughput. There are various specifications around them called overlay. I was mentioning you're stacking layers on top of each other. You'll do some EUV, you'll do a bunch of different process steps—depositing stuff, etching stuff, cleaning the wafer—dozens of those steps before you do another EUV layer. There's a spec called overlay, which is: you did all this work, you drew these lines on the wafer, now I want to draw these dots. Let's say I want to draw these dots to connect these lines of metal to holes, and then the next layer up is another set of lines going perpendicular, so now you're connecting wires going perpendicular to each other. You have to be able to land them on top of each other. It's called overlay. Overlay is a spec that's been improved rapidly by ASML. Wafer throughput has been improved rapidly by ASML. The price of the tool has gone up, but not as much as the capabilities of the tool. Initially, the EUV tools were $150 million. Over time, they're now $400 million as I look out to 2028. But the capabilities of the tools have more than doubled as well, especially on throughput and overlay accuracy, which is the ability to accurately align the subsequent passes on top of each other even though you do tons of steps between. ASML is improving super rapidly. It's also noteworthy to say that ASML is maybe one of the most generous companies in the world. They have this linchpin thing. No one has anything competitive. Maybe China will have some EUV by the end of the decade, but no one else has anything even close to EUV, and yet they haven't taken price and margins up like crazy. You go ask some other folks that we talk to all the time, like Leopold, and they're like, "Let's have the price go up." Because they can. The margin is there. You can take the margin. Nvidia takes the margin. Memory players are taking the margin. But ASML has never raised the price more than they've increased the capability of the tool. In a sense, they've always provided net benefit to their customers. It's not that the tool is stagnant, it's just that these tools are old. Yes, you can upgrade them some, and the new tools are coming. For simplicity's sake, we're ignoring the advances in overlay or throughput per tool for this podcast.

I think there are a couple factors here. ASML has not decided to just go YOLO, let's expand capacity as fast as possible. In general, the semiconductor supply chain has not. It's lived through the booms and busts, and we can talk a bit more about it. Basically some players have recently woken up, but in general no one really sees demand for 200 gigawatts a year of AI chips, or trillions of dollars of spend a year in the semiconductor supply chain. They're not AI-pilled. They're not AGI-pilled.

Yeah, I feel you, but I'm saying no one really understands this in the supply chain. Constantly, we're told our numbers are way too high, and then when they're right, they're like, "Oh, yeah, but your next year's numbers are still too high." ASML's tool has four major components. It has the source, which is made by Cymer in San Diego. It has the reticle stage, which is made in Wilmington, Connecticut. It has the wafer stage. It has the optics, the lenses and such. Those last two are made in Europe. When you look at each of these four, they're tremendously complex supply chains that, (A) they have not tried to expand massively, and (B) when they try to expand them, the time lag is quite long. Again, this is the most complicated machine that humans make, period, at any sort of volume. Let's talk about the source specifically. What does the source do? It drops these tin droplets. It hits it three subsequent times with a laser perfectly. The first one hits this tin droplet, it expands out. It hits it again, so it expands out to this perfect shape, and then it blasts it at super high power. The tin droplets get excited enough that they release EUV light, 13.5 nanometer, and then it's in this thing that is collecting all the light and directing it into the lens stack. Then you have the lens stack, which is Carl Zeiss, as you mentioned, and some other folks, but Zeiss being the most important part of it. They also have not tried to expand production capacity because they don't see... They're like, "We're growing a lot because of AI. We're growing from 60 to 100." It's like, "No, no, no. We need to go to a couple hundred, but it's fine. Whatever." Each of these tools has, I think, 18 of these lenses, effectively. They are multilayer mirrors, which are perfect layers of molybdenum and ruthenium, if I recall correctly, stacked on top of each other in many layers, and then the light bounces off of it perfectly. When we think about a lens, it's in a shape, and it focuses the light. This is like a mirror that's also a lens, so it's pretty complicated. Any defect in these super thinly deposited stacks will mess it up. Any curvature issues will mess it up. There are a lot of challenges with scaling the production. It's quite artisanal in this sense because you're not making tens of thousands of these a year, you're making hundreds, you're making thousands. 60 tools a year, 18 of these per tool, you're still in the hundreds, of tools, or you're at the thousand number roughly for these lenses and projection optics. Then you step forward to the reticle stage, which is also something really crazy. This thing moves at, I want to say, nine Gs. It will shift nine Gs because as you step across a wafer, the tool will go... The wafer stage is complementary. It's the wafer part. You line these two things up. You're taking all the light through the

Lenses that's focused, and here's the reticle, here's the wafer. The reticle's moving one direction, the wafer's moving the other direction as it scans a 26 by 33 millimeter section of the wafer, and then it stops. It shifts over to another part of the wafer and does it again. It does that in just seconds. Each of them is moving at nine Gs in opposite directions. Each of these things is a wonder and marvel of chemistry, fabrication, mechanical engineering, and optical engineering, because you have to align all these things and make sure they're perfect. All of these things have crazy amounts of metrology because you have to perfectly test everything. If anything is messed up, the yield goes to zero, because this is such a finely tuned system. By the way, it's so large that you're building it in the factory in Eindhoven, Netherlands, and they're deconstructing it and shipping it on many planes to the customer site, and then you're reassembling it there and testing it again. That process takes many, many months. There are so many steps in the supply chain, whether it's Zeiss making their lenses and projection optics or Cymer, which is an ASML-owned company, making the EUV source. Each of these has its own complex supply chain. ASML has commented that their supply chain has over ten thousand people in it.

Yes. It might not be directly. It might be through Zeiss having so many suppliers and XYZ company having so many suppliers. If you just think about it, you're talking about two physically moving objects that are the size of a wafer, and it has to be accurate to the level of single-digit nanometers or even smaller because the entire system, the overlay, the layer-to-layer overlay variation, has to be on the order of 3 nanometers. If the overlay is 3 nms, that means each individual part, the accuracy of its physical movement has to be even less than that. It has to be sub-one nanometer in most cases, because the error of these things stack up. There's no way to just snap your fingers and increase production. Things as simple as power. The US going from zero percent power growth to two percent power growth, even though China's already at thirty, was so hard for America to do. And that's a really simple supply chain with very few people in it who make difficult things. There are probably 100,000 electricians and people who work in the electricity supply chain, or more, in the US. When you look at ASML, they employ so few people. Carl Zeiss probably employs less than a thousand people working on this, and all of those people are super, super specialized. You can't just train random people up for this in the snap of a finger. You can't just get your entire supply chain to get galvanized. Nvidia's had to do a lot to get the entire supply chain to even deliver the capacity they're going to make this year. When you go talk to Anthropic, they're like, 'We're short of TPUs, we're short of training, and we're short of GPUs.' When you go talk to OpenAI, they're like, 'We're short of these things.' OpenAI and Anthropic know they need X. Nvidia is not quite as AGI-pilled. They're building X minus 1. You go down the supply chain, everyone's doing X minus 1. In some cases, they're doing X divided by 2, because they're not AGI-pilled. You end up with this time lag for the whip to react. The AI-pilledness and the desire to increase production takes so long. Once they finally understand that they need to increase production rapidly… They think they understand. They think AI means we have to go from 60 to 100, in addition to the tools getting better and faster, the source getting higher power from 500 watts to 1,000, and all these other aspects of the supply chain advancing technically and increasing production. They think they're actually increasing production a lot. But if you flow through the numbers… What does Elon want? He wants 100 gigawatts a year in space by 2028 or 2029. Sam Altman wants 52 gigawatts a year by the end of the decade. Anthropic probably needs the same, and Google needs that. You go across the supply chain, and it's like, wait, no, the supply chain can't possibly build enough capacity for everyone to get what they want on the side of compute.

Yeah, 312.

We potentially do go crazy enough that this happens because we just need incremental compute, and the compute is worth the higher cost and power of these chips. But it's also unlikely to a large extent because some of these are not fair comparisons. For example, from A100, which is 312 teraflops, to Blackwell, which is 1,000 or 2,000 FP16, and then Rubin is 5,000 or so FP16… It's not a fair comparison because these chips have vastly different design targets. With A100, Nvidia optimized for FP16 and BF16 numerics. When you look at Hopper, they didn't care as much about that; they cared about FP8. When you look at Rubin, they don't care about FP16 and BF16 so much, they care mostly about FP4 and FP6. Numerics are what they've designed their chip for. Let's say we make a new chip design on 7nm, optimized for the numerics of the modern day. The performance difference is still going to be much larger than the FLOPS difference you mentioned. Often it's easy to boil things down to FLOPS per watt or FLOPS per dollar, but that's not a fair comparison. Let's look at Kimi K2.5 and DeepSeek. When you look at those two models and their performance on Hopper versus Blackwell on very optimized software, you get vastly different performance. Most of this is not attributed to FLOPS or numerics, because those models are actually eight-bit. So it's not like Blackwells and Hopper are both optimized for eight-bit, and Blackwell is not really taking advantage of its four-bit there. The performance gulf is actually much larger. Sure it's one thing to shrink process technology and make the transistor smaller so each chip has X number of FLOPS, but you forget the big gating factor. These models don't run on a single chip. They run on hundreds of chips at a time. If you look at DeepSeek's production deployment, which is well over a year old now, they were running on 160 GPUs. That's what they serve production traffic on. They split the model across 160 GPUs. Every time you cross the barrier from one chip to another, there is an efficiency loss. You have to transmit over high-speed electrical SerDes, which brings a latency cost and a power cost. There are all these dynamics that hurt. As you shrink and shrink the process node, you've increased the amount of compute in a single chip. Now in-chip movement of data is at least tens of terabytes a second, if not hundreds of terabytes a second. Whereas between chips, you're on the order of a terabyte a second. Then you have this movement of data between chips that are super close to each other physically. You can only put so many chips close to each other physically, so you have to put chips in different racks. The movement of data between racks is on the order of hundreds of gigabits a second, 400 gig or 800 gig a second, so roughly 100 gigabytes a second. So you have this huge ladder: on-chip communication is super fast, within the rack is an order of magnitude slower, and outside the rack is an order of magnitude lower than that. As you break the bounds of chips, you end up with a performance loss. The reason I explain this is because when you look at Hopper versus Blackwell, even if both are using a rack's worth of chips, Hopper is significantly slower. The amount of performance you have leveraged to the task within each domain—tens of terabytes a second of communication between these processing elements versus terabytes a second between these processing elements—is much, much higher and therefore the performance is much higher. When you look at inference at 100 tokens a second for DeepSeek and Kimi K2.5, the performance difference between Hopper and Blackwell is on the order of 20x. It's not 2x or 3x like the FLOPS performance difference indicates, even though those are on the same process node. There are just differences in networking technologies and what they've worked on. You can translate some of these back, but when you look at what they're doing on 3nm with Rubin, some of those things are simply not possible to do all the way back on A100, even if you make a new chip for 7nm. There are certain architectural improvements you can port and certain ones you cannot. The performance difference is not just going to be the difference in FLOPS. It's in some senses cumulative between the difference in FLOPS per chip, networking speed between chips, how many FLOPS are on a chip versus a system, and memory bandwidth on a single chip versus an entire system. All of these things compound.

Even within Blackwell, there are differences in performance when you're communicating on the chip versus across the chips. Those bounds are obviously much smaller than when you're going out of the entire chip. When you scale the number of chips up, there is some performance loss. It's not perfect, but it is way better than different entire packages. How large can advanced packaging scale? The way Nvidia is doing it is CoWoS. Google, Broadcom, MediaTek, and Amazon's Trainium are all doing CoWoS. But actually you can go look back at what Tesla did with Dojo, which they cancelled and restarted. Dojo was a chip that was the size of an entire wafer. They had 25 chips on it. There were some tradeoffs. They couldn't put HBM on it. But the positive side was that they had 25 chips on it. To date, it is still probably the best chip for running convolutional neural networks. It's just not great at transformers because the shape of the chip, the memory, the arithmetic, and all these various specifications are just not well-suited for transformers. They're well-suited for CNNs. Dojo chips were optimized around that, and they made a bigger package. But as you make packages bigger and bigger, you have other constraints: networking speed, memory bandwidth, and cooling capabilities. All of these things start to rear their heads. It's not simple. But yes, you will see a trend line of more chips on the package, and yes, you're going to be able to do that on 7nm. In fact, that's what Huawei did with their Ascend 910C or D. They initially put one, and then they did two. They're focusing on scaling the packaging up because that is an area where they can advance faster than process technology where they can't shrink. But at the end of the day, that's something you can do on the leading-edge chips too. Anything you do on 7nm, you can also probably do on 3nm in terms of packaging.

To date, China still does not have an entirely indigenized semiconductor supply chain. But would they in 2030? By 2030, it's possible that they do. But to date, all of China's 7nm and 14nm capacity uses ASML DUV tools. The amount that they can import from ASML is large. But the vast majority of ASML's revenue, especially on EUV all of it, is outside of China. The scale advantage is still in the favor of the West plus Taiwan, Japan, and Korea, et cetera.

The question is how fast can they advance and scale up production as well as quality. To date, we haven't seen that. Now I'm quite bullish that they're going to be able to do these things over the next five to ten years. They will really scale up production and kick it into high gear. They have more engineers working on it and more desire to throw capital at the problem. So by 2030, will they have fully indigenized DUV? I think for sure. DUV, yes.

I think they'll have working tools. I don't think that they'll be able to manufacture a bunch yet. There's having it work, and then there's production hell. ASML had EUV working in the early 2010s at some capacity. The tools were not accurate enough. They were not scaled for high-volume manufacturing or reliable enough. They had to ramp production, and that all took time. Production hell takes time. That's why it took another five to seven years to get EUV into mass production at a fab rather than just working in the lab.

ASML?

That's a great question. It's a bit of a challenge to look into this supply chain especially. We try really hard. In some instances, they're buying stuff from Japanese vendors. If they want a fully indigenized supply chain, they need to not buy these lenses, projection optics, or stages from Japanese vendors. They need to build it internally. It's really tough to say where they'll be able to get to. I honestly think it's a shot in the dark. But it's probably not unlikely that they'll be able to do on the order of 100 DUV tools a year, whereas ASML is currently doing hundreds of DUV tools a year. No company has a process node where they make a million wafers a month. Elon says he wants to do it and China is obviously going to do it. TSMC is trying to do that. The memory makers may get to a million wafers a month as well, but not in a single fab. It's mind-boggling to think of that scale, and challenging to see the supply chain galvanized for that. I don't want to doubt China's capability to scale. I guess this is an interesting question. I think at some point SemiAnalysis will do the deep dive on this.

SemiAnalysis has got to write the definitive model on this. It's really challenging when you move timescales out that far. What we tend to focus on is tracking every data center, every fab, and all the tools. We track where they're going, but the time lags for these things are relatively short. We can only make reasonably accurate estimates for data center capacity based on land purchasing, permits, and turbine purchasing. We know where all these things are going, that's the data we sell. As you go out to 2035, things are just so radically different. Your error bars get so large it's hard to make an estimate. But at the end of the day, if takeoff or timelines are slow enough, I don't see why China wouldn't be able to catch up drastically. In some sense, we've got this valley where, three to six months ago, or maybe even now, Chinese models are as competitive as they've ever been. I think Opus 4.6 and GPT 5.4 have really pulled away and made the gap a little bit bigger, but I'm sure some new Chinese models will come out. As we move from selling tokens where they provide the entire reasoning chain, to selling automated white-collar work—an automated software engineer, you send them the request, they give you the result back, and there's a bunch of thinking on the back end that they don't show you—the ability to distill out of American models into Chinese models will be harder. Second, look at the scale of the compute the labs have. OpenAI exited the year with roughly two gigawatts last year. Anthropic will get to two-plus gigawatts this year. By the end of next year, they'll both be at ten gigawatts of capacity. China is not scaling their AI lab compute nearly as fast. At some point, when you can't distill the learnings from these labs into the Chinese models, plus with this compute race that OpenAI, Anthropic, Google, and Meta are all racing on, they end up getting to a point where the model performance should start to diverge more. Then look at all this CapEx being spent on data centers. Amazon is spending $200 billion, Google $180 billion. All these companies are spending hundreds of billions of dollars on CapEx. There's nearly a trillion dollars of CapEx being invested in data centers in America this year, roughly. What's the return on invested capital here? You and I would think the return on invested capital for data center CapEx is very high. If we look at Anthropic's revenues, in January they added $4 billion. In February, which was a shorter month, they added $6 billion. We'll see what they can do in March and April, given that compute constraints are what's bottlenecking their growth. The reliability of Claude is quite low because they're so compute constrained. But if this continues, then the ROIC on these data centers is super high. At some point, the US economy starts growing faster and faster over this year and next year because of all this CapEx, all the revenue these models are generating, and the downstream supply chain. China doesn't have that yet. They have not built the scale of infrastructure to invest in models, get to the capabilities, and then deploy these models at such scale. When you look at Anthropic, they're at $20 billion ARR. The margins are sub-50 percent, at least as last reported by The Information. So that's $13 or $14 billion of compute that it's running on rental cost-wise, which is actually $50 billion worth of CapEx that someone laid out for Anthropic to generate their current revenue. China has just not done this. If and when Anthropic 10Xs revenue again—and I think our answer would be when, not if—China doesn't have the compute to deploy at that scale. So there is some sense that we're in a fast takeoff. It's not like we're talking about a Dyson sphere by X date, it's more like the revenue is compounding at such a rate that it does affect economic growth. The resources these labs are gathering are growing so fast. China hasn't done that yet, so in that case, the US and the West are actually diverging. The flip side is that these infrastructure investments have middling returns. Maybe they're not as good as hoped. Maybe Google is wrong for wanting to take free cash flow to zero and spend $300 billion on CapEx next year. Maybe they're just wrong and people on Wall Street who are bearish and people who don't understand AI are correct. In that case, the US is building all this capacity but doesn't get great returns. Meanwhile, China is able to build a fully vertical, indigenized supply chain, instead of the US/Japan/Korea/Taiwan/SE Asia/Europe countries together building this less vertical supply chain. In a sense, at some point China is able to scale past us if AI takes longer to get to certain capability levels than the vast majority of your guests on this podcast believe. It's fast timelines, the US wins; long timelines, China wins.

At the end of the day, the incremental purchaser who's willing to pay the highest price for tokens also ends up being the one that's less price-sensitive. Compute should be allocated, in a capitalistic society, towards the goods that have the highest value, and the private market determines this by willingness to pay. To some extent, Anthropic could actually release a slow mode. They could release Claude Slow Mode and increase tokens per dollar by a significant amount. They could probably reduce the price of Opus 4.6 by 4-5x and reduce the speed by maybe just 2x. The curve on inference throughput versus speed is already there just on HBM. And yet they don't, because no one actually wants to use a slow model. Furthermore, on these agentic tasks, it's great that the model can run at a time horizon of hours. But if the model was running slower, those hours would become a day. Vice versa, if the model is running faster, those hours become an hour. No one really wants to move to a day-long wait period, because the highest-value tasks also have some time sensitivity to them. I struggle to see… Yes, you could use regular DRAM. There are a couple of challenges with this. One of the core constraints of chips is that a chip is a certain size, and all of the I/O escapes on the edges. Often, the left and right of the chip are HBM—so the I/O from the chip to the HBM is on the sides—and then the top and bottom are I/O to other chips. If you were to change from HBM to DDR, all of a sudden this I/O on the edge would have significantly less bandwidth, but significantly more capacity per chip. But the metric you actually care about is bandwidth per wafer, not bits per wafer. Because the thing that is constraining the FLOPS is just getting in and out the next matrix, and for that you just need more bandwidth.

In many cases, these GPUs are not running at full memory capacity. It's obviously a system design thing: model, hardware, and software co-design. You have to figure out how much KV cache you need, how much you keep on the chip, how much you offload to other chips and call when you need it for tool calling, and how many chips you parallelize this on. Obviously, the search space for this is very broad, which is why we have InferenceX, an open-source model that searches all the optimal points on inference for a variety of different chips and models. The point is, you're not always necessarily constrained by memory capacity. You can be constrained by FLOPS, network bandwidth, memory bandwidth, or memory capacity. If you really simplify it down, there are four constraints, and each of these can break out into more. If you switch to DDR, yes, you produce four times the bits per DRAM wafer, but all of a sudden the constraints shift a lot and your system design shifts. You go slower. Is the market smaller? Maybe. But also, all these FLOPS are wasted because they're just sitting there waiting for memory. You don't need all that capacity because you can't really increase batch size because then the KV cache would take even longer to read.

An HBM4 stack—let's talk about the stuff that's in Rubin, because that's what we've been indexing on—is 2048 bits across, connected in an area that's 13 millimeters wide. It transfers memory at around 10 giga-transfers a second. So a stack of HBM4 is 2048 bits on an area that's roughly 11 to 13 millimeters wide. That's the shoreline you're taking on the chip. In that shoreline, you have 2048 bits transferring at 10 giga-transfers per second. You multiply those together and divide by eight, bits to a byte, and you're at roughly 2.5 terabytes a second per HBM stack. When you look at DDR, in that same area, it's maybe 64 or 128 bits wide. That DDR5 is transferring at anywhere from 6.4 to maybe 8,000 giga-transfers a second. So your bandwidth is significantly lower. It's 64 times 8,000 divided by eight, which puts you at 64 gigabytes a second. Even if you take a generous interpretation of 128 times 8 giga-transfers, you're at 128 gigabytes a second for the same shoreline, versus 2.5 terabytes a second. There's an order of magnitude difference in bandwidth per edge area. If your chip is a square, or 26 by 33 millimeters—which is the maximum size for an individual die—you only have so much edge area. On the inside of that chip, you put all your compute. There are things you can do to try and change that, like more SRAM or more caching. But at the end of the day, you're very constrained by bandwidth. Then there's the question of where you can destroy demand to free up enough for AI.

Yes.

Yes. Obviously, there's some level of margin stacking that Nvidia does, so you have to separate that out and apply their margin to the memory and the logic. But at the end of the day, a third of their CapEx is going to memory.

The memory crunch will continue to get harder, and prices will continue to go up. This affects different parts of the market differently. Are people going to hate AI more and more? Yes, because smartphones and PCs are not going to get incrementally better year on year. In fact, they're going to get incrementally worse. If you look at the bill of materials for an iPhone, what fraction of it is the memory? How much more expensive does an iPhone get if the memory is two times more expensive? I believe an iPhone has 12 gigabytes of memory. Each gig used to cost roughly $3-4, so that's $50. But now the price of memory has tripled. Let's say it's $12 per gig for DDR. Now you're talking about $150 versus $50. That's a $100 increase in cost for Apple. Apple has some margin, they're not just going to eat the margin. NAND also has the same market dynamics, so in reality, it's probably a $150 increase on the iPhone. So now that's a $100 cost increase and that's just on the DRAM. The NAND also has the same sort of market. So in fact it's probably a $150 increase on the iPhone. Apple either has to pass that on to the consumer or eat it. I don't see Apple reducing their margin too much, maybe they eat a little bit. But at the end of the day, that means the end consumer is paying $250 more for an iPhone. Now that's just on last year's pricing versus today's. There is some lag before Apple feels the heat because they tend to have long-term contracts for memory that last three months to a year. But at the end of the day, Apple gets hit pretty hard by this. They won't really adjust until the next iPhone release. But that's the high end of the market, which is only a few hundred million phones a year. Apple sells two or three hundred million phones annually. The bulk of the market is mid-range and low-end. It used to be that 1.4 billion smartphones were sold a year. Now we're at about 1.1 billion. Our projections are that we might drop to 800 million this year, and down to 500 or 600 million next year. We look at data points out of China from some of our analysts in Asia, Singapore, Hong Kong, and Taiwan. They've been tracking this, and they see Xiaomi and Oppo cutting low-end and mid-range smartphone volumes by half. Yes, it's only a $150 BOM increase on a $1,000 iPhone where Apple has some larger margin. But for smaller phones, the percentage of the BOM that goes to memory and storage is much larger. And the margins are lower, so there's less capacity to even eat the margins. And they have also generally tended not to do long-term agreements on memory. Why this is a big deal is that if smartphone volumes halve, that drop will happen in the low and mid-range, not the high end. So it's not like the bits released are halving. Currently, consumer devices account for more than half of memory demand. Even if you halve smartphone volumes, because of the shape of the halving, the low end gets cut by more than half, while the high end gets cut by less than half, because you and I will still buy the high-end phones that cost north of a thousand dollars. We'll buy them even if they get a little bit more expensive. And Apple's volumes will not go down as much as a low-end smartphone provider. The same applies to PCs. What this does to the market is quite drastic. DRAM gets released and goes to AI chips, who are willing to do longer-term contracts and pay higher margins, because at the end of the day the margin they extract from the end user is much larger. This probably leads to people hating AI even more. Today, you already see all the memes on PC subreddits and gaming PC Twitter. It's cat dancing videos saying, 'This is why memory prices have doubled and you can't get a new gaming GPU or desktop.' It's going to be even worse when memory prices double again, especially DRAM. Another interesting dynamic is that it's not just DRAM, it's also NAND. NAND is also going up in price. Both of these markets have expanded capacity very slowly over the last few years, NAND almost zero. The percentage of NAND that goes to phones and PCs is larger than the percentage of DRAM that goes to phones and PCs. As you destroy demand, mostly for DRAM purposes, you unlock more NAND that gets allocated and can go to other markets. The price increases of DRAM will be larger than those of NAND because you've released more from the consumer, and in fact, you've produced more memory for AI.

They are, but not in as large quantities as DRAM.

The constraints, as I was mentioning earlier, are not necessarily EUV tools today or next year. They become that as we get to the latter part of the decade. Currently, the constraints are more that they physically just haven't built fabs. Over the last three to four years, these vendors have not built new fabs because memory prices were really low. Their margins were low, and in fact, they were losing money in 2023 on memory. So they decided they weren't building new fabs. The market slowly recovered over time but never really got amazing until last year. In 2024, we were banging on the drums that reasoning means long context, which means a large KV cache, which means you need a lot of memory demand. We've been talking about that for a year and a half, two years. People who understand AI went really long on memory then. So you've seen that dynamic, but now it has finally played out in pricing. It took so long for what was obvious: long context means the KV cache gets bigger, you need more memory. Half the cost of accelerators is memory. Of course they're going to start going crazy on it. It took a year for that to actually reflect in memory prices. Once memory prices reflected that, it took another three to six months for the memory vendors to start building fabs. Those fabs take two years to build. So we won't have really meaningful fabs to even put these tools in until late 2027 or 2028. Instead, you've seen some really crazy stuff to get capacity. Micron bought a fab from a company in Taiwan that makes lagging-edge chips. Hynix and Samsung are doing some pretty crazy things to try and expand capacity at their existing fabs, which also have large knock-on effects in the economy.

There's nowhere to put the tools. It's not just EUV; there are other tools involved in DRAM and logic. In logic, for N3, about 28% of the cost of the final wafer is EUV. When you look at DRAM, it's in the teens. It's going up, but it's a much smaller percentage of the cost. These other tools are also bottlenecks, although their supply chains are not as complex as ASML's. You see Applied Materials, Lam Research, and all these other companies expanding capacity a lot as well. But you don't have anywhere to put the tool, because the most complex buildings people make are fabs, and fabs take two years to build.