Episode 20: The Future of AI Supercomputing Clusters with Dylan Patel

Yeah, sure. So we have a research and consulting firm based in the US but all over the world—US, Taiwan, Japan, Singapore, France, etc. And we do research and consulting around the semiconductor supply chain, the data center supply chain, and broadly those areas. We work with hyperscalers, semiconductor companies, etc. And that's the main focus, but there's a newsletter which everyone reads, which is fun.

Yes, I mean the CHIPS Act is very well-meaning, right? It's required, obviously. $52 billion is just a drop in the bucket, right? It is nowhere close enough, in my view, to get the US to even 20% of global production, let alone a higher number, which is what we'd like—going back to maybe the 80s levels, right? 30s, even percentages or 40 percentage points. And so this is a real big challenge. As far as what happens with the CHIPS Act, that's up for debate slash we're not sure. Sounds like it's being rebranded or even killed. But what's clear is the US needs a secure semiconductor supply chain. Even with TSMC's announcement, even if they do it right, and it'll take them to like 2030, 31 to spend that much money and build the fabs, etc., most of it is after this administration's gone. Because all of the fab shell is built—less than 20% of the cost—takes a couple years, and then the equipment gets moved in. So you're going to see all this equipment move in in the next administration mostly, at which point, do they just tear up the deal? And so even if they do fully the full hundred billion dollars they commit to, completely still relying on R&D in Taiwan. The R&D fab they announced is not going to secure R&D supply chain, right? It's not going to secure new node development. It's still less than 30% of their capacity. It doesn't have many nodes. It's still not making America's chip supply chain secure. But it's a good step in the right direction. So we'll see what happens with that.

So I think we should really focus on where America can win. Taiwan, for example—TSMC is already approaching 20% of power consumption in Taiwan, and if they were to continue to build out what we expect them to build out in the next four or five years, it's going to get to 30, 40%. Meanwhile, they're turning off all their nuclear power over the next 18 months. They have a humongous percentage of GE Vernova's dual combined cycle gas plants being shipped there, and their power—they have to import LNG from the US primarily. This is great, but their power costs are three, four times that of the US. And fabs take a lot of power. Even Intel's Ohio fab, which is relatively small on the grand scheme of TSMC's fabs in Hsinchu and Taichung and now Arizona—quite small actually, Ohio is—that's gigawatt scale. These things are not small. And the US can actually win by providing super cheap power. Most of the cost of the fab is the equipment—80%—that costs the same no matter where you are. Flying an EUV tool from Veldhoven to Taiwan versus the US. So I think there's a lot of things that can be done to equalize on cost and make us win. And actually, yes, we do pay our people more, but our power is so much cheaper. Let's exploit that advantage, and we can actually make America economic to build all the fabs in—or not all of them, but a higher percentage. And I think that's the real hope, is that we can sort of accelerate and drive to win. Now, obviously, there's a lot of studies out there that the US is slower to build fabs, and we can do stuff to accelerate that.

Yeah, absolutely.

So the diffusion rule is quite a challenge. You put Portugal in the same bucket as Singapore, who are generally a military ally, in the same bucket as Yemen. Countries are very mad. It is sort of on a silver platter handed to the Trump administration to do some deals. Hey, if you go follow through with this, you tweak it. Like India, you want to be a tier one country, you actually want to build out these data centers that you've started planning. Maybe stop buying Russian military equipment and buy some F-35s. I feel like deals like this could be done. But at the end of the day, GPUs and AI are the capital good that creates intelligence and human factor labor. There's a lot of GDP growth that can come from intelligence, and I think the substrate for that is compute. Limiting it is obviously very—it's like spitting on your allies if you limit it to them, or spitting on countries that are in the middle ground between the US and China. And so this has a lot of geopolitical ramifications. I don't know that much about geopolitics, to be clear. What I do know is Washington, so you should know everything now. The thing that I think is pretty clear though is that there's this false narrative that if we do limit with the diffusion rules, that China will now take over this compute market. While the equipment restrictions on China are very poor, like Swiss cheese, there's still a ton that flies through. The restrictions—China does not have the capacity to—their natural amount of compute they should have is like a third. Before, in 2022, they imported roughly 40% of the A100s. This is pre-ChatGPT. Now we fast forward to today, over 70% of GPUs are going to America. So what's the natural state? Well, they should have 40% of FLOPS maybe, and America should have like 40% in a sort of status quo. And diffusion rules are aimed at making sure this stays 70% America and much less for China. The question is, how can we effectively do this without blowing up allies? And China taking over these foreign markets is erroneous because they can't produce enough for their domestic market, let alone to export, and they won't be able to build enough for this year or next year. Now, long term, if AI doesn't take off—like many people in the tech industry, like myself, I'm not wearing a suit so clearly not in DC world—there's the question of, can China take—is diffusion going to slow down America in the long term? And if AI doesn't take off, then obviously AI diffusion is like a policy map because people are going to be closer to China in three to five years if AI doesn't build this exponential growth that everyone is pushing for.

So it's an absolute fact that China's been slowed down massively. SMIC, CXMT, these companies would be much further ahead and have much more capacity if it weren't for the restrictions—whether it be HBM, whether it be leading-edge silicon, whether it be shrinking to 5 nanometer and beyond. So they've absolutely been effective. The challenge has been that there have been enough loopholes, enough Swiss cheese through it, all these holes that they've still been able to advance. And China has fantastic engineering capabilities and tons of money to pour into this, so they're going to advance. The question is just how big can the gap be. And restrictions need to get a lot better on equipment before they get better on chips, to some extent. Because you know, I mentioned natural state is maybe 40% China of AI compute, and they're much lower than that today. In effect, the restrictions have been quite effective on chips, even though there's all these problems with the H20 and smuggling and so on and so forth. On equipment though, last year, ASML—many quarters—they've had 45% of their equipment and even more of their revenue come from China. And likewise for ASML, Lam Research, and all these companies. I think a lot of this is because the equipment restrictions are so technical, and the only people to talk to on these restrictions generally tends to be the industry. And the industry is obviously benefiting from loopholes with 45% of their revenue coming from China. And so, you know, I have someone who worked at ASML on the reticle, I have someone who worked at Lam Research. We have a lot of technical capabilities, but it takes so much to explain to any one regulator. And we don't have a ton of conversations with policymakers, but when we do, you have to end up having conversations with ten different people, and then there's like a smorgasbord of a hundred people that are actually involved, and it gets diluted. And then there ends up being some loophole or some technicality where you can still ship equipment. And this is the problem. If you don't limit equipment properly, then yes, they are just going to take over—they're going to be able to accelerate their chip production industry, design chips, AI chips, etc. But if you limit equipment properly, so that they can only make this level of chip, and then you allow companies from the US to sell maybe this level of chip, then you end up retaining the China market, slowing down their internal efforts because they can't catch up to what the West can sell them, and they can't export to the foreign world what the West will sell them. So I think that's the correct strategy. And right now we've kind of got it flipped, which is that China can make chips better with the equipment we sell them than the chips we sell them. And now, of course, there's still some catching up. Huawei is still—the 910B is still behind the H20. The 910C will look to be more closely matched coming out this year in volume. But there is that gap, and this is not the correct policy to have.