Dylan Patel: NVIDIA's New Moat & Why China is "Semiconductor Pilled”

This is the biggest change in human history maybe ever. What's about to happen with AI? This is the biggest revolution, bigger than the industrial revolution. Jensen is very paranoid about losing. If he just kept making his mainline chip, people crush him on cost and performance. Acquiring Grock is how you get those resources to make more solutions for different parts of the market to stay king. At the end of the day, this is an economic war. If the US and the West win in AI, China will not rise to be the global hegemony. But without AI, China definitely will rise. They're just going to outrun America.

Hello. How are you?

It's very clear. We're not sure where AI models are headed in terms of, you know, over the next few years, what happens to the architecture, but you know, the thing that I think everyone is sort of like agreed on is models are pretty autoregressive, right? Next token generation is like the thing but beyond that right attention mechanisms changed how it works, everything changes right, could change. And so what's interesting is the reason Nvidia won is because they just took like the widest surface area bet and then people kept developing models on that and that kind of shape worked. But now the workload is so large that there is room for specialization that will give you 10x increases in certain domains, right? In a general purpose workload Grock doesn't work, right? You know, it can't train, it can't inference really large models cost efficiently, right? You can't serve many many many users, but what it can do is it can go screamingly fast, right? Same with the Cerebras OpenAI deal, but that's like one workload, right? Very decode focused, right? Generating autoregressive tokens in a single stream super fast. Another direction AI models could head, right? We don't know are models going to think in one token stream or is it actually they're constantly context switching, right? And they're going from they have this humongous humongous context and they're generating in multiple parallel streams, right? And so Google and OpenAI have both released mechanisms of this with their pro models where the model actually doesn't just have one single chain of thought for reasoning, it has multiple, right? And then I don't exactly like you know and how they choose which one and what the final answer to you delivers is is an area of research, but there is room for that kind of chip, right? Something that works on very parallel a lot lot of streams of chain of thought and maybe the latency requirements are not as crazy, right? Maybe you don't want to go blindingly fast, right? Maybe you're okay with it being, you know, because I can spin up 100 parallel, you know, streams of thought or agents or whatever you want to call them. Maybe I care a lot about cost there. And because it's 100 in parallel instead of one going super super fast, it's not as deep, right? The tree search or the depth of the inference is not as deep, but it is much wider. You know, there's other parts of inference. Hey, process creating the KV cache. So, Nvidia has a chip for that, right? That's the CPX. So they've made the CPX, they bought Grock for decode, and then they still have their general purpose GPU. So they're kind of trying to cover their bases because unlike the first wave of AI chip companies where they sort of just made chips and then tried to figure out where it would work, right? They had a thesis, Grock and Cerebras, both as well as SambaNova, right, which was put a lot of memory on the chip and not necessarily in the case of Cerebras and Grock, no memory off chip. And in the case of SambaNova, less memory offchip or slower memory offchip with higher capacity. You know, they sort of all made similar bets in that direction. And it didn't work for a while until it kind of did, right? Because there's a workload that now necessitates it. Nvidia recognizes they're the leader. They're at the tent pole. Hey, in one respect they can just run faster than everyone, but it's kind of hard to be 2x better than Google or OpenAI or whoever else's internal chip, right? To justify their, you know, 75% plus margins, right? And then they have to be 2x to 4x better to justify their margins because that's what they're charging above COGS. You know, the question is what architecture will deliver that? Well, yes, keep the programmability of their GPUs is great for training and for a lot of workloads, but you know, guess what? I think a lot of people will just be downloading an open source model, downloading an inference framework and pressing go, right? A little bit more complicated than that, but that's going to be the consumption method for a lot of enterprises, a lot of startups, a lot of tech companies is they're just going to do that or they're going to rent the GPUs or rent the chips and then download an open source framework and model and go, right? And Nvidia recognizes this and hey, there is room for products that aren't general purpose, right? The general purpose GPU will still probably be the main line for training and for a lot of inference and for cost-efficient inference, but maybe blindingly fast or workloads that have a ton of prefill, i.e. creating the KV cache. Maybe those workloads could be different chips, right? And the CPX chip they announced, right? They say it's for the context processing, creating the KV cache. It's also really useful for video models because video models don't care about memory bandwidth and so you know why pay for the expensive memory that the general purpose chip has or why do what Grock is doing which is tying hundreds or thousands of chips together and not having memory but keeping the entire model on chip. The trade-off for that of course is you need thousands of chips and you have less compute per chip and so like Nvidia's trying to capture the whole surface area because again you don't know where models are headed and it's hard to say where the research is headed.

I certainly think it's not good from an anti-competitive sense, right? I don't think people should just be able to buy companies without like any antitrust like process at all. Now, in the case of like a large company buying a startup, I'm completely fine with it. The flip side is like, hey, we know the deal is happening, right? This happened for a company I was an adviser for Nvidia acquired Infabrica just maybe a few months before they did Grock and similar style of deal, right? If someone wanted to strike it down that's the biggest limbo, right? We've seen this happen in venture and you probably know more stories of this but like a company trying to get acquired they get stuck in limbo for like a year.

Yeah, it falls apart, the deal did because some regulatory BS and now the company was and the founders were focused on getting the deal done instead of like making the product better for a year and now they're like behind or you know they weren't focused on growth as much, right? You know, you only have so much time as a founder so in that sense I like the license deals, right?

I think the thing about Nvidia is they take the Andy Grove mentality like more serious than anyone else, right? Like okay fine Google implemented OKRs because Intel did it but that's like management stuff, right? Only the paranoid survive, right? This is like core to the Bay Area, core to Nvidia. Jensen is very paranoid about losing, right? These specializations, if he just kept making his mainline chip, would mean people could, you know, point solutions for specific parts of the market would crush him on cost and performance. Then he can't justify his margin. That's a threat to Nvidia's business model as a whole, especially if the best model only changes every 3 months or the model you want to roll out. Okay, well then you're going to have three months to figure out how to make a model work on one chip architecture for that point solution and you know it's fine. Software advantage of Nvidia is not that important. Then Jensen's super paranoid about losing and frankly it's really hard to hire enough talented chip people. When you look across the market, there is only a few companies who have successfully created a chip architecture software to run the models accurately, right? Like because you can look at random APIs of say an Alibaba Qwen model and different people are doing all sorts of tricks like quantizing it, but also many other tricks which then end up like making the model quality lower. You know, building a rack scale solution, networking thousands of chips together and then deploying an API and Grock did the whole thing with frankly not that many people. So now it's like okay well I'm Nvidia I want to make four different chip architectures and actually four different point solutions maybe the general purpose and then one here one here one here and in addition my general purpose thing is actually not just like a GPU chip it's like GPU chips CPU chips networking chips NVSwitch NICs like you know there's many many chips and each of those chips has many chiplets you don't have enough engineering resources, right? And so like acquiring Grock is like how you get those resources to make more solutions for different parts of the market. As far as like are they threatened like I think obviously there's some cool startups out there, right, that are raising a lot, right, currently or have raised such as Etched, MatX, Positron, these new age of AI companies. There's also the prior age of like Cerebras is out there still, right? You know, Tenstorrent, etc. And there's a lot of AI chip companies on the startup side, but then there's also, you know, Google's TPU, AMD GPUs, Amazon Trainium, who are all really credible competitors. And then, you know, Meta's MTIA is somewhat credible. And then you know Microsoft Maia is not credible but like you know maybe it will be one day, right? So you sort of have like a lot of competition they've got to hold the gates back and so I think.

Yeah.

I think they do. I think networking is super important. I think the CUDA software moat is very important but it's also like changing rapidly, right? It's an incredible amount of the software that Nvidia GPUs run on is not from Nvidia. It's the developer ecosystem that's open sourcing it. When you look at, for example, vLLM and SGLang, right? These support AMD GPUs almost as first class citizens now. And vLLM is getting significant support for TPUs, for Trainium, and there will be other chips coming out from startups that also support vLLM, SGLang. Now like how difficult is it? You know the reason why CUDA is so important is like okay I can do whatever I need to do right programming a GPU. I think most AI chips will not be consumed by people programming anything for it. They will download an open source inference engine and they will download an open source model and then they will put it on the and it's really simple to download vLLM and like make it work like it's not that hard to set up a server and Nvidia's putting out a lot of open source software like Triton inference server and Dynamo and all these things to make it easy because that is the consumption model ultimately for the majority of AI right is and it might be like oh it's my own inference engine but most servers will not run code besides the inference engine and the model. It's not like people are actually like researchers are like writing code for GPUs to see ideas if they'll work and train models and all these things or just mess around with them to figure out you know infra performance or whatever it is but most of it won't be there and so CUDA as a moat, CUDA language is like you know like it's like fine right like you know no one actually writes CUDA right like most people write PyTorch and then like torch compile and then they just run it on the GPU they don't write CUDA but a lot of this CUDA moat is like how does PyTorch translate into high performance GPUs and that surface area from when people were just writing like hardcore when people are hardcore writing CUDA kernels to like hey they're writing PyTorch and then it's compiling down to GPUs versus oh I'm just downloading vLLM is a curve of like not a ton of people that can do CUDA kernels a whole lot more people can do PyTorch right random you know PhDs and random people it's very simple right a crapload of people can do vLLM download it run it on a server well if it now supports other chips what is the CUDA moat? Nvidia has recognized this and they've been building software that is not necessarily the CUDA moat and I can give some examples. All right. So the name of the game is fast tokens and lowest cost tokens, right? And lowest cost tokens happens by your chip being fast. But there's also tricks, right? One example, right? Like I mentioned with, you know, the CPX versus Grock, right? Is processing your prefill context, right? Super cheap CPX, right? If I care a lot about speed, then Grock. These are optimizations on the hardware side. There's optimizations on the software side as well, right? And so one example is when I'm doing for example if I look at a Claude Code or a Cursor type application right the workload is like it takes your repo takes the relevant parts of your repo puts it in the context of the LLM it prompts it generates right and if it's an agent mode it circulates the context a couple times it'll collapse put things off to the side access different contexts but what's you know especially when you think about an agent for software and you can see this in Codex you know Codex actually not as good as Claude Code, but it can do work on time horizons of like 9 10 hours. And do like a big refactor better than Claude Code can, even though most of the times Claude Code is better. And what's interesting about Codex does is it'll like take your repo, it'll identify parts if you're asking it to refactor it, identify parts, write stuff, you know, make like these notes for itself everywhere, collapse the context, switch from this part of the repo to that part of the repo to this part of the repo. But when you think about it, it's like, oh, if this thing is just generating tokens all the time, plus it's switching what my context is constantly, that's really expensive, right? If you think about like what's the cost of inference, I want to say it's like it's $10 per million tokens of output and or $3 for decode or 10 for decode and three for prefill. And so if you think about, oh, it just worked for nine hours on one task, one refactor, huge value. But if it changed context a ton of times and your context is like 30k usually or 50k or you know heading to hundreds of thousands you know how big your repository is and how much context switch now you're spending all this money on prefill right not the decode tokens but actually why am I regenerating the KV cache I can actually just like store the KV cache elsewhere and then when I need it again I can pull it and plop it into CPU memory or into GPU memory. And so Nvidia's got this like KV cache manager and they've been working really hard on like making it so they can interface SSDs and stick the KV cache on there and pull it out whenever they want. So for this kind of workload and then if you do this and you look at like coding as an application and you like look at these coding companies and how much they're paying for prefill versus decode actually majority of their cost is prefill tokens not decode tokens because their context is just so large and it's switching all the time even in agent modes. You know, if you can now not have to do the prefill, your costs go down dramatically. But that's a very complicated thing to do from a software perspective. You know, companies like Anthropic, Google, OpenAI have already done it. But what about the wide world, right? And so Nvidia is trying to make the open source software for this. And that's like CUDA moat, but it's like actually no, none of this is CUDA, right? Like it's like memory management and like you know storage management and when do you call what and how do you transfer it and how do you like spread the KV cache across a bunch of different storage nodes and what happens when you read it and the network congestion just like all these things yeah it's like Nvidia's wheelhouse but it's not CUDA and I think like the easy way to say it is it is the CUDA moat, right? And so things like this KV cache manager and many other things they're trying to do to reduce the cost of inference like is how they build the new CUDA moat because again today it's quite I mean AMD is like not fully there yet and TPU is being added right now and Trainium is being added soon as well to vLLM but all of them will have a very good UX for download model run model on vLLM by the middle of the year I think right certainly AMD is already there by the end of this quarter we have something that like tests this right it's called InferenceMax it's open source all the code is and the results are but we run across I think $60 million of GPUs which are donated to us by companies like Nvidia AMD OpenAI Microsoft Amazon on Crusoe, CoreWeave, Together AI, all these companies are sponsoring GPUs for us to run this. We're running vLLM and SGLang every night on, you know, nine different kinds of GPUs on a variety of different models and different context lengths and all these things, right? To see the performance and you can see the performance moving every day or pretty often because the software changes all the time. And so like the fact that this exists is the CUDA moat, right? It's not that like AMD you can do this on their chips, Nvidia can do this on their chips. It's oh when the new model comes out, how fast does it get to peak performance? because you know it's a moving target or hey can I implement this KV cache management thing how hard is it how many engineers do I need oh just one great or 10 great if I need a hundred people to develop it like Google and you know so on and so forth did then that's much harder.

I think AMD will be caught up at times and very behind at other times like currently they're super far behind right because Blackwell is just way better than MI355 and then you know Rubin comes out and they'll be way way behind but then AMD's new chip comes out and AMD will be caught up or even slightly ahead on a hardware perspective. Software's behind, right? And you have this like leapfrogging and AMD is a very credible second competitor. I don't think they'll go beyond like I think they'll stay in single digits market share. Single digit percentage market share.

Yeah. I mean, Nvidia's revenue this year is going to be like.

The three gajillion dollars.

You sort of the whole specialization game, right? You have to specialize because you're never going to beat Nvidia at their own game, right? They're going to have the supply chain unlock. They're going to get to the newest memory technology or process technology or whatever packaging technology, whatever it is, sooner than you and they're just going to crush you, right? If you play their game, you have to AMD is trying to play Nvidia's game, but AMD is like extremely good at engineering silicon, right? Everyone else has to try something weird or different, right? And so when you look at Etched or MatX or Positron or Cerebras or Tenstorrent, you go to look at all these companies, right? There are unique things about what they're doing and it's not clear if AI models will still be within that realm when that comes out, right? Does oh now people use like engrams and other sparse attention techniques. Is that like does that change like some of the specializations people are doing or hey people are now doing like you know models are now sparse instead of being dense models does that change things. There's so many optimizations and changes on the model side and you can't predict what's going to happen with the ML research easily at least you can't. The thing you're optimizing for today has to be a vision of where AI will be in 2 years and Nvidia's fully accepted they don't know where that's going to be that's why they have a portfolio of chips now not just one GPU line, right? It's not just Hopper, Blackwell, Rubin. Now, it's going to be, you know, it's not Ampere, Hopper, you know, it's not that line. It's like there's a variety of chips to serve the different markets and different possible scenarios. They think each of them has this vision today, but oh, it might turn out the general purpose one sucks and actually AI models have developed in a way where CPX or Grock style chips are the best, right? Well, okay, now we have a solution for that market. And so, I think that's the challenge with the startups. With that said, I think they're all taking very interesting bets. I think it's much more exciting than the first wave of AI hardware bets. Graphcore, Cerebras bringing the memory on the chip they sort of just made a bet and they optimized for a certain kind of model all similar kinds of model and it didn't end up working out for a long time right they had to pivot and they had to work on a lot of things and it took a long time. I think these companies have like a really clear vision of what they think models will look like right like Etched does MatX does, Positron does, and that's what's really cool about it between the three of them, these new age. So, I mean, I'm excited for them. I'm very very skeptical. I don't know what a venture capitalist views as likely chances of succeeding, but I think all of them are less than 1%. Right?

But the world where they win is a multi-silicon kind of world where any given customer uses a range of different GPUs. It could be any given customer has like one workload they care a lot about. Anthropic clearly does not give a crap about video gen, image gen right they just don't care. On the flip side, company like Midjourney cares a lot about image and video gen, right? Image and video gen is very very like like I mentioned like it's a very like it's not very memory bandwidth heavy. It loves loves loves compute, right? Whereas inference of large language models in the style of like you know these you know say for example coding agents cares a lot about decoding for long streams of time. And that's very memory bandwidth heavy, right? And so there's like that's like a simple example, but there's a lot more nuance there in terms of like even like the size of like the matrix multiply, you know, the tensor cores that you use or the ratios of networking and memory and like what's that memory hierarchy look like and you know what are you doing for different kinds of attention and like oh like all these sorts of things like there's a lot of specialization here and so some people are betting big on different types of specialization and I think like you could clearly see a world where companies do care about different stuff right like if for example a chip optimized for video and image generation existed today and it was better than Nvidia or Nvidia made it. I think Midjourney would absolutely only use that for inference. I think for training they'd still use the general purpose thing and as would like Meta and Google would like they should do that, right? And hey, Meta actually has two lines of AI chips there. MTIA there's a line that's focused on recommendation systems and then there's a line that's focused on Gen AI. The Gen AI one is a new line, but that recommendation systems line is still continuing, right? It's not sexy. No one cares because there's no and ByteDance also has a recommendation system line of chips and it's not really focused on Gen AI which is fine because you know this is a $200 billion business or something which is just deciding what ad to serve me right and what order to put my friends stories and you know things like this so I think like it's perfectly fine for there to be specialized AI chips given the target market is big enough and you have to have vision to know what that target market is unless you're a hyperscaler then you can just use general purpose until it's clearly there and then you can make your ASIC right.

It's a variety of things actually in some quarters last year it was even north of 20 I think but I don't remember exactly but anyways you know if you look at 2022 China was almost the size of the US in terms of buying server hardware right almost not quite but getting there and it looked like they were going to be the same size as America in like a year or two after that right and if you look at like global data center capacity global cloud capacity etc etc etc it's American companies and Chinese companies right that dominate the world American companies obviously doing a lot better here but both of those dominate the world and if you look at like every industry right you know it's very clear that like China wants to insource stuff, right? So in 2015, they made these 5-year plans for 2020 and 2025 where they set the percentage of semiconductors they wanted domestically produced and they've missed the goal both times which is fine, right? They set really aggressive goals and even you know shoot for the moon even if you miss you hit the stars, right? And that's sort of what's happened, right? Like look, China is not caught up on, you know, leading edge semiconductors, but microcontrollers from China are almost as good as the microcontrollers are as good and cheaper than the ones from Texas Instruments or ST Micro or, you know, etc., right? Or like this random power chip is better than or the same as the one from like another company, right? And so they've really built up a semiconductor industry and started insourcing a lot more. I don't see why China wouldn't be buying you know 30 40% of the world's AI chips and the US like 50 60% and then the rest of the world like you know and when I say US I mean US origin companies that seems like a more natural state for the world but there are restrictions and hey this is the biggest change in human history maybe ever knowledge work and you know everything that's going to happen there and then eventually like robotics and all these things like you know obviously there's a lot of geopolitical stuff and so there are restrictions Nvidia's been handicapped from selling their best chips to China. And so that's obviously impacted the sales a lot because like why would you do that? And so when you look at who rents the most GPUs in the world, it's three companies, right? So one of them is obviously OpenAI. Second one, actually they were bigger than OpenAI. They are bigger than OpenAI today or no, they were bigger than OpenAI then OpenAI eclipsed them recently is ByteDance. ByteDance runs rents tons of chips from Oracle and Google and you know many other cloud companies because they couldn't get the chips they need in China. They're mostly just serving TikTok, right? Okay. Well, they're not allowed to buy them and that sucks, but you know, they're allowed to rent them. And so, okay, if I'm not allowed to get the best ones, I'm going to rent externally. And if ByteDance is the second biggest renter of GPUs in the world, that's substituting demand that would have been built in China in many cases. It's instead being built in Malaysia. And Oracle has over a gigawatt of capacity in Malaysia that ByteDance is going to take, right? So, things like this are, you know, hundreds of thousands, if not millions of chips, tens of billions of dollars of capacity that would go to China, but it's not. That it's going to Malaysia instead as an example. Another sort of point around this is China's like you know they've had these 5-year plans. So and you know the way these initiatives work from China is there is like some top down ordering but then they just kind of whip the whole like everyone just kind of gets into it and it's really cool like I don't think it's as top down as many people think. Like I think the entire country is like.

Semiconductor pill right there are dramas where people fall in love in the fab or dramas where people fall in love and they're photovoltaic like solar cell researchers and engineers and it's like this is just the backdrop and it's like actually this is super cool for your significant other to be that semiconductor engineer or to be that photovoltaic solar panel researcher.

As opposed to an influencer. Right. Like I'm sorry. Love Island is I watched like for 10 minutes because I was forced to. I was like this is freaking terrible.

No, you know, seriously we're cooked. We're cooked. And I think like when you think about like this happens it's like it's diffused into drama even people like there's multiple dramas like taking place about semiconductor industry and they're like romance comedy like the entire spectrum, right? Drama like it's like what the heck is going on? Anyways, you have all these provinces, you have all these local cities studying out ordinances and giving out subsidies and all sorts of stuff, right? It's truly like crazy. Like there's some national level stuff like, 'Oh, no taxes on this. Oh, we're going to ban a few things.' But as far as I understand, the national government has not banned Nvidia's H20 or H200. But the local ones have, right? A lot of local ones have said, 'No, you know, you must use China manufactured chips.' And it's like, who told you that, you know, you're here to uphold this? It's like, does it matter, right? I mean, like, it's cool because then you have this like survival of the fittest, all these provinces and cities are trying to attract different companies with different types of subsidies and grants and industrial parks and like all these different things.

This how one thinks of China, right? It almost sounds like more like the US or like with the federal government and states where the provinces have authority over their purchasing. It's actually like great. There's this one TikTok or not TikTok, TikTok and Instagram like person and they're like they sing it. They're like if you want to buy things in China, make sure you go to the right place. And then they just say the most random stuff and name the city. And then you look into it and you're like wow this city has the entire supply chain for this. And it's like lampshades and then it names the city. It's like what the... There's a city that specializes in lampshades. Like it's like and it's like microphone arms like microphones. It's like literally there's a city in China that specializes in...

It's literally everything. Literally everything. There's a city and it's not like hey specifically for camera arms for example, there's ball bearings in this and the ball bearings are like there's ball bearings. There's multiple manufacturers of ball bearings for camera arms.

And so like the semiconductor industry I think people don't realize is absurdly specialized. I'm not answering your question. I'm just going a little bit of a rant because I think people don't understand China semiconductors. It's really sick or semiconductors in general. But like you know like in Japan they like focus on a few different types of chemicals and they're the best at it and it's like almost a cultural thing, right? Japanese people were so precise like with sushi and like it's all about the trade and the craft and like you know the French food in Japan is better than the French food in France because the Japanese chefs went there and then come back and they perfected it in Japan and like because they're so precise and there's so many different things that like Japan is so good at because they're so precise and like dedicated to the craft and it comes out of like I don't know like samurai culture or something I don't know right like I don't exactly know how that culture came up and so when you look at like and it's like across the world there's different places where things like this happen right like Oh, like the Netherlands makes EUV tools. Cool. I guess so. And you look across the semiconductor industry. There's a famous economic essay called I pencil or something like that. Or talking about how the pencil like a simple pencil comes from like oh the rubber comes from like Indonesia for the eraser and the graphite comes from this mine here and the wood comes from these aspen trees in Canada and like you actually can't make a pencil without aggregating this entire supply chain. Semiconductor industry is like way crazier because like I would say there's like 15 or 20 countries that could shut down the entire semiconductor industry, right? Even like Austria could, right? And it's like what? And it's like well yeah, there's two different companies there who have like 90% share in like some random niche stuff.

And there's lynchpin critical things everywhere because the process is so complicated. And so China's been trying to replicate this.

I think there's a lot of things. I think if you were to close your eyes and say or if you were to cut off every country and say there's no more globalism, China has the most vertical stack in semiconductors today and they're the best at semiconductors in the world because their fabs could still run somewhat on a lot of things because they have built some of these chemical supply chains, right? Like TSMC for certain kinds of chemicals 100% share from Japan, right? Or Intel same thing, right? Or you know for certain kinds of tools 100% share from Netherlands or 100% share from you know this American company or that Austrian company or this or that right like there's just all these like you know this Swiss company like there's just all these different places have 100% share it might be one company might be three companies but geographically or in the same area and China's built that up right because they've created this made in China initiatives which just plowed money into it and they've got this culture of like the diffused like you know these provinces like yeah I just decided I'm going to focus on or might not even be might not even be the Right. It may be the like, you know, someone brought it there and decided and then people were like, 'Oh, wow. You're doing that?' Me, too. Like, I'm a Patel and I grew up in a motel and guess what? We like almost all the Patels I know grew up in a motel and it's because some random Patel immigrated to America and like worked at a hotel motel and then bought a motel and then like it just started happening, right? Like you sort of like these things are serendipitous of sorts and like I don't know like and it's like I view it as the same kind of specialization, right? Chinese cities are like starting to do these things. China's missing a lot of things, right? I would say like if you say minus 10 years tech, China's complete and no one else is complete, right? Taiwan is not complete. Their fabs would shut down without foreign supply, you know, and you go down or you go across the stack. But if you go to 10-year tech, maybe maybe more like 20-year tech, you could get a fully vertical supply chain in China, which I do not think any country could do. Like America could not build a fully vertical fab without stuff from elsewhere, even if it's 20-year old tech.

And so, so that's interesting. But then when the flip side is like well like you kind of do need specialization. That's how that chemical gets the purest best, you know, most engineered, you know, or that slurry of chemicals or that, you know, that gas or like that tool because every smart person or a lot of them in that country grew up around that culture and like every the supply chain is there and like everyone kind of knows and like it's like a driveaway and like sort of like this is what makes supply chains work is that there is this specialization and the best of the best only comes when you have that hyper specialization. So, China doesn't have lithography. Their lithography is like 10 years behind and I think it'll be 5 years behind in a couple years, right? They're catching up fast. I don't think they'll be as good as ASML for a long time. You know, maybe I don't know, maybe they will be, you know, China. You shouldn't ever underestimate China, but like and Chinese engineers or, you know, but like for a while, right? Or like, you know, I don't think they'll be able to make leading edge chemicals like many Japanese companies or many American companies and their tools and like you just go across the supply chain. They're not hey forefront on really anything in the manufacturing supply chain on the design supply chain. There's some things that they're starting to be similar par but like cheaper or like a year or two behind but cheaper and that's like fine for a lot of stuff. An example of that is Huawei, right? Huawei in mobile phones was on par with Apple like entirely. Yeah. And they had become Apple TSMC's biggest customer and they were designing the best thing and they are number one in telecom and their tech is just literally better. And so when you think what happens, you know, is China missing anything? It's like they don't have the best of much, you know, today in the AI supply chain. They have a complete package and a couple years behind and they'll figure out how to make it cheaper slash do more slash catch up and create a robust industry. But there's a reason like I don't think that like Jensen is scared of AMD really. He's paranoid. I mentioned he's paranoid. I'm sure he's a little bit scared of them, right? Like I think some of the things that they've done are reactions and competitive dynamics with AMD or Google's TPUs or whatever. Right. There was a Core Weave deal today and I think that's directly the result of what Google's been doing.

Nvidia invested two billion in Core but what's more important is that that's like sort of just like the sticker what's really relevant is Nvidia is going to work with Core Weave to acquire and backstop and all these things the land the power the energy the transmission that help build the data center all this capital side stuff that because Nvidia has so much money they can backstop Core Weave doing it because Core Weave then can be the one who generates demand anyways there's like because Google was doing And they did that with like a couple companies such as Fluid Stack and Terowolf and Cipher. These are some public deals that have been announced. And so Google is doing that with TPUs and Nvidia reacted, right? And so in the same way, I think Nvidia's reacted to AMD. And in the same way, I think the thing is Nvidia is like deathly terrified of Huawei.

They did just crush Nokia, Sony, Ericsson, etc., right? Like the entire telecom supply chain. They just like completely destroyed them. And there's so many other areas like they straight up made a folding phone, right? You know, I have a Samsung folding phone. They have a folding phone that's better than Samsung's folding phone.

And so, of course, they're terrified of Huawei and Huawei is the most vertical company in the world. No company is more verticalized than Huawei, which then leads to huge innovations. It's something that we don't fully appreciate in the US, but like when you travel in Europe, you see everybody who's like Honor phones and it's like the footprint of Huawei is huge in phones in a way that people...

A lot of training on the...

Exactly. Exactly. I think Huawei is terrifying right and so like yes their chips are not as good today.

Well they shipped a little bit but like mostly just like sticker capacity like there's nothing like no no like I would say like a little bit as in like a few servers not like a billion dollars worth of stuff right the thing is China's supply chain has to ramp up right China's express goal is to have all internalized but then like a company like Alibaba's like I don't want to use Huawei, right? Like I want to use Nvidia and just make the best freaking models, right? Because that's my business. My business is not, you know, using a Huawei thing, but it's like, okay, it's being pushed upon me. There's other companies too, like Cambricon and so on and so forth. And so the sort of like supply chain, you know, companies in China don't want to use they're kind of encouraged obviously and pushed, you know, you must some local provincial government be like, well, you're doing this much business here. You got to do this, right? Like there's all sorts of like crazy stuff that, you know, pushing of companies to use Huawei. The challenge is probably can't manufacture enough, right? We've like done a lot of work on this. And we've just put it for free, you know, instead of like to our customers because it's like something that's like national security, which is how was Huawei actually building chips? Well, actually they were using shell companies to get chips from TSMC and using different methods of like sneaking HBM, which is memory, from Korea through Taiwan to China, right? Like all sorts of crazy stuff we've reported on and people it's like a whack-a-mole, right? They shut it down or like tools that get shipped to China and they shouldn't be for you know making leading edge chips but they actually are and all these sorts of things are happening because they can't make everything and if they want to make the leading edge stuff they do need to rely on the foreign supply chain quite a bit in terms of the upstream supply chain right memory logic chips tools for fabs chemicals for fabs etc Huawei cannot satisfy the market because there's not enough advanced leading edge capacity in memory logic and all these other things domestically and they're trying to build it as fast as they can, but that means there's just not enough to satisfy the market. So, Nvidia has a market. I think they'll figure out how to sell chips to China. And Jensen's in China, I think, like right now or was yesterday. And so, like he's clearly like wheeling and dealing to try and get his chips into China because, you know, I think Nvidia's argument is if we sell them chips, then they won't, you know, there won't be enough of as much of a domestic market. The feedback loop for software and everything else won't be there. That will sort of like really challenge it, right? Like most of the open source software for AI has a lot of Chinese contributors, right? vLLM and PyTorch, SGLang and like all of these other like libraries and things that are just like you know and it goes to low-level software especially right like a lot of the best open source stuff is actually just from like a Chinese company who decided to open source it and same with models right and so like it's like okay well if they can't use Nvidia chips anymore then this open source stuff won't be designed for Nvidia chips it'll be designed for Huawei chips and now does that like weaken the CUDA moat and now like not only is China domestic now they have like a feedback loop internally and then they can externalize across the rest of the world right so This is the like argument Nvidia makes. I'm not sure if I am like I'm like you know I think my AI timelines are so fast. I'm not that fast like not in terms of like AGI but like hey AI is hundred billion dollars of revenue across the industry. I think the industry could hit 100 billion ARR by the end of this year like 45-50 for OpenAI like 35-40 for Anthropic and then you know Vertex, DeepMind's models at Google Gemini right and then Vertex API for Anthropic models and Bedrock APIs and Azure Foundry APIs like I think hundred billion dollars like end of this year.

ChatGPT has a billion users roughly and you know, then you add on Gemini and Meta claims they have 500 million users. I don't know. I think people just accidentally click like generative sticker or something. But like anyways, like there's like a lot of usage of AI in the west already and it's going to climb. It's going to keep climbing and like you kind of have to get used to it and so like the question is like what's the economic benefit to the world, right? And at the end of the day, this is an economic war, right? If the US and the West win in AI and control, you know, more powerful AI systems that have this feedback loop that improved economic growth and weapons systems and whatever else, right? Engineering of grids and cyber attacks and all these sorts of things. They have this like advantage over China, then China will not rise to be the global hegemony. But without AI, China definitely will rise to be the global hegemony. They're just going to outrun America. And so the question is like you know that's I think like the other view right and how fast are super powerful AI systems versus you know China building a domestic ecosystem for chips and models and everything that is a few years behind like what's actually the value right like that's sort of like...

I think TSMC's manufacturing wafers and they're like building real wafers and there's real fabs and like you know there's some other fabs that have been announced and like they're doing well and there's like a bunch of like different kinds of plants like a Korean company making a random gas plant in Texas for you know their chips right like for chips and all these sort of things are happening. I think the CHIPS Act did really well with its $50 billion. It's just I don't think people understand the scale of the semiconductor industry. It is the most complicated supply chain in the world, right? It's much bigger than, you know, say manufacturing airplanes. It's much bigger than like, you know, really anything else, right? If you look at the top 10 companies like of the world, I think eight of them designed semiconductors, right? Now, obviously like Google designed semiconductors, but it's like, oh wait, no, but their cost of search would be like 10x higher if they didn't have TPUs and TPUs were super optimized for search, right? Or like, you know, you go down the list, right? Like Meta serves recommendation systems with their chips, right? Like you go down the list, it's everyone is making their own chips. Apple devices would be materially worse if they didn't have their own chips, right? And you just go down the list, it's like it's the most complicated supply chain and they're spending something on the order of like $150 billion roughly in subsidies a year to the chip industry.

Yeah, there's a difference in scale here, right? The collective total amount of like capex that has been spent in Taiwan is like 500 billion plus, right? Across the industry, across all the companies that are making semiconductors in Taiwan and Taiwan doesn't have a domestic industry. How is $50 billion of subsidies going to change America's needle? Right? It does move it a little bit, right? I want to be clear like the CHIPS Act is awesome. I don't understand why like EVs or like solar was given this massive massive like trillion dollar package. Semiconductors were only given 50. Like semiconductors need a lot bigger package to actually incentivize onshoring. I think what's happened so far has proven that it's working well. TSMC is literally making chips for Nvidia and Apple and AMD and others in Arizona today, right? And I think that's really great.

Or not worse, but like this is just like a thing, right? Just in time inventory systems. COVID happens, sales plummet fabs that were making, you know, random power ICs or random microcontrollers for engines got repurposed to the boom from COVID, which was data centers and PCs and smartphones. So, that stuff was booming. And then when people were like, 'Oh, wait. Actually, like, you know, I have some money. I stayed at home. I didn't go out. I didn't drink. I have some cash, right? Let me buy a car.' They went out and bought cars and cars started skyrocketing in prices. Oh, let's restart and let's Oh, yeah. Can you sell me that microcontroller for the engine again? It's like, 'No, I'm making a slightly different microcontroller that works for, you know, let's say a keyboard or a mouse, right, or whatever.' And it's like, and they actually didn't just leave me flatfooted and they were like a partner through COVID, right? You know, versus you just left me. Screw you Ford or whoever, Toyota, or automotive OEM, you up that supply chain. And so, CHIPS Act did not get passed, only got passed because that happened. And people are like, 'Oh my god, the semiconductors are why cars can't be made.' If that didn't happen, we wouldn't even have the CHIPS Act. It's like silly. So like I don't know like I think you know whereas like and even though that's what was pitched to all the senators like I know people who were running around Capitol Hill just pushing that narrative and story and that's why it finally got passed in reality it was all for advanced leading edge chips, right? Nothing that goes in a car, right? And so it's like this like funny thing.