GPUs, TPUs, & The Economics of AI Explained | Gavin Baker Interview

In every way, data centers in space from a first principles perspective are superior to data centers on earth.

I mean, I think the first thing is you have to use it yourself. And I would just say I'm amazed at how many famous and August investors are reaching really definitive conclusions about AI. Well, no, based on the free tier. The free tier is like you're dealing with a 10-year-old and you're making conclusions about the 10-year-old's capabilities as an adult. And you could just pay and I do think actually you do need to pay for the highest tier whether it's Gemini Ultra, you know, Super Grok, whatever it is, you have to pay the $200 per month tier whereas those are like a fully fledged 30, 35-year-old. It's really hard to extrapolate from an eight or a 10-year-old to the 35-year-old and yet a lot of people are doing that. And the second thing is there was an insider post about OpenAI and they said to a large degree OpenAI runs on Twitter vibes and I just think AI happens on X and you know there have been some really memorable moments like there was a giant fight between the PyTorch team at Meta and the Jax team at Google on X and the leaders of each lab had to step in publicly say no one from my lab is allowed to say bad things about the other lab and I respect them and that is the end of that. The companies are all commenting on each other's posts. You know, the research papers come out. You know, if on planet Earth there's 500 to a thousand people who really really understand this and are at the cutting edge of it and a good number of them live in China. I just think you have to follow those people closely and I think there is incredible signal to me. Everything in AI is just downstream of those people.

Yeah. He's incredible. And then I would say anytime at one of those labs, the four labs that matter, you know, being OpenAI, Gemini, Anthropic, and XAI, which are clearly the four leading labs. Like anytime somebody from one of those labs goes on a podcast, I just think it's so important to listen. And then for me, one of the best use cases of AI is to keep up with all of this. You know, just like listen to a podcast and then if there are parts that I thought were interesting, just talk about it with AI. And I think it's really important to like have as little friction as possible, I'll bring it up. You know, I have it like you know, I have I can either press this button and pull up Grok or I have this.

Yeah, it brings it right up. What do you think of Patrick O'Shaughnessy?

Gurley? Yes. It's so Can you believe we have this?

So something like Gemini 3 comes out, you know, the public interpretation was, oh, this is interesting. It seems to say something about scaling laws and the pre-training stuff. What is your frame on like the state of general progress in frontier models in general? Like what are you watching most closely?

Yeah. Well, I do think Gemini 3 was very important because it showed us that scaling laws for pre-training are intact. They, you know, stated that unequivocally and that's important because no one on planet Earth knows how or why scaling laws for pre-training work. It's actually not a law. It's an empirical observation and it's an empirical observation that we've measured extremely precisely and has held for a long time. But our understanding of scaling laws for pre-training and maybe this is a little bit controversial with 20% of researchers but probably not more than that is kind of like the ancient British people's understanding of the sun or the ancient Egyptians' understanding of the sun. They can measure it so precisely that the east-west axis of the great pyramids are perfectly aligned with the equinoxes and so are the east axes of Stonehenge. Perfect measurement. But they didn't understand orbital mechanics. They had no idea how or why, you know, it rose in the east, set in the west, and, you know, kind of moved across the horizon.

Yeah. Our god in a chariot. And so it's really important every time we get a confirmation of that. So Gemini 3 was very important in that way. But I'd say I think there's been a big misunderstanding maybe in the public equity investing community or the broader more generalist community based on the scaling laws of pre-training. There really should have been no progress in 24 and 25. And the reason for that is, you know, after XAI figured out how to get 200,000 Hoppers coherent, you had to wait for the next generation of chips. Because you really can't get more than 200,000 Hoppers coherent. And coherent just means you could just think of it as each GPU knows what every other GPU is thinking. They kind of are sharing memory. You know, they're connected. They scale up networks and scale out. And they have to be coherent during the pre-training process. And I think there's a lot of misunderstanding about Gemini 3 that I think is really important. So everything in AI has a struggle between Google and Nvidia and Google has a TPU and Nvidia has their GPUs and each of I mean Google only has a TPU and they use a bunch of other chips for networking. You know Nvidia has the full stack and Blackwell was delayed. Blackwell was Nvidia's next generation chip and the first iteration of that was the Blackwell 200. A lot of different SKUs were cancelled and the reason for that is it was by far the most complex product transition we've ever gone through in technology. Going from Hopper to Blackwell, first you go from air cooled to liquid cooled. The rack goes from weighing round numbers 1,000 lb to 3,000 lb. Goes from round numbers 30 kilowatts which is 30 American homes to 130 kilowatts which is 130 American homes. So I analogize it to imagine if to get a new iPhone you had to change all the outlets in your house to 220 volt, put in a Tesla power wall, put in a generator, put in solar panels, put in a whole home humidification system and then reinforce the floor because the floor can't handle this. So it was a huge product transition and then just the rack was so dense it was really hard for them to get the heat out. So Blackwells have only really started to be deployed and really scaled deployments over the last 3 or 4 months. Had reasoning not come along, there would have been no AI progress from mid 2024 through essentially Gemini 3. There would have been none. Everything would have stalled and can you imagine what that would have meant to the markets like for sure we would have lived in a very different environment. So reasoning kind of bridged this like 18-month gap. Reasoning kind of saved AI because it let AI make progress without Blackwell or the next generation of TPU which were necessary for the scaling laws for pre-training to continue. The reason we've had all this progress, maybe we could show like the ARC AGI slide where you had, you know, you went from 0 to 0 to 8 over four years, 0 to 8% intelligence and then you went from 8% to 95% in 3 months when the first reasoning model came out from OpenAI is, you know, we have these two new scaling laws of post-training, which is just reinforcement learning with verified rewards. Verified is such an important concept in AI. Like one of Karpathy's great things was with software, anything you can specify, you can automate. With AI, anything you can verify, you can automate. It's such an important concept and I think an important distinction. And then test time compute. And so all the progress we've had, and we've had immense progress since October 24th through today was based entirely on these two new scaling laws. And Gemini 3 was arguably the first test since Hopper came out of the scaling law for pre-training and it held. And that's great because all these scaling laws are multiplicative. So now we're going to apply these two new reinforcement learning with verified rewards and test time compute to much better base models. Google came out with the TPU v6 in 2024 and the TPU v7 in 2025. And in semiconductor time, it's like almost like imagine like Hopper is like, you know, it's like a World War II era airplane. And it was by far the best World War II era airplane. It's P-51 Mustang with the Merlin engine. And two years later in semiconductor time, that's like, you know, you're an F4 Phantom. Because Blackwell was such a complicated product and so hard to ramp, Google was training Gemini 3 on 24 and 25 era TPUs, which are like F4 Phantoms. Like Blackwell, it's like an F-35. It just took a really long time to get it going. So, I think, you know, Google for sure has this temporary advantage right now. From a pre-training perspective, I think it's also important that they've been the lowest cost producer of tokens. Okay. And this is really important because AI is the first time in my career as a tech investor that being the low-cost producer has ever mattered. Apple is not worth trillions because they're the low-cost producer of phones. Microsoft is not worth trillions because they're the low-cost producer of software. Nvidia is not worth trillions because they're the low-cost producer of AI accelerators. It's never mattered. And this is really important because what Google has been doing as the low-cost producer is they have been I would say sucking the economic oxygen out of the AI ecosystem which is an extremely rational strategy for them and for anyone who's a low-cost producer let's just make life really hard for our competitors. And so what happens now I think this has pretty profound implications. One, we will see the first models trained on Blackwell in early 2026. I think the first Blackwell model will come from XAI. And the reason for that is just it's a according to Jensen, no one builds data centers faster than Elon. Yes, Jensen has said this on the record. Even once you have the Blackwells, it takes 6 to 9 months to get them performing at the level of Hopper because the Hopper is finally tuned. Everybody knows how to use it. The software is perfect for it. Engineers know all its quirks. You know, everybody knows how to architect a Hopper data center at this point. And by the way, when Hopper came out, it took 6 to 12 months for it to really outperform A100, which was the generation before. So, if you're Jensen or Nvidia, you need to get as many GPUs deployed in one data center as fast as possible in a coherent cluster so you can work out the bugs. And so this is what XAI effectively does for Nvidia because they build the data centers the fastest. They can deploy Blackwells at scale the fastest and they can help work with Nvidia to work out the bugs for everyone else. So because they're the fastest, they will have the first Blackwell model. We know that scaling laws for pre-training are intact and this means the Blackwell models are going to be amazing. Blackwell is I mean it's not an F-35 versus an F4 Phantom, but from my perspective it is a better chip, you know, maybe it's like an F-35 versus a Rafale. And so now that we know pre-scaling holding, we know that these Blackwell models are going to be really good. And you know, kind of based on the raw specs, they should probably be better. Then something even more important happens. So the GB200 was really really it was really hard to get coherent. The GB300 is a great chip. It is drop in compatible in every way with those GB200 racks. Now, you're not going to replace the GB200s. No new power walls. Yeah. Just any data center that can handle those. You can slot in the GB300s. And now everybody's good at making those racks and you know how to get the heat out. You know how to cool them. You're going to put those GB300s in and then the companies that use the GB300s, they are going to be the low-cost producer of tokens. Particularly if you're vertically integrated. If you're paying a margin to someone else to make those tokens, you're probably not going to be. I think this has pretty profound implications because it I think it has to change Google's strategic calculus. If you have a decisive cost advantage and you're Google and you have search and all these other businesses, why not run AI at a negative 30% margin? It is by far the rational decision. You take the economic oxygen out of the environment. You eventually make it hard for your competitors who need funding unlike you to raise the capital they need. And then on the other side of that, maybe have an extremely dominant share position. Well, all that calculus changes once Google is no longer the low-cost producer, which I think will be the case. The Blackwells are now being used for training. And then when that model is trained then you start shifting Blackwell clusters over to inference and then all these cost calculations and these dynamics change and I do think it's very interesting like during the strategic and economic calculations between the players. I've never seen anything like it. You know everyone understands their position on the board, what the prize is, you know what play their opponents are running. And it's really interesting to watch. So, I just think if Google changes its behavior, because it's going to be really painful for them as a higher cost producer to run that negative 30% margin, it might start to impact, you know, their stock. That has pretty profound implications for the economics of AI. And then when Rubin comes out, we'll know the gap is going to expand significantly versus TPUs. Versus TPUs and all other ASICs. Now, I think Trainium 3 is probably going to be pretty good. Trainium 4 are going to be good.

A couple of things. So one, for whatever reason, Google made more conservative design decisions. I think part of that is so Google round numbers Google like let's say the TPU Google is so there's front end and back end of semiconductor design and then there's dealing with Taiwan Semi. You can make an ASIC in a lot of ways. What Google does is they do mostly the front end for the TPU and then Broadcom does the back end and manages Taiwan Semi. It's a crude analogy but like the front end is like the architect of a house. They design the house. The back end is the person who builds the house and the managing Taiwan Semi is like stamping out that house like Lennar or you know Dr. Horton. And for doing those two latter parts Broadcom's a 50 to 55% gross margin. We don't know what on TPUs. Okay, let's say in 2027 TPU I think it sits estimates maybe somewhere around 30 billion again who knows I mean 30 billion I think is a reasonable estimate 50 to 55% gross margins so Google is paying Broadcom $15 billion. Okay that's a lot of money. And at a certain point it makes sense to bring a semiconductor program entirely in house so in other words Apple does not have an ASIC partner for their chips they do the front end themselves, the back end and they manage Taiwan Semi and the reason is they don't want to pay that 50% margin so at a certain point it becomes rational to renegotiate this and just as perspective the entire opex of Broadcom's semiconductor division is round numbers $5 billion so it would be economically rational now that Google's paying if it's 30 billion we're paying them 15 Google can go to every person who works in Broadcom Semi double their comp and make an extra 5 billion. You know, in 2028, let's just say it does 50 billion. Now it's 25 billion. You could triple their comp. And by the way, you don't need them all. And of course, they're not going to do that because of competitive concerns. But with TPUv8, all of this and V9, all of this is beginning to have an impact because Google is bringing in MediaTek. This is maybe the first way you send a warning shot to Broadcom. We're really not happy about all this money we're paying but they did bring MediaTek in and the Taiwanese ASIC companies have much lower gross margins so this is kind of the first shot across the bow and then there's all this stuff people say oh but Broadcom has the best SerDes. Broadcom has really good SerDes and SerDes is like an extremely foundational technology because it's how the chips communicate with each other you have to serialize and deserialize but there are other good SerDes providers in the world a really good SerDes is not at a certain point Maybe it's worth 10 or 15 billion a year, but it's probably not worth 25 billion a year. So because of that friction, and I think conservative design choices on the part of Google and maybe the reason they made those conservative design choices is because they were going to a bifurcated supply. You know, TPU is slowing down. I would say has kind of the GPUs are accelerating. This is the first, you know, the competitive response of Lisa and Jensen to everybody saying we're gonna have our own ASIC is, hey, we're just going to accelerate. We're going to do a GPU every year and you cannot keep up with us. And then I think what everybody is learning is like, oh wow, that's so cool. You made your own accelerator as an ASIC. Wow, what's the NIC going to be? What's the CPU going to be? You know, what's the scale-up switch going to be? What's the scale-up protocol? What's the scale-out switch? What kind of optics are you going to use? What's the software that's going to make all this work together? And then it's like, oh I made this tiny little chip and you know, like whether it's admitted or not, like you know, I'm sure the GPU makers don't love it when their customers make ASICs and try and compete with them and like whoops what did I do? I thought this was easy. How do you know? And it also it takes at least three generations to make a good chip like the TPU V1. I mean it was an achievement and that they made it. It was really not till TPU V3 or V4 that the TPU started to become like even vaguely competitive.

100%. Yeah. And even if you've made like the first from my perspective, the best ASIC team at any semiconductor company is actually the Amazon ASIC team. You know, they were the first one to make the Graviton CPU. They have this Nitro. It was the first, it's called Supernic. They've been extremely innovative, really clever. And like Trainium and Inferentia 1, you know, they maybe they're a little better than the TPU V1, but only a little. Trainium 2, you get a little better. Trainium 3 it's I think the first time it's like okay and then you know I think Trainium 4 will probably be good. I will be surprised if there are a lot of ASICs other than Trainium and TPU and by the way and Trainium and TPU will both run on customer-owned tooling at some point. We can debate when that will happen but the economics of success that I just described mean it's inevitable. Like no matter what the companies say, just the economics make it and reasoning from first principles make it absolutely inevitable.

That's what I mean. It's crazy.

If I were to posit like an event path, I think the Blackwell models are going to be amazing. The dramatic reduction in per token cost enabled by the GB300 and probably more the MI450 than the MI355, you know, will lead to these models being allowed to think for much longer, which means they're going to be able to, you know, do new things. Like I was very impressed Gemini 3 made me a restaurant reservation. It's the first time it's done something for me. And I mean, other than like go research something and teach me stuff, but you know, if you can make a restaurant reservation, you're not that far from being able to make a hotel reservation and an airplane reservation and order me an Uber and all of a sudden you got an assistant. Yeah. And you can just imagine, everybody talks about that, but you can just imagine it's on your phone. I think that's pretty near-term, but you know, it's some big companies that are very tech forward. You know 50% plus of customer support is already done by AI and that's a $400 billion dollar industry and then if you know what AI is great about is persuasion that's sales and customer support and so of the functions of a company if you think about them they're to make stuff, sell stuff and then support the customers so right now maybe you're in late 26 you're going to be pretty good at two of them. I do think it's going to have a big impact on media like I think robotics you know we talked about the last time are going to finally start to be real. You know, there's an explosion in kind of exciting robotic startups. I do still think that the main battle is going to be between Tesla's Optimus and the Chinese because, you know, it's easy to make prototypes. It's hard to mass-produce them. But then it goes back to that what Andre Karpathy said about AI can automate anything that can be verified. So any function where there's a right or wrong answer, a right or wrong outcome, you can apply reinforcement learning and make the AI really good at that.

Does the model balance? They'll be really good at making models. Do all the books globally reconcile? They'll be really good at accounting because it you know was double entry bookkeeping. It has to balance. There's a verifiable you got it right or wrong. Support or sale. Did you make the sale or not? That's very clear. I mean that's just like AlphaGo. You know, did you win or you lose? Did the guy convert or not? Did the customer ask for an escalation during customer support or not? It's like its most important functions are important because they can be verified, right? So I think if all of this starts to happen and starts to happen in 26 like there'll be an ROI on Blackwell and then all this will continue and then we'll have Rubin and then that'll be another big quantum of spend Rubin and the MI450 and the TPU V9 and then I do think just the most interesting question is what are the economic returns to artificial super intelligence because all of these companies in this great game they've been in a prisoner's dilemma. They're terrified that if they slow down, they're just gone forever. And their competitors don't, it's an existential risk. And you know, Microsoft blinked for like six weeks earlier this year. Yeah. And like I think they would say they regret that. Yeah. But with Blackwell and for sure with Rubin, economics are going to dominate the prisoner's dilemma from a decision-making and spending perspective just because the numbers are so big. And this goes to kind of the ROI on AI question. And the ROI on AI has empirically, factually, unambiguously been positive. Like I just always find it strange that there's any debate about this because the largest bidders on GPUs are public companies. They report something called audited quarterly financials. And you can use those things to calculate something called a return on invested capital. And if you do that calculation, the ROIC of the big public spenders on GPUs is higher than it was before they ramped spending. And you could say, well, part of that is, you know, opex savings. Well, at some level that is part of what you expect the ROI to be from AI. And then you say, well, a lot of it is actually just applying GPUs, moving the big recommender systems that power the advertising and the recommendation systems from CPUs to GPUs, and you've had massive efficiency gains. And that's why all the revenue growth at these companies has accelerated. But like, so what? The ROI has been there. And it is interesting like every big internet company, the people who are responsible for the revenue are intensely annoyed at the amount of GPUs that are being given to the researchers. It's a very linear equation. If you give me more GPUs, I will drive more revenue. Give me those GPUs, we'll have more revenue, more gross profit, and then we can spend money. So, it's this constant fight at every company. One of the factors in the prisoner's dilemma is everybody has this like religious belief that we're going to get to ASI and at the end of the day what do they all want? Almost all of them want to live forever. Okay. And they think that ASI is going to help them that right good return. That's a good return. But we don't know. And if as humans we have pushed the boundaries of physics, biology and chemistry, the natural laws that govern the universe.

There's one really obvious bear case and it is just edge AI and it's connected to the economic returns to ASI. In three years on a bigger and bulkier phone to fit the amount of DRAM necessary, you know, and the battery won't probably last as long, you will be able to probably run a pruned down version of something like Gemini 5 or Grok 4, Grok 4.1 or you know, ChatGPT at I don't know 30, 60 tokens per second and then that's free. And this is clearly Apple's strategy. It's just we're going to be a distributor of AI and we're going to make it privacy safe and run on the phone and then you can call one of the big models, you know, the god models in the cloud whenever you have a question. And if that happens, if like 30 to 60 tokens at like a 115 IQ is good enough. I think that's a bear case other than just the scaling laws break, you know. But in terms of if we assume scaling laws continue and we now know they're going to continue for pre-training for at least one more generation and we're very early in the two new scaling laws you know for post-training, mid-training, RLVR, whatever people want to call it and then test time compute, inference, we're so early in those and we're getting so much better at helping the models hold more and more context in their minds as they do this test time compute and that's really powerful because you know everybody's like well how's the model going to know this? Well, eventually if you can hold enough context, you can just hold every Slack message and Outlook message and company manual in a company in your context. And then you can compute the new task and compare it with your knowledge of the world, what you think, what the model thinks, all this context. And you know, it may be that like, you know, just really really long context windows are the solution to a lot of the current limitations. And that's enabled by some all these cool tricks like KV cache offload and stuff. But I do think like other than scaling laws slowing down, other than there being low economic returns to ASI, edge AI is to me by far the most plausible and scariest bear case.

If you're paying for Gemini Ultra or Super Grok and you're getting, you know, the good AI, it's hard to see differences. Like, I have to go really deep on something like, do you think PCI Express or Ethernet is a better protocol for scale-up networking and why? Show me the scientific papers. And if you shift between models and you ask a question like that where you know it really deeply, you know the answers. Then you see differences. I do play fantasy football. Winnings are donated to charity, but it is like, you know, these new models are quite a bit better at helping like who should I play? You know, and they think in much more sophisticated ways. And by the way, if you're a historically good fantasy football player and you're having a bad season, it's why this is why because you're not using it, you know. And I think we'll see that in more and more domains. But I do think they are already at a level where unless you are a true expert or just have an intellect that is beyond mine it's hard to see the progress and that's why I do think we need to shift from getting more intelligent to more useful unless more intelligence starts leading to these massive scientific breakthroughs and we're curing cancer in 26 and 27.

What are the building blocks of usefulness in your mind?

And we're going to see I think something like that in every vertical and that's AI being used for the most core function of any company which is designing the product and then it will be you know there's already lots of examples of AI being used to help manufacture the product and distribute it more efficiently you know whether it's optimizing a supply chain you know whether it's having a vision system watch a production line you know so I think a lot of stuff is happening. The other thing I think is really interesting in this whole ROI part is Fortune 500 companies are always the last to adopt a new technology they're conservative they have lots of regulations lots of lawyers. Startups are always the first. So let's think about the cloud which was the first which was the last like really truly kind of transformative new technology for enterprises. Being able to have all of your compute in the cloud and use SaaS. So it's always upgraded. It's always great etc. etc. You can get it on every device. I mean those were dark days before the cloud. You know the first AWS re:Invent I think it was in 2013. Every startup on planet Earth ran on the cloud. The idea that you would buy your own server and storage box and router was ridiculous. And that probably happened like even earlier that had probably already happened before the first re:Invent. And then like you know the first big Fortune 500 companies started to standardize on it like maybe 5 years later. You see that with AI I'm sure you've seen this in your startups and I think one reason VCs are more broadly bullish on AI than public market investors is VCs see very real productivity gains. There's all these charts that for a given level of revenue, a company today has significantly lower employees than a company of two years ago. And the reason is AI is doing a lot of the sales, the support and helping to make the product. And I mean there is, you know, Iconiq has some charts. a16z, by the way, David George is a good friend, great guy. You know, he has his model busters thing. So there's very clear data that this is happening. So people who have a lens into the world of venture see this. And I do think it was very important in the third quarter, this is the first quarter where we had Fortune 500 companies outside of the tech industry give specific quantitative examples of AI-driven uplift. So C.H. Robinson went up something like 20% on earnings. And should I tell people what C.H. Robinson does?

So, one of the most important things they do is they quote price and availability. So, somebody a customer calls them up and says, 'Hey, I urgently need three 18-wheelers from Chicago to Denver.' You know, in the past they said it would take them, you know, 15 to 45 minutes and they only quoted 60% of inbound requests. With AI, they're quoting 100% and doing it in seconds. And so they printed a great quarter and the stock went up 20% and it was because of AI-driven productivity that's impacting the revenue line, the cost line, everything. And so I actually think that's pretty important because I was actually very worried about like the idea that we might have this Blackwell ROI air gap because we're spending so much money on Blackwell. Those Blackwells are being used for training and there's no ROI on training. Training is you're making the model. The ROI comes from inference. So I was really worried that you know we're going to have you know maybe this three-quarter period where the capex is unimaginably high. Those Blackwells are only being used for training, bars staying flat, eyes going up. Yeah. Exactly. And so ROIC goes down and you can see like Meta, Meta they printed you know because Meta has not been able to make a frontier model. Meta printed you know a quarter where ROIC declined and that was not good for the stock. So I was really worried about this. I do think that those data points are important in terms of suggesting that maybe we'll be able to navigate this potential air gap in ROIC.

Well, I think that people are going to start to care if you have more and more companies print these C.H. Robinson-like quarters that companies that have historically been really well-run. The reason like they have a long track record of success is they have a long you cannot succeed without using technology well and so if you have a kind of internal culture of experimentation and innovation I think you will do well with AI. You know so like I would bet on the best investment banks to be earlier and better adopters of AI than maybe some of the trailing banks. You know just sometimes past prologue. One thing that I strong opinion I you know, all these VCs are setting up these holding companies and, you know, we're going to use AI to make traditional businesses better and, you know, they're really smart VCs and they're great track records. But that's what private equity has been doing for 50 years. You're just not going to beat private equity at their game.

Yeah. Private equity's maybe had a little bit of a tough run. You know, just multiples have gone up. Now, private assets are more expensive. The cost of financing has gone up. It's tough to take a company public because the public valuation is 30% lower than the private valuation. So PE's had a tough run. I actually think these private equity firms are going to be pretty good at systematically applying AI.

The first thing let me just say about frontier models broadly. You know in 2023 and 24 I was fond of quoting Eric Vishria and Eric Vishria's statement our friend brilliant man and Eric would always say foundation models are the fastest appreciating assets in history. And I would say he was 90% right. I modified the statement. I said foundation models without unique data and internet scale distribution are the fastest appreciating assets in history. But reasoning fundamentally changed that in a really profound way. There was a loop, a flywheel to quote Jeff Bezos that was at the heart of every great internet company and it was you made a good product, you got users, those users using the product generated data that could be fed back into the product to make it better. And that flywheel has been spinning at Netflix, at Amazon, at Meta, at Google, for over a decade. And that's an incredibly powerful flywheel. And it's why those internet businesses were so tough to compete with. It's why they were increasing returns to scale. Everybody talks about network effects much more and you know network effects are they were important for social networks. I don't know to what extent Meta is a social network anymore. It's more like a content distribution but they just had increasing returns to scale because of that flywheel. And that dynamic was not present in the pre-reasoning world of AI. You pre-trained a model, you let it out in the world, and it was what it was. And it was actually pretty hard. They would do RLHF, reinforcement learning with human feedback, and you try and make the bot model better, and maybe you'd get a sense from Twitter vibes that people didn't like this, and so you tweak it, and you know, there were the little up and down arrows, but it was actually pretty hard to feed that back into the model. With reasoning, it's early but that flywheel has started to spin and that is really profound for these frontier labs. So one, reasoning fundamentally changed the industry dynamics of Frontier Labs.

Because if a lot of people are asking a similar question and they're consistently either liking or not liking the answer, then you can kind of use that as a verifiable reward. That's a good outcome. And then you can kind of feed those good answers back into the model. And we're very early at this flywheel spinning like it's hard to do now, but you can see it beginning to spin. So, this is important fact number one for all of those dynamics. Second, I think it's really important that Meta, you know, Mark Zuckerberg at the beginning of this year in January said, you know, I anticipate, you know, I'm highly confident, I'm going to get the quote wrong, that at some point in 2025, we're going to have the best and most performant AI. I don't know if he's in the top hundred. Okay, so he was as wrong as it was possible to be. And I think that is a really important fact because it suggests that what these four companies have done is really hard to do because Meta threw a lot of money at it and they failed. Yann LeCun had to leave. They had to have the famous billion dollar for AI researchers. And by the way, Microsoft also failed. They did not make such an unequivocal prediction but they bought Inflection AI and you know there were a lot of comments from them that we anticipate our internal models quickly getting better and we're going to run more and more of Copilot, our AI, on our internal models. Amazon they bought a company called Adept AI they have their models called Nova no I don't think they're in the top 20 so clearly it's much harder to do than people thought a year ago and there's many many reasons for that like it's actually really hard to keep a big cluster of GPUs coherent. A lot of these companies were used to running their infrastructure to optimize for cost instead of performance complexity and performance complexity and keeping the GPUs running at high utilization rate in a big cluster. It's actually really hard.

And there are wild variations in how well companies run GPUs. If you're running, if the most anybody, because the laws of physics, you know, maybe you can get two or 300,000 Blackwells coherent, we'll see. But if you have 30% uptime on that cluster and you're competing with somebody who has 90% uptime, you're not even competing. So one, there's a huge spectrum in how well people run GPUs. Two, then I think there is, you know, these AI researchers, they like to talk about taste. I find it very funny. You know, oh why do you make so much money? I have very good taste. You know what taste means is you have a good intuitive sense for the experiments to perform. And this is why you pay people a lot of money because it actually turns out that as these models get bigger, you can no longer run an experiment on a thousand GPU cluster and replicate it on 100,000 GPUs. You need to run that experiment on 50,000 GPUs and maybe it takes, you know, days. And so there's a very high opportunity cost. And so you have to have a really good team that can make the right decisions about which experiments to run on this. And then you need to do, you know, all the reinforcement learning during post-training well and the test time compute well. Complicated. It's really hard to do. And everybody thinks it's easy, but all those things, you know, I used to have this saying like, hey, I was a retail analyst long ago. Pick any vertical in America. If you can just run a thousand stores and have them clean, well lit, stocked with relevant goods at good prices and staffed by friendly employees who are not stealing from you, you're going to be a $20 billion company, a $30 billion company. Like 15 companies have been able to do that. It's really hard. And it's the same thing. Doing all of these things well is really hard. And then reasoning with this flywheel, this is beginning to create more separation. And what's even more important, every one of those labs, xAI, Gemini, OpenAI, and Anthropic, they have a more advanced checkpoint internally of the model. Checkpoint is just, you're kind of continuously working on these models and then you release kind of a checkpoint and then the reason these models get fast, the one they're using internally is better and they're using that model to train the next model. And if you do not have that latest checkpoint, you're behind. It's getting really hard to catch up. Chinese open source is a gift from God to Meta because you can use Chinese open source to try and that can be your checkpoint and you can use that as a way to kind of bootstrap this and that's what I'm sure they're trying to do and everybody else. The big problem and the big giant swing factor, I think China's made a terrible mistake with this rare earth thing. You know, I think China, because you know they have the Huawei Ascend and it's a decent chip and versus something, you know, like the deprecated Hopper, preserving something, it looks okay. So they're trying to force Chinese open source to use their Chinese chips, their domestically designed chips. The problem is Blackwell is going to come out now and the gap between these American frontier labs and Chinese open source is going to blow out because of Blackwell. And actually DeepSeek in their most recent technical paper v3.2 said like one of the reasons we struggle to compete with the American frontier labs is we don't have enough compute. That was their very politically correct, still a little bit risky way of saying, you know, because China said, 'We don't want the Blackwells, right?' And they're saying, 'Guys, that might be a big mistake. That might be a big mistake.' And so, if you just kind of play this out, these four American labs are going to start to widen their gap versus Chinese open source, which then makes it harder for anyone else to catch up because that gap is growing. So, you can't use Chinese open source to bootstrap. And then geopolitically, China thought they had the leverage. They're going to realize, oh, whoopsie daisy. We do need the Blackwells. And unfortunately, they'll probably for them, they'll probably realize that in late '26. And at that point, there's an enormous effort underway. DARPA has, there's all sorts of really cool DARPA and DoD programs to incentivize really clever technological solutions for rare earths, you know, like using enzymes to refine them or there's all sorts of really cool things happening. And then, you know, there's a lot of rare earth deposits in countries that are very friendly to America that, you know, don't mind actually refining it in the traditional way. So, I think rare earths are going to be solved way faster than anyone thinks. You know, they're obviously not that rare. They're just misnamed. They're rare because, you know, they're really messy to refine. And so geopolitically, I actually think Blackwell is pretty significant. And it's going to give America a lot of leverage as this gap widens. And then in the context of all of that, going back to the dynamics between these companies, xAI will be out with the first Blackwell model and then they'll be the first ones probably using Blackwell for inference at scale. And I think that's an important moment for them. And by the way, it is funny like, you know, if you go on OpenRouter you can just look, they have dominant share now. OpenRouter is whatever it is, it's 1% of API tokens, but it's an indication. They process 1.35 trillion tokens. Google did like eight or 900 billion. This is like whatever it is last seven days or last month, you know, Anthropic was at 700 billion. Like xAI is doing really, really well and the model is fantastic. I highly recommend it. But you'll see xAI, you know, come out with this. OpenAI will come out faster. OpenAI's issue that they're trying to solve with Stargate is because they pay a margin to people for compute and maybe the people who run their compute are not the best at running GPUs. They are a high-cost producer of tokens. And I think this kind of explains a lot of their code red recently.

You know, and the reason they have a code red is because of all these dynamics. So then they'll come out with a model but they will not have fixed their per token cost disadvantage yet relative to both xAI and Google and almost Anthropic at that point. Anthropic is a good company. You know, they're burning dramatically less cash than OpenAI and growing faster. So I think you have to give Anthropic a lot of credit and a lot of that is their relationship with Google and Amazon for the TPUs and the Trainiums. And so Anthropic has been able to benefit from the same dynamics that Google has. I think is very indicative in this great game of chess. You know, you can look at Dario, Jensen maybe have taken a few, there have been a few public comments, you know, that were, you know, made between them.

A little bit of jousting. Well, Anthropic just signed the $5 billion deal with Nvidia. That is because Dario is a smart man and he understands these dynamics about Blackwell and Rubin relative to TPU. And so Nvidia now goes from having two of the fighters, two fighters, xAI and OpenAI to three fighters. So that helps in this Nvidia versus Google battle. And then if Meta can catch up, that's really important. And so I am sure Nvidia is doing whatever they can to help Meta, you know, whatever. Like, let us, you're running those GPUs this way. Maybe we should twist the screw this way or turn the dial that way. And then it will be also if Blackwell comes back to China which it seems like it probably will happen, that will also be very good because then Chinese open source will be back.