The Supply and Demand of AI Tokens | Dylan Patel Interview

What used to matter a lot was execution, which was very, very difficult, and ideas were cheap. Now ideas are cheap and plentiful, but execution is very easy. So really, only the good ideas are the ones that can justify the spend on super cheap implementation.

Last year we thought we were heavy users of AI. Everyone's using ChatGPT, everyone's using Claude, everyone's got subscriptions. I'm providing whatever subscriptions anyone wants on the order of spend of like tens of thousands of dollars for our firm. This year the spend has just skyrocketed, and it really started in late December with Opus. That included Doug, who's president, Doug Olaflin. He's very much leading the charge in the sense of non-technical people using AI for coding. He's basically led the whole firm slowly over time. Obviously the engineers were using it anyways, but spend in January just started to inflect and rocket. We signed an enterprise contract with Anthropic, and it's gone to the point where now, I think when I last talked to you it was a 5 million spend rate. It's actually 7 million spend right now.

A lot of that is just the usage. People who have never coded before are using Claude Code and spending thousands of dollars sometimes a day. Across the firm, we're spending $7 million a year now on Claude Code at the current rate, versus our salary expense being in the neighborhood of $25 million. So we're north of 25% of spend on Claude Code as a percentage of salary. And if this trajectory continues, then we'll spend more than 100% by the end of the year, which is a bit terrifying. Thankfully, I don't have to decide between people and AI because our company's growing so fast. It's more so like, okay, well, I don't have to hire nearly as fast and I can spend a lot more on AI and it works and we just grow faster. But I think other folks will start to reckon with the fact that, huh, if this person can do the work of five to 10 to 15 people using Claude Code, then all of a sudden I should probably cut people. But right now, I think the use cases are so broad.

For example, one thing is we have a reverse engineering lab in Oregon that we've been building for a year and a half. We have a bunch of fancy microscopes, scanning electron microscopes. The whole purpose of this is you reverse engineer chips. You get the architecture out of it, you get the materials that they're using to manufacture, and this is some of the data we sell. This is a very slow process of analyzing that data. Instead, one person on the team, they've been able to spend with a couple thousand dollars of Claude tokens. They've been able to create this application that is GPU accelerated, runs on a server that we have at Coreweave, and anytime we send it an image, it's able to take the picture of the chip and overlay where every single material is. Oh, this part is copper. Oh, this part of the gate is tantalum. This part of the gate is germanium. This part of the gate is cobalt. And so you can do a finite element analysis of the entire stackup of the chip very, very quickly with a visual dashboard GUI. It took a few thousand dollars with Claude. The person previously worked at Intel and he said that was an entire team's job to build that and maintain that. Now rack that up across the entire firm, it's insane.

Another example that I think is super fun is Malcolm, who's an economist at a major bank before. Their economist department was like 100 or 200 people. What he built was the most incredible thing ever. He piped all of this different data, FRED data and all these other data, employment reports and all these other things from various APIs. We signed a couple contracts with folks to get API access to data. Pulled it all in, started running regression, started looking at the impact of various economic revolutions on the economy from a deflationary inflationary perspective. The Bureau of Labor Statistics has this entire set of like 2,000 tasks. And so he did that with AI, which ones can be done by AI, which ones cannot, and grading them across a rubric. About 3% are doable now with AI. And so he's created this metric so that you can measure things that can be done by AI, what the massive deflationary aspect of it is, what the cost of being able to do those with AI is. Output can go up, but because cost falls so much, actually GDP theoretically shrinks. So he created this whole analysis and a brand new benchmark of language models, a set of evals across 2,000 different evals.

This is all by himself. Yeah. And he's like, dude, this would have taken a team of 200 economists a year. He's just completely cracked out on Claude. He's like, everything has changed.

Ultimately, I'm in the information business, right? We sell analysis, we do consulting, we create data sets. I don't see why this wouldn't be completely commoditized on a pretty rapid basis if I'm not constantly improving. My first product that I was selling as a data set, there's more people trying to do it now. We've made it constantly better and better and more detailed and so therefore it sells. But the way we were doing it in 2023 is not terribly different than what everyone else is doing now. If I don't move up the bar, then I will be commoditized. If I don't move fast enough, I will also lose my edge. So the question is, yes, AI commoditizes things just like it commoditizes software. Those who can move fast and keep control of their customers and keep providing them an awesome service and keep improving the service won't shrink. They'll grow. They'll grow faster. Those who are incumbent and not doing anything, they're going to lose. And so, it's a bit of an existential like if I don't adopt AI, someone else will and they will beat me.

Another easy example is the energy space. We've had a few energy analysts for like a year now. We've been trying to build out this energy model. It's very complex. Energy data services market is something like $900 million. So obviously a huge market for me to try and break into, but we really hadn't broken into the energy data services business despite a year of having multiple people on the team. Then Claude Code psychosis hits one of the people who leads the data center energy and industrial sort of business at SemiAnalysis, Jeremy, and now all of a sudden in 3 weeks, he spent a lot, he was spending like $6,000 a day. It was an insane amount, but he scraped every single power plant in the US, every single transmission line above a certain voltage, and created this entire mapping of the entire US grid as well as a lot of demand sources all from various public sources of data. And we built this dashboard where you can view and check, you can see all the micro regions of the US where there's power deficits and surpluses. All of these details built in a handful of weeks. We started showing some of our customers who buy our data center data set but are energy traders. We showed some of them and they're like, wow, how long did this take you? This is really good. This is better than XYZ company. And then we dig deeper. XYZ company has 100 people and have been working on this for a decade now. Obviously our thing is not as robust, but in some ways it is better. I'm going to commoditize these energy data services companies. Who's going to come commoditize me if I don't move faster? And so the question from a business owner's perspective is, yeah, I'm spending a lot, but what does that spend getting me? Is it getting more revenue? Yeah.

First of all, any information services business, obviously I don't generate as much value as my customer does from such information. Because if I sell you information for a dollar, you're only buying it for a dollar because you know that information helps you make a decision that lets you make more than $1. And so therefore, you have made more money off of me than I did from the information myself. An investment fund, these investment funds all have their own information services, especially like the Jane Streets of the world and the Citadels. They're really detailed on their data. And yet, these sort of folks also purchase data from us and continue to do so and continue to grow with us because I think there's just some it factor. We move faster, we're more nimble, we're a smaller team that's focused on just one specific thing, AI infrastructure and the huge revolution that causes in AI and tokenomics and all these things. And we sort of really see where it's headed and so we're moving faster and building faster. I think investment professionals just, yes, they'll try and build some of the stuff we do and more likely they'll just buy the data from us and it's cheaper for them to buy the data from us and then build on top of it than it is to build it themselves. But ultimately some may try.

If we take a step back and look at the macro lens, Anthropic has gone from 9 billion revenue to what they're at, 35-40 billion now. Probably by the time this airs 40-45 billion ARR. Their compute has not grown to the same degree. And if you do the calculations and you assume they didn't decrease their research and development compute, they clearly didn't. Their release, they have Mythos, they have Opus 4.7. So they clearly didn't decrease their research compute spend. So ultimately what they've done, even if you assume all incremental compute they've gotten has gone towards inference, their margins are at a floor of 72%. In reality, some of that incremental compute they've got probably went to research and development. It may be higher than 72% gross margins. To be clear, at the start of the year, there was a leak by someone from their funding round docs. Someone leaked it, 30-something percent gross margins. Where on earth does a business like this grow margins like that? And it's in principle, right? Their demand is so high. They're able to cut back on usage limits, rate limits, all these things. What really matters is having an Anthropic rep and having an enterprise contract with them and getting the rate limit increases that you need because otherwise tokens are ultimately super, super in demand. Whoever can pay for them, Anthropic has the same problem. It's just the reality of how capitalism works. Yes, people are sending them $40 billion ARR in tokens, but those tokens are generating way more than $40 billion in value. Various businesses will have different value generation per token. But as we get more and more intelligent, what really matters is access to these most intelligent tokens and leveraging them at things. You as a person deciding what is the best way to leverage these tokens to grow business and generate value, because a lot of folks will want tokens and generate tokens. But the shitty SaaS startup in SF who is using Claude to generate their software product is not necessarily actually creating a ton of value and therefore they're going to get priced out of tokens soon enough.

Without a doubt. One of my funniest memories in the past month and a half is myself and a buddy of mine, Leopold, being on our knees in front of an Anthropic co-founder begging him for access to Mythos and then pretending it doesn't exist because we knew it existed. We were like, 'Please give us access.' And he's like, 'I don't know what you're talking about.'

It was rumored in the Bay Area. Everyone, we sort of knew it was supposed to be really good, but if you just look at the benchmarks, and obviously benchmarks change over time, Mythos is potentially the biggest step up in model capabilities in like 2 years. I think that's really, really an important detail that it's so good that they don't want to release it even though they already announced the price to their people that they did a selective release for cyber for, and it's like five or 10x the token cost. They just don't want to release it because they're worried about the impact on the world and they're releasing a shitty worse version of Opus 4.7 to us and they explicitly said in the model card, hey, we actually preferentially made it worse at cyber. I don't know if you read that. Whoever you are, if you have enough capital, you should get a freaking enterprise Anthropic subscription where you pay per token, not with these subscriptions, because then you won't get rate limited much. And then you need to figure out how to leverage those tokens to the highest value task and make money off of it because ultimately what you're doing, maybe a year from now or two years from now, the business is actually just arbitraging tokens, right? The tokens are amazing, but let's figure out what direction to point them in and then three or four years from now the model will know what to do with the tokens and how to make the most value.

You can look at this retroactively. Pick any benchmark. The cost to hit a certain capability tier used to cost X and now it cost 1/100th or 1/1,000th of that. DeepSeek, for example, on GPT-4 was 1/600th the cost. And since then, the costs have fallen further for GPT-4 class models. Of course, no one gives a crap about GPT-4 class models. They want the frontier because the frontier lets them create the economically valuable things. But GPT-4 class models can still be used in some stuff and so people are using them in some tiny use cases. It's just the costs have fallen so fast. It's not really what's driving the demand. What's driving the demand is all these new use cases. Current 4.6 Opus or 4.7 Opus tier models a year from now, my spend for the same exact quality of the model would probably be like 70k. I bet you it'll be 100 times cheaper. Irrelevant because I'm going to be using a way, way, way better model which can do way, way better things. Anthropic Mythos is more expensive as a model but it spends a lot less tokens to do the thing and therefore it is actually cheaper in most tasks than 4.6 Opus because it's just way more efficient even though each individual token is smarter.

Anthropic's whole goal in 2025 was, and even a lot of 2024, they're like, hey, by the end of 2025 we need an L4 software engineer in our model. And they by and large achieved that with 4.6 Opus. What they didn't say is that, and if you look at Mythos and if you compare the benchmarks, it's like an L6 engineer. So L4 is like pretty new. L6 is like quite well experienced. I think Anthropic said that the model internally was available in February. So in two months they've gone from L4 engineer to L6 engineer. What's next? When you think about the model progress, it's only accelerated. Anthropic's release cadence has compressed. OpenAI's release cadence has compressed. Why? Because these models generally, to make a better model you need a few things, right? You need amazing compute. Compute is very expensive and it has a time scale that we track and it's growing but it's sort of set in stone for the next short term. It's kind of set in stone what you've already signed. There will be delays and shifts and somehow you can find a little more but it's generally pretty set in stone. There's amazing researchers that people are paying tens of millions of dollars for. And then lastly there's implementation. Historically has been very difficult. If I have an idea now I have to implement it. Implementing is hard. Now ideas are there. Implementation is very easy. It's expensive but it's very easy. So how does one decide what ideas to implement? And it turns out if your implementation is just so much easier now you can just implement more ideas and move on the treadmill faster and faster and faster. Whether that is AI model research and so now your model release cadence is shrunk down to 2 months from where it was 6 months before, or hey, I want to take every power plant in the US and every transmission line and model it and run regressions and see the micro supply and demand. I can also do that. The idea is cheap. Which idea makes sense? Which idea is worth the capital that you have to spend on the tokens because the implementation is there. That's the key learning. And if implementation costs continue to tank, which they are, we don't even have Mythos yet. It's only been a handful of hours since Opus 4.7 launched but my team is pretty excited about it internally. What now comes to the world, it's a complete reordering of how economies work. What used to matter a lot was execution, which was very, very difficult, and ideas were cheap. Now ideas are cheap and plentiful, but execution is very easy. So really only the good ideas are the ones that can justify the spend on super cheap implementation.

Uncertainty is there. But I do think that causes some fear in terms of how does society reform itself? How does one exist in a world where actually your ability to implement something is not actually that important. Your ability to choose the correct idea for AI to implement and then your ability to sell that idea or sell what the AI has implemented is what matters. Your ability to garner capital towards that is what matters. And going back to the point of it's very important to have the newest model always. Who's going to have access to the newest model? Anthropic's project. I know it's not called Earwig, but I troll Anthropic people by calling it Earwig. Glasswing, Anthropic, Earwig, you know, where they only release Mythos to certain companies for cyber. That's just going to be something that continues. Models will have less broad and less broad deployment. I know OpenAI and Anthropic and all these people are like, we want to have great AI for everyone. AI is very expensive. Who's going to pay for the trillion dollars of infrastructure? People who have money and can build useful things with AI. And then you don't want people to distill your models. So you don't release them broadly. You release them to a fewer and fewer set of customers. Those customers are also now wrestling over the tokens unless Anthropic jacks them. They could double their pricing on Opus and I would continue to pay and I bet most users would continue to pay.

I don't have Mythos. You know who has Mythos? Top freaking banks. Now they're only using it for cyber security. But at some point I can envision a world where maybe I, because I have an enterprise Anthropic contract and because Anthropic people kind of like me, they're willing to give us slightly earlier access or slightly higher rate limits or something for a model. I hope that's what happens. And then my competitor, whoever that is, doesn't have that and I'm able to crush them. There are people who are like Ken Griffin of Citadel is super well-connected and super rich and he just signs a deal with OpenAI or Anthropic that's like, 'Yeah, I'm going to get access to your models. And I'll buy the first $10 billion worth of tokens each year. So, whenever you release the model, I'll spend the first 10 billion tokens and then everyone else can get the model after that.' And it's like, okay, well, now what does that do? Well, now he's going to crush everyone in the market. That's just an example. Could be cyber like Anthropic is worried about, oh, now I can hack people. Could be information services business like myself where I crush someone else. I think it's such a broad base. We don't know what these models can do. Anthropic doesn't know what these models can do. No one knows what these models can do. It's up to the end user to figure out where they can leverage the tokens to see what they can build and imagine, which is tremendously productive and uplifting for humanity. But then what happens to the concentration of resources and usage of it?

So there's this concept of software only singularity, which is that the world has AI singularity, but only in software. And now what about the rest of the world? Vast majority of the world is physical. You can see the world orient around hardware, not software. That's actually why I think software only singularity is just a blip and not like a, you know, we do get everything else because once software is super easy, what makes robots really hard is like programming microcontrollers and actuators and controlling all this stuff is very difficult. Right now the interesting thing about AI models is they're actually really inefficient in learning, it's just we're able to give them so much data that they're able to learn and surpass us in certain ways. Robots currently, the robot models, vision language action models, which is very popular right now, is probably not going to be the thing that ultimately scales beyond.

inefficient in data and we can't scale the data for them fast enough. There is going to be some way to large scale pre-train robot models where just like humans see all this data throughout their lives. And what's interesting is humans, the reason why we're so good is we're sample efficient. One example, two example, we're good. And so applying that to robotics. So once you have this software only singularity implementation is super cheap. Anyone can start to build these models that now robots are actually useful. And so I think in the next six to 18 months we'll start seeing real breakthroughs in robotics that enable few shot learning, i.e. there's a pre-trained robot model and now there's a robot that you have hired or bought or whatever. You show it a few examples and it's able to do it. You tell it to stack these two things or you tell it, hey, this can actually balance perfectly, you know, and it starts doing these things.

No, trust me, I've spilled many of times. So, I think robots will get few shot learning right now. Now, you know, there's a lot of companies doing robots for like, you know, advertisement or robots for like simple stuff like that, but it'll be like, oh, folding clothes, but it's going to get really niche like robots just for cleaning chalkboards. And it's a rental service or, you know, it'll be a model package that you download onto your standard robot that then does that, right? And you pay for that. And anyways, there will be a huge explosion in physical good acceleration and deflationary effects there. But and so that's ultimately going to keep token demand going crazy. I don't think token demand slows down personally.

So Methos is a materially larger model than prior models and so yes it is a much larger model. Now whether or not it's what chip it's trained on is not really relevant. It's the scale and obviously, you know, to a 100,000 Blackwells is equivalent to hundreds of thousands of prior generation chips. TPUs and Trainium have their different release cadence. So it's not exactly like mirrored one to one. But ultimately yes Methos is a significantly larger model. It's proof that the scaling laws still work. Everything about it shows the trend line continues of models. More compute into model makes model better. And along the whole way it's not just more compute into model makes model better. Along the whole way we're also getting these compute efficiency wins which are, you know, as all this research compute that the labs are spending is actually turning into if I want X capability tier model every six months that cost or every two months that cost is dramatically decreasing but then if I scale it up massively I get a humongous capability jump as well. And so yes it's proof that this is still happening. Google and Anthropic are not heavy heavy users of GPUs on the training side but OpenAI, they'll start having their new class of models. I think they're taking a more sensible principled approach to scaling in small steps. Anthropic really went for a huge jump. We'll see better and better models throughout the year and the release cadence is only going to get faster.

So, this is the interesting thing. Everyone's like, okay, so Anthropic's just won, right? You know, they had Methos in February. They never even released it because they didn't feel the need to. They're already sold out. Their revenue is already adding $10 billion a month. And then you've got Opus 4.7 today all before OpenAI's, you know, alleged Spud release which, you know, media such as The Information and others have posted about. So clearly Anthropic is in the lead right and OpenAI is cooked. What's interesting is because Anthropic has such bounds on compute and they can only grow it so fast and sort of to the point of, you know, Dario used to gloat about how OpenAI was being too aggressive on compute and Anthropic was more sensible in their scaling and now Anthropic is like, we should have, I wish we had a lot more compute. OpenAI is able to pay the bills perfectly fine. In fact, they've raised a ton of money to get incremental compute in addition to the irresponsible levels of compute that they were buying from Oracle and CoreWeave and SoftBank and all these people and Microsoft, you know, such as Trainium. Now they're getting Trainium as well from Amazon. So they've done this like insane thing on compute and they need, they also know they need more. But what's interesting is if you were to say Opus 4.6, you know, let's ignore models getting better over time. Let's just take diffusion of this technology. You and I may get jump on the model immediately day one, but other businesses take time and it takes time for people to learn and the spark of oh, Claude psychosis moment doesn't hit everyone at the same time. And so by the end of the year, let's say a 4.6 Opus tier model the economy would spend $100 billion on. I don't think that's unreasonable. It's spending $40 billion right now.