Here in this, I must say, very, very nice surroundings, very Swiss, we are going to do something: we are going to find clarity amid chaos, as it says up there. Now, I don't know how long you have all been in Davos, but I think finding clarity amidst what's going on in the world right now is a tall order, but it is what we try and do at The Economist. We try and join the dots and we try and find clarity. And we are lucky in the next 90 minutes to be able to have the chance to hear from some of the smartest thinkers in the world on some of the most important issues, but particularly in this first conversation, Dario Amodei, who I am really not exaggerating when I say that you are one of a very small handful of people who is actually going to determine when we get to full AGI, super intelligence, how we get there, what it looks like. So the future that we are all going to live in is disproportionately going to be shaped by this man. So that's why he's here, that's why I'm really excited to have this conversation. And I wanted to start, but before I do that, I must thank Zurich, of course, for making this possible. You have fantastic surroundings, thank you, thank you for having us, we're delighted to be here. But Dario, two things have happened in the last few weeks and months that I think collectively might have a bigger impact on the development of this technology even than ChatGPT. And remember that moment two years ago when everyone was amazed? Yet by you, but by some of your competitors, and the other is kind of seemingly from nowhere, the development of really good cheap models from China. DeepSeek, if you haven't heard of it, is an incredibly powerful model that has kind of come from nowhere. And I think these two events together are going to take this whole technology in a slightly different direction than we might have thought six months ago. So I wanted to start by exploring those. And firstly, let's start with the US cutting edge models. How big a deal is it that we now have these reasoning models which learn by thinking? They're kind of like what you tell your kids: spend a little time on it and you'll get a better answer. Isn't that sort of what they're doing, and what can they do, and what will we see?
For the longest time, most of the training that was done in models was what was called pre-training. So this was you train a model on a bunch of data on the internet, and that, plus a tiny bit of reinforcement learning or reinforcement learning from human feedback, is what language models were until recently. There was another paradigm, if we think back to AlphaGo in 2016, done by DeepMind, believe Demis Hassabis is here as well, was one of the pioneers in that area. So this other paradigm of train via reinforcement learning, learn via trial and error, has also been developed and has been around for a long time. What has happened is that early RL-heavy models were being trained on a narrow set of tasks. What is going to happen is that first we pre-train the model and it learns about the world in general, and then we do large-scale reinforcement learning where the model does very large-scale tasks. It learns to do coding tasks, it learns to do math, it learns to solve scientific problems, it learns to solve puzzles, it even learns to do things like comedy routines at some point. And so we are thinking about over the next year or two how to develop models to do that task much better than has been possible before. We have seen with some of the early models that they are getting as good as humans, better than humans across a very wide range of thought tasks. So that's the excitement about it. But the interesting thing I think is also how this changes the economics of these models. The old style models, you spent a ton of money training them, a ton of compute, a ton of capital to train them, but then once they were trained, it's pretty cheap to use them. Now, because they're thinking as they're doing, you're going to use compute while they're actually working. That means they're more expensive to run. What does that mean for you? Because it's going to cost you more for your models to be used.
Yeah, so I actually on the cost side, I actually don't think of it as different. I think of it as you're renting someone's brain, you're renting a virtual brain for a certain amount of time. Now what's happening is all that's changed is the questions we're asking the virtual brain are different. So now sometimes you ask the virtual brain a question instead of 'what sports team won this contest in some year,' you're asking 'hey, can you prove this mathematical theorem? Can you implement this coding feature?' And of course, just like a human, the model says 'okay, it's going to take me 10 or 15 minutes to do that.' So you're paying for 10 or 15 minutes of its thinking time instead of one or two minutes. Now I do think of it more like labor, more like effort, than like software. As these models get smarter, they're going to do things like, and I've talked about in 2025 we're going to release probably in pieces parts of a vision we call a virtual collaborator. So it'll be a model that you give a task, it works on that task for several hours, maybe it checks in with you every once in a while. And that could easily generate thousands of dollars of value, tens of thousands of dollars of value. If I give this model to a medical chemist, it finds a better drug candidate than it would have found otherwise, or it reduces the number of animal trials or phase one trials it has to run. It's actually power users, consumers which Claude is actually very popular with, may in some ways dominate. They are the main source of revenue because they're paying thousands, tens of thousands of dollars a year because the quality of the model thinking is worth that. The model is being paid like a coworker, at least like an assistant.
But let me continue your analogy because it's a very interesting one, that what you're paying for is the labor of the AI. Right along comes at the same time a cheap Chinese version that's pretty much nearly as good. Frankly, DeepSeek is not far off Claude 3.5 Sonnet. In fact, as good in some ways.
It's not currently as good, but we're actually living in an interesting moment in time, and it actually relates to the first question you asked, which is this switch to reasoning models. So if you imagine there's a lot of compute, right? We're increasing the amount of compute at all times. In the past, 99.9% of the compute went into one kind of training, which is pre-training. We're now executing a switch over where we figured out how to put small amounts of compute into this second stage, this reinforcement learning stage. And because none of it was being done before, there are big paradigm switches. The landscape scrambles a bit, but then it kind of reestablishes itself. So later this year, we and probably others will have hundreds of thousands of chips. I think there will be millions of chips from various companies in 2026 and more in 2027. DeepSeek, the Chinese company, it's marketed as cheap, but it's at least been reported that DeepSeek has 50,000 H100s, which for reference is about half of what Elon's Colossus cluster has. I won't say exactly how many Anthropic has right now. So I actually think we have an opportunity. As we go to hundreds of thousands and millions of chips, there's two possible futures. In one of those futures, the US and its allies are able to provision that many chips fast enough, and because of the export controls on chips to China and because Chinese Huawei chips are inferior, China cannot get to that scale. There's another world where both sides get to that scale.
Let's dwell on that a little bit more because I think given the geopolitical backdrop, this is really interesting. For four years now, I've heard from American officials that the single most important thing is that the US retains its lead in AI. And US policy, particularly export controls, is designed to ensure that. But DeepSeek is not far off Claude 3.5 Sonnet. Were you expecting it to be that good?
Honestly, yeah. I think it's an expected consequence of the switch over in paradigm. I think DeepSeek in particular has higher quality than some of the other companies in China. We were aware of DeepSeek for quite a while and kind of identified it as more likely to succeed than some of the others, like Alibaba. So it's a particularly talented team, but the general dynamic that we've seen is to be expected, particularly in the tens of thousands of chips. Smuggling is very possible. I obviously can't speak for either the last administration or the current one, but that's really hard to smuggle and that's really hard to replace with Huawei chips. So there's this bifurcation in front of us. We can have a big lead, or as you're suggesting, things can be parity. And those are two very different futures for the world.
I guess my question is, can you really have such a big lead when maybe you expected them to be this good, but I think a lot of people didn't? A lot of people even in your industry have been pretty blindsided by how good these new Chinese models are. And it leads me to wonder whether actually the half-life of export controls is not a lot shorter than you think. Even if you have export controls, the Chinese innovate around them and they come up with other ways.
I believe this is possible. First of all, this may or may not be something that works, this may or may not be something that's well implemented. But I do think that if it is well implemented, it still has a fairly good chance of working, and it is something that we should try because of this critical period. I think I maybe didn't come earlier in this interview, but in other interviews I've said 2026, 2027 is the critical window. And if you're ahead then, the models start getting better than humans at everything, including AI design, including using AI to make better AI, including using AI to make all kinds of intelligence and defense. Half of 2025, DeepSeek's cluster is the same order of magnitude as the cluster of the frontier companies in the US. I don't think that could ever have been prevented.
Interesting. So 2026, 2027 is when you effectively get to AGI across the board, and it's the threshold moment. Whoever's ahead then is ahead forever. Is that what you're saying?
Potentially. I mean, we don't know these things, but there's a risk of this happening at a particularly uncertain time geopolitically. But we'll get to that. I've heard those same years floated.
No, I'm aware of how frightening this is. So you've made the case for export controls. What else does the US need to do to ensure that it is ahead?
Enough compute. There's another issue that's kind of the mirror image of this, which is, for all our concern about authoritarians, and I'm very worried about it because when AI is so powerful, there's the opportunity for authoritarians to abuse them. Many limits and constraints on the power of authoritarians comes from the fact that they need the consent of humans, of followers, to do what they want. If we start to automate this, then very dark things become possible. So that's very important. But at the same time, if we have these models that are better than humans at everything, that are like a country of geniuses in the data center, we also need to worry about the dangers of the models that we ourselves are building. Are they dangerous in an autonomous sense? The right approach, because we don't know for sure how to run these tests, and the tests have focused on national security risks like bioweapon risks, cyber risks, emergent autonomy of the AI systems. And these tests have actually been helpful to us. In some cases, they've even helped us do a better job of deploying our commercial product because we understand better what the models are capable of, for example in areas like cyber. So we've been an advocate of keeping this kind of testing around. And actually, we found some bipartisan support for this. For example, when we talk in Congress, the idea of measuring models for national security risks makes a lot of sense to people. I think people have the impression that this kind of testing, maybe the name Safety Institute, is correct.
One thing, what impact does that have? And isn't there some element of the AISI being separate from the executive order? The executive order focused on reporting requirements for models that were above a certain size. I think those reporting requirements were good, but they were really small potatoes. So there's a lot of discussion of the executive order, it has this kind of totemic significance as something Biden did and Trump is ripping up, but actually there wasn't that much to it. I don't think it's actually the important thing. And generally, I kind of advise that we focus on the important things and not the things that gather this polarized, totemic significance.
We're going to come to safety a bit later because this is something you're known for. But I want to talk about the commercial side. You mentioned coding, which is the most obvious area where AI diffuses throughout economies. What are you seeing there?
Without a doubt, the fastest area of adoption has been coding. Multiple hundreds of millions of dollars of revenue have materialized overnight over the last three months. Anthropic's revenue in general has grown from roughly 100 million to roughly a billion over the last 12 months. Coding is not the majority of it, but it's the largest single area. There's many companies: there's Cursor, there's Codium, GitHub Copilot, which is owned by Microsoft, and several others. So this is happening very fast. Coders across the industry are using it, and the quality of coding is only going to get better. So I think that explosion is only going to inflect upwards. And with things like the virtual collaborator, it's going to extend to other areas. I think the growth we're seeing in coding, that growth is going to happen in every area. And that's the way that this industry is going to get relatively soon to 10 billion, 100 billion dollar revenue amounts. You have to imagine if you take seriously the country of geniuses in a data center, better than humans at everything in two or three years, what are the commercial consequences of that? It doesn't make any sense unless the revenue amounts get up, and I think they will for multiple companies within 2 or 3 years.
I'm not going to ask you to pick a stock, but I don't think it's a secret.
No, no, no intended to. But it doesn't take a genius to see that probably Nvidia has a bright future. The only way in which they might not have a bright future is that there are other chips being developed by other players.
Tell me about what kinds of companies are going to reinvent themselves fast enough to be able to take advantage of all of this. Because I was in a really interesting conversation yesterday, and one argument is big enterprises have the capital, the scale, they can kind of... what the firm is, and so startups will be much faster. What do you think?
We are seeing the following dynamics. Spending from startups has grown much faster than spending from enterprises. There are startups that spend 50 million annualized or even more. We've seen that it's very hard for enterprises to get to that level that quickly. However, the potential for enterprises to spend, given the value of the technology, is much higher. So you have this smaller pool that's spiking very quickly and this larger pool that's starting to spike but is doing so slowly. I think the enterprises will ultimately overtake the startups. Where across industries, just all the incumbents are knocked down one by one by these innovative startups that can adopt AI much faster. Either is possible. They kind of both lead to the same world, but they look different while they're happening.
And what kind of time frame do you expect this in? I mean, if you have the geniuses in a data center by 26, 27, over what kind of time horizon is every single industry completely overhauled by this?
