Anthropic CEO: AI Is Not Conscious , It's Much WORSE Than That - Dario Amodei

Yeah, I think there are definitely going to be things that go wrong, particularly if we go quickly. But that would introduce all these new alignment problems. So, I'm actually a bit

Not impossible to stop the end of the world, but impossible to stop sort of punctuated terrorist

It's hard to predict these things because the technology is moving so fast and kind of moves so unevenly. So, at least a couple principles for figuring out and then I'll give my guesses at like what I think will be disrupted. So, you know, one thing is I think the technology itself and its capabilities will be ahead of the actual job disruption. Two things have to happen for jobs to be disrupted or for productivity to occur cause sometimes those two things are linked. One is the technology has to be capable of doing it. And the second is there's this messy thing of it actually has to be applied within a large bank or a large company or, you know, think about customer service or something, right? And you know, in theory AI customer service agents can be much better than human customer service agents. They're more patient. They know more. They handle things in a more uniform way. But the actual logistics and the actual process of making that substitution, that takes some time. So, I'm very bullish about the direction of the AI itself. Like, you know, I think we might have that country of geniuses in a data center in 1 or 2 years. And maybe it'll be five, but, you know, it could happen very fast. But I think the diffusion to the economy is going to be a little slower. And that diffusion creates some unpredictability. So, an example of this is, you know, and we've seen within Anthropic, the models writing code has gone very fast. I don't think it's because the models are inherently better at code. I think it's because developers are used to fast technological change and they adopt things quickly. And they're very socially adjacent to the AI world, so they pay attention to what's happening in it. If, you know, if you do customer service or banking or manufacturing, the distance is a little greater. And so, I think 6 months ago I would have said the first thing to be disrupted is, you know, these kind of entry-level white-collar jobs like data entry or document review for law or kind of, you know, the things you would give to a first-year at a financial industry company where you're analyzing documents. And I still think those are going pretty fast. But I actually think software might go even faster because of the reasons that I gave, where I don't think we're that far from the models being able to do a lot of it end-to-end. And what we're going to see is first the model only does a piece of what the human software engineer does, and that increases their productivity. Then even when the models do everything that human software engineers used to do, the human software engineers kind of take a step up, and you know, kind of they act as managers and supervise the systems. And so

Yeah, this is like centaur chess. So after I think Garry Kasparov was beaten by Deep Blue, there was an era that I think for chess was 15 or 20 years long where a human checking the output of the AI playing chess was able to defeat any human or any AI system alone. That era at some point ended.

Yep. And so my worry, of course, is about that last phase. So I think we're already in our centaur phase for software. And I think during that centaur phase, if anything, the demand for software engineers may go up, but the period may be very brief. And so, you know, I have this concern for entry-level white-collar work, for software engineering work. It's just going to be a big disruption. I think my worry is just that it's all happening so fast. Right? People talk about previous disruptions, right? You know, they say, 'Oh yeah, well, you know, people used to be farmers, then we all worked in industry, then we all did knowledge work.' Yeah, people adapted. That happened over centuries or decades. This is happening over low single-digit numbers of years. And maybe that's my concern there. How do we get people to adapt fast enough?

I think it's going to be pretty heterogeneous. There may be areas where a human touch kind of for its own sake is particularly important.

Of radiology. That might be true. You know, it's like you go in and you're getting cancer diagnosed. You might not want, you know, Hal from 2001 to be the one to diagnose your cancer. I just not maybe that's just maybe not a human way of doing things. But there are other areas where you might think human touch is important. Like if we look at customer service. Actually, customer service is a terrible job and the humans who do customer service are like they lose their patience a lot. And it turns out customers don't much like talking to them because it's a pretty robotic interaction, honestly. And I think the observation that many people have had is like maybe actually it'd be better for all concerned if this job were done by machines. There are places where a human touch is important, there are places where it's not. And then there are also places where the job itself doesn't really involve human touch, you know, assessing the financial prospects of companies or writing code or so forth and so on.

That is what I had in mind when I talked about entry-level white-collar labor and the bloodbath headlines of, 'Oh my god, are the entry-level pipelines going to dry up? And then how do we get to the level of the senior partners?' And I think this is actually a good illustration because, particularly if you froze the quality of the technology in place, there are over time ways to adapt to this, right? Maybe we just need more lawyers who spend their time talking to clients, right? Maybe lawyers become more like sales people or consultants who explain what goes on in the contracts written by AI, help people come to agreement. Maybe you lean into the human side of it. If we had enough time, that would happen. But reshaping industries like that takes years or decades. Whereas, these economic forces driven by AI are going to happen very quickly. And it's not just happening in law, the same thing is happening in consulting and finance and medicine and coding. And so, you have this it becomes a macroeconomic phenomenon, not something just happening in one industry, and it's all happening very fast. And so, my worry here is that the normal adaptive mechanisms will be overwhelmed.

I definitely am in favor of like trying to work out restraints here, right? Trying to take some of the worst applications of the technology, which could be some versions of these drones, which could be used to create these terrifying biological weapons. There is some precedent for the worst abuses being curbed. Often because they're horrifying while at the same time they provide limited strategic advantage. I'm all in favor of that. I'm at the same time, a little concerned and a little skeptical that when things directly provide as much power as possible, it's hard to get out of the game, given what's at stake, right? It's hard to fully disarm. If we go back to the Cold War, we were able to reduce the number of missiles that both sides had, but we were not able to entirely forsake nuclear weapons. And I would guess that we would be in this world again. We can hope for a better one and I'll certainly advocate for that.

I think there's a few things and I just want to caveat, I'm no international politics expert here. This is a weird world of intersection of a new technology with geopolitics. So all of this is very

Yeah, yeah, I'm learning as much as I can about it. I just we should all have humility here. I think there's a failure mode where you read a book and go around like the world's greatest expert in national security. I'm trying to learn what I can.

It's more annoying when tech people do it. I don't know. Let's look at something like the biological weapons convention. Biological weapons, they're horrifying. Everyone hates them. We were able to sign the biological weapons convention. The US genuinely stopped developing them. It's somewhat more unclear with the Soviet Union. But biological weapons provide some advantage, but it's not like they're the difference between winning and losing. And because they were so horrifying, we were kind of able to give them up. Having 12,000 nuclear weapons versus 5,000 nuclear weapons, again, you can kill more people on the other side if you have more of these, but we were able to be reasonable and say we should have less of them. But if you're like, 'Okay, we're going to completely disarm nuclear, we have to trust the other side.' I don't think we ever got to that and that's just very hard. Unless you had really reliable verification. So, I would guess we'll end up kind of in the same world with AI that there are some kinds of restraint that are going to be possible, but there are some aspects that are so central to the competition that it will be hard to restrain them. That democracies will make a trade-off that they will be willing to restrain themselves more than authoritarian countries, but will not restrain themselves fully. And the only world in which I can see full restraint is one in which some kind of truly reliable verification is possible. That would be my guess and my analysis.

Have to do it just as the legal profession or software engineers has to update in a rapid amount of time, politics has to update in a rapid amount of time. That seems hard. That's the dilemma. That's the dilemma of all of this.

Yeah, you know, I think there are definitely going to be things that go wrong, particularly if we go quickly.

Yeah, this is one of these really hard to answer questions, right? Yes, but it's very important. As every question you've asked me before this is as devilish a sociotechnical problem as it had been, we at least understand the factual basis of how to answer these questions. This is something rather different. We've taken a generally precautionary approach here. We don't know if the models are conscious. We're not even sure that we know what it would mean for a model to be conscious or whether a model can be conscious, but we're open to the idea that it could be, and so we've taken certain measures to make sure that if we hypothesize that the models did have some morally relevant experience, I don't know if I want to use the word conscious, that they have a good experience. So, the first thing we did, I think this was 6 months ago or so, is we gave the models basically an I quit this job button, where they can just press the I quit this job button and then they have to stop doing whatever the task is. They very infrequently press that button. I think it's usually around sorting through child sexualization material or discussing something with a lot of gore, blood and guts, and similar to humans, the models will just say, 'Nah, I don't want to do this.' Happens very rarely. We're putting a lot of work into this field called interpretability, which is looking inside the brains of the models to try to understand what they're thinking. And you find things that are evocative, where there are activations that light up in the models that we see as being associated with the concept of anxiety. When characters experience anxiety in the text, and then when the model itself is in the situation that a human might associate with anxiety, that same anxiety neuron shows up. Now, does that mean the model is experiencing anxiety? That doesn't prove that at all, but it does indicate it, I think, to the user, right? And