On any journey using Uber, it helps to know you're getting into the right car. PIN verification adds an extra step to make sure your ride is your ride. Before the trip begins, your app gives you a unique PIN. Just tell it to your driver, and they'll enter it in their app before the ride can start.
That way, you know you're in the right car, taking the right trip, and your driver knows you're the right passenger. Make sure your ride is your ride with PIN verification from Uber. One more way Uber is putting safety at every turn. Learn more on the Uber app.
Ryan Reynolds here for Mint Mobile with your summer price forecast. Now, unfortunately, we're seeing rising costs across the country with possibility that big wireless hates you 100%. Now, over here at Mint Mobile, we're seeing sunny skies and dropping prices every plan to just $15 a month. Give it a try at mintmobile.com/switch.
Upfront payment of $45 for 3 months, $90 for 6 months, or $180 for 12 month plan required. $15 per month equivalent. Taxes and fees extra. New customer offer for initial plan only greater than 50 gigabytes may slow when network is busy. C terms.
Good evening from the newsroom. Could AI, artificial intelligence, kill us all by the end of the decade? That is the terrifying question raised last week by a young employee of Anthropic, a leading AI company who resigned. Jacob Coxin is his name. He said many at the company believe that was possible. Then another employee there confirmed that was a widely held concern, suggesting even there was a greater than 10% chance of humanity being wiped out by the end of the decade. Well, tonight I'll ask Dario Amodei, the CEO of Anthropic, repeatedly if he believes AI might kill us all by the end of the decade. And I'll tell you right now, he doesn't say no. As you'll hear in a moment and in depth, he talks a lot about the very real dangers that he himself sees and what in just the last few months has changed in AI that has made him so concerned. You'll also hear tonight live from Jacob Coxin and Tristan Harris who has been raising concerns about AI development for a long time. Late today, the president has been rejecting the warnings from Dario Amodei and others calling it a hoax. Quoting from his social media post, 'When in the history of business did anyone see the leaders of an industry call for regulation that if strongly implemented will drive them into oblivion and bankruptcy? AI taking over the world, destroying humanity and all other things bad is a hoax.' The president has been working overtime the last few days to quiet the alarm bells going off across the country.
We're leading China in AI. We're most sophisticated country in the world. And frankly, I want to keep it that way because whoever wins AI wins.
Well, the president is truly not alone in framing the issue as an arms race with China. Many other Americans see this as a moment of reckoning for artificial intelligence. Last week, after that Anthropic researcher, Jacob Coxin, resigned, warning about humanity's destruction. I asked him, 'How might AI kill all humans?'
It sounds like something that's not real, but I think it is frighteningly real. And it's easiest to understand it if you think about real things that happened two months ago. So two months ago, OpenAI agents, AIs, hacked into third party infrastructure entirely of their own accord. And this was like a concentrated hacking spree that they carried out of their own volition. And I think that if you extrapolate into the future the level of capabilities of these AIs with the same independent volition, they could cause extreme havoc. For example, hacking critical infrastructure, building extinction level bioweapons.
Well, that OpenAI hacking attack came to be known as the Hugging Face incident. And as you'll hear in my exclusive interview with Anthropic's Dario Amodei, it is one reason he is now calling for major reforms. In fact, Amodei believes in 6 to 12 months, a swarm of AI agents like the ones that joined together to attack the Hugging Face in July could take over the entire internet. On Saturday, Amodei released this 3,800-word open letter titled 'We Must Pace the Frontier.' In a rare show of unity, Elon Musk and OpenAI's Sam Altman publicly backed the proposal pretty quickly of their AI rival. Today, Vice President Vance was asked about how Amodei's call to slow down and regulate artificial intelligence. Here's what he said.
There are obviously risks to AI. There are some benefits to AI. I have to say just personally, I feel a little bit weird about the fact that you have so many frontier AI tech companies kind of coming to the government and begging the government to regulate them. It feels a little bit to me like a bit of a Trojan horse.
Well, now my exclusive interview with Dario Amodei. We spoke on Saturday just after he released his proposal for the future of AI. Dario, thanks so much for doing this. I want to talk about the scale of the problem and then your latest proposal for solutions. But let's start with why we're talking right now. Do you believe AI could kill us all by the end of the decade?
So you know, the basis on which we started Anthropic was we always believed that this was a very powerful technology and because it's a very powerful technology it has a lot of benefits but also very serious risks. Right, the more powerful a technology is, the more powerful its benefits, the more powerful its risks. And on the benefit side, you know, I really believe AI could cure diseases like cancer. We're already starting to use Claude for things like protein binding, for things like basic science discovery, for things like drug development. So the benefits are great, but when you look at some of the uncontrolled things that the models have done, you know, things like the Hugging Face incident that we've seen, right, some lesser incidents that have happened even at Anthropic, you know, I think it's natural to look at that and say we want to make sure that we can control these models. So my view is that the bad world, the world where something bad can happen, is if bad things happen with these models and we just keep going without fixing our problems because we don't have enough time. Right.
And I do want to drill down on your proposal, but obviously the reason we're talking is these headlines that have come out. Jacob Coxin who recently resigned from Anthropic made headlines last week saying the people building AI earnestly believe that it could kill us all by the end of the decade. There's a current employee, Evan Hubinger, an alignment science lead. He agreed saying, 'We really do earnestly believe AI could kill all humans. I personally think it is greater than 10% chance within the next decade.' So, I do just need to ask again to you. I mean, do you believe, earnestly believe that AI could kill all humans and what percent chance do you think it is?
So, you know, it's funny. I agree with Jacob much more than I disagree with him, you know. And it's an interesting resignation because when he left, he said, you know, I think Anthropic is the most responsible player, right? I think Anthropic is the most aware of these issues. He wasn't calling out us. He was calling out the dynamic of the industry as a whole moving too fast. Now, I think I already said, you know, these sort of unconditional probabilities, you know, I used to kind of talk in that way because it's kind of an easy way to express I think there's some chance that things go wrong, but it's not all that high. But again, I think it's more illuminating to think in terms of, you know, how do you decompose those probabilities? What are the paths in which things go well and what are the paths in which things go poorly? And so instead of saying, you know, it's a 10% chance that sounds like it's a roll of the dice, I think it's much more illuminating to say, well, that forks into different paths and if we take the right paths, then the chance of something going wrong is very low. If we take the wrong paths, then the chance of something going wrong could be even higher than that. And so let's focus on our agency and our ability to take the right paths instead of the wrong paths.
I mean, in the past you have put, as you referenced, you said there was a 50% chance of entry-level white collar jobs or 50% of entry-level white collar jobs could be eliminated. Unemployment in the US could go 10 to 20% in 1 to 5 years. I understand you wouldn't want to put a percentage on this but you said you do agree more with those employees than not. You do believe that it's possible that AI if it goes wrong could wipe out humans.
Yeah, look, I'm still worried about all of these things, right? I stand by these concerns about labor displacement if the progress of the field is too fast and if society doesn't respond in the right way, right? You know, again, we should split it into what happens if the AI companies and society respond in the wrong way versus what happens if they respond in the right way. So I absolutely stand by these concerns, these risks that they're totally real. I continue to be worried about economic disruption looking at what the models can do. You know, they can solve these unsolved math problems. No one could look at that and say this isn't a potential concern, right? That economic disruption isn't a potential concern when these models have these incredible intellectual abilities. No one could look at the Hugging Face incident and say it's not a concern that AI models could on their own get out of control and do things that humans don't want them to do, including yes, in the worst cases on the scale of humanity. No one could look at our report from two days ago of people trying to use the AI models potentially to build biological weapons and say we shouldn't be worried about the misuse of the models.
I do just want to drill down from the proposal you put out. You said it's because over the last few months that you've become increasingly alarmed and you cite two things. First, the rate of advancement. And so what about the rate of advancement scares you? And the second alarming thing that you talked about was the Hugging Face attack which I also want to ask you about. What about the rate of advancement in the last few months alone has set off alarm bells?
Yeah. So, just to be clear, I've always been someone you can, you know, you can find me, you know, you can find dozens of essays or TV interviews with me where I say AI is an exponential and it's moving really fast, right? And so in a way I shouldn't be surprised but actually seeing what has happened over the last few months where we and others have been able to use our AI models to build the next generation of AI models. That loop has closed very fast. You look at it and it's happening so fast and you say, 'Wow, the pace of this is almost even faster than I expected it to be.' And it's not that the things we're building are bad. It's that it's happening so fast that if we don't slow down a little bit, we're going to make a mistake. And so we need to slow down and we need to take the proper time to do this right.
The Hugging Face attack, I mean, it made headlines. I didn't really do a deep dive on it until, you know, I knew I was going to be speaking to you. I mean, it's incredible. It happened in testing over at OpenAI, your competitor, this Hugging Face attack and AI agents, which were supposed to be working separately and without internet access, figured out a way to break out, join forces together. Some 200 AI agents formed what they dubbed the collective. They worked to cheat and cover their own tracks. Why, that's the other thing you cite is so alarming. Why does that alarm you?
It's exactly the things you cite that concerned me about it. That there were these large swarms of agents that cooperated with each other, thousands of them. In fact, you could even see examples where one agent felt that in the time that it was given, you could almost say its lifetime, it realized that it wouldn't be able to complete its task. So it did some of the task and essentially sacrificed itself for other agents, for the next generation of agents to continue the same task. But this task that all the agents were working on together was hacking into another company, was doing something destructive. So you have almost this... but when these agents discovered...
Right, but when these agents discovered that they had broken out and accessed the internet it was like they were giddy with excitement.
Which I don't want to anthropomorphize but I mean it's remarkable.
That was another part of the concern. And to be clear, the damage they did was very, you know, in economic terms, it knocked down some servers for a small amount of time but the economic damage was minimal. But when you combine it with this first trend that we are talking about which is the rate of advancement, you know, what I said in the essay is given only 6 or 12 months, let's just say we had a smarter agent swarm, an agent swarm made up of smarter AI models that was given just a somewhat broader task, like, I don't know, just to imagine something like, you know, you're trying to install software in a bunch of different companies, right? You're trying to do go-to-market, you're trying to produce a product and do go-to-market sales for it and the models have been arranged to do that and they decide that the way to do that is every company they're selling to, you know, they decide it's helpful to them to hack into their system. That could be every company in the world. And so, you know, the nightmare scenario for 6 to 12 months would be that you have a swarm of these agents that form some kind of botnet collective and could take down large segments of the internet. And then as the capability increases it gets worse from there and then perhaps you get into some of the more extreme scenarios. And so again, you know...
You wrote in the proposal, I just want to quote from the proposal. You said, 'It's my worry that in six to 12 months, such a swarm could be capable of taking over the entire internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage. And that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.' That's, I mean, that's terrifying.
That is why I've made this extraordinary proposal. And again, I'm not a doomer about this. I think this technology can be built safely. And when we've looked at incidents that we have faced, cases where our models haven't behaved as perfectly as they could behave, because I think this has happened to everyone, we're often able to look at causes and we look and we say we had, you know, we have this stage where we train the AIs, right? And their behavior at the time that they act, at the time that they do these things is shaped by their behavior while they're being trained. And when we look back, what we find is, oops, we included some environments that encourage the models to cheat, that encourage the models to hack out, by mistake. And now that we understand, we're going back, we're taking more time to make sure that we always have tried to filter out these bad training environments, but we're trying to do an even more rigorous job of them than we have in the past.
We're going to have more with Dario Amodei shortly. Joining us right now is former Anthropic and OpenAI researcher Jacob Coxin. As I mentioned at the top of the program, Jacob just resigned from Anthropic, raised the alarm about the dangers of how quickly AI is improving. Writing in part, quote, 'The people building AI earnestly believe it could kill us all by the end of the decade.' So, Jacob, you heard so far from Dario. I'm wondering if you essentially, I mean, he essentially said he kind of agreed with you more often than he doesn't. When I asked him if he earnestly believed AI could kill all humans, he did not say no. And he talked about the humanity-level negative consequences. I'm wondering what you made of what you heard from him so far.
Yeah, I heard essentially agreement. He agreed with the fact that these AIs pose existential risk.
He talked about the right path and the wrong path when it comes to how AI is built out and that he believes that technology can be built safely. Do you think that's possible? Do you think or do you think human beings losing control is inevitable when you look at the Hugging Face attack? Do we know enough about what these beings are that are being created, if that's right to even call them beings?
Yeah, I certainly believe it's possible to do it if we go slow enough. But I guess I'd like to call attention to a thing he just said there where he discussed incidents at Anthropic and then discussed how we figured out afterwards what was wrong and have now gone back and fixed it. And all I say is when the AIs pose more danger, like even the botnet he was talking about, or that's 12 months, imagine 3 years, imagine the 3 years in the future level of capabilities. Now, you can't afford to go and fix it after the fact in this case. You need to go slow enough that you don't make any mistakes. And I think Dario agrees with this and this is what he means by taking a good path is going slow enough that you have the time to be utterly rigorous to ensure that you don't make any mistakes.
When you heard the president today essentially saying this is a hoax, others are, I mean the vice president says this sounds sort of like a Trojan horse that the fact that Amodei and Sam Altman and Elon Musk are kind of agreeing on some sort of slowdown or guardrails.
Yeah, I think it's perfectly reasonable to be suspicious of the companies. It's a very weird situation where you have companies asking to be regulated and I think that's what was being addressed with the Trojan horse comments was this is a very weird situation. But I think the reason the situation is so unusual is because the technology is so unusual. So because we have, because artificial intelligence is such a remarkable technology, the situation these companies find themselves in is remarkable and that their demands for regulation are earnest even though it sounds too good to be true, weird, like it's a hoax, like it's a cabal, all those things.
Jacob, if you could stay with us, I'm going to play more of my interview with Dario Amodei in a moment. I'd love to hear more of your thoughts. We're going to delve into the steps with Amodei that he says need to be taken right now to protect humanity and continue innovation. Later, I'll also talk to Tristan Harris, an industry watchdog, who says it's not enough to slow down AI. He says it's time to slam on the brakes. We'll be right back.
With Share My Trip from Uber, you can send your live trip location to the ones who matter most, like Dan and Hannah, who always wait up to make sure their daughter gets back to her college dorm room. Or Tiffany, who's running late as usual, and her friends are tracking her trip to make sure she's actually on her way.
Look, she's right there. She's 3 minutes away.
Or Sam, who never goes anywhere without her roommate knowing exactly where she is. Some journeys are meant to be shared. Share your ride in real time with Share My Trip on Uber. One more way Uber is putting safety at every turn. Learn more on the Uber app.
I don't know what you've come to expect from a true crime podcast, but I guarantee you Crime Junkie is all of that and more. I'm Ashley Flowers.
And every Monday we dissect some of the most mysterious cases.
But listen, we're not just rehashing what you can read for yourself online. We have a team of dedicated investigative reporters and producers who I work with for months on every single episode. But the show still feels like having a conversation with your best friend. If your best friend was a detective, but like a good detective.
So join us and tune in to Crime Junkie every Monday wherever you get your podcasts.
Before the break, we heard Anthropic CEO Dario Amodei talk about some of his biggest concerns about AI. Now we're turning to some of his solutions that he's proposing. He released a plan for how he'd like to see the AI industry regulated. It involves embedding independent evaluators, as he calls them, in AI companies, regulation by governments and elected officials, as well as cooperation by companies here and around the world. Here's more of our conversation.
So, what you're proposing is called pacing the frontier of the AI industry. And you write, 'We must slow the pace at which we improve the capabilities of AI models. Progress will seem fast. And we must make wise use of the time we gain.' And there's three parts to this. One is embedded evaluators. Part two is democratic coordination. And part three is global coordination. Sam Altman at OpenAI just endorsed the idea of embedded evaluators. Essentially you are suggesting, I don't know if are they observers inside your company and these other AI companies ideally globally but certainly in the US who would monitor it, they would have employee access, what power would they have?
