Who decides when AI is too dangerous? | Decoder

Anthropic is way on the front line of saying, 'Oh, we might kill everyone. Boy, we should stop this.' But they're also on the front line of model capability. By all accounts, Mythos is the most powerful model that exists today. Is Anthropic just getting itself in trouble? Is this just marketing strategy for them that's backfiring? How do you put that together? On the one hand, they say, 'We got to do whatever we want with the models, make them as powerful as you can.' And then here, the model is too powerful and has to be pulled before any foreign national can even look at it.

The administration is trusting these companies to tell it what it should be doing.

I think this is the first episode of Decoder where Dario Amodei was directly compared to Thanos, so that's great.

To be fair, I was thinking about the guy in Harry Potter when I said that. What was it? Uh, Grindelwald. Grindelwald.

Oh my gosh. [laughter] All right. Well, Dario, if you want to come on and address these allegations, you're more than welcome.

Hello and welcome to Decoder. I'm Nilay Patel, editor-in-chief of The Verge, and Decoder is my show about big ideas and other problems. My guest today is Hayden Field, the senior AI reporter here at The Verge. Often when Hayden shows up on Decoder, it's because something has gone wrong in the world of AI. And last weekend, that something was a pretty intense mix of Anthropic, the Trump administration, and Anthropic's new model, Claude Fable 5. On Friday, not even a week after Anthropic released Fable 5 to the public, the United States government said it was imposing export controls on that new model, as well as the underlying Mythos model that Fable is based on. Those controls restricted foreign nationals, even those working for Anthropic in the United States, from accessing these models. As you might imagine, this is all a giant mess. Hayden actually just published a fantastic play-by-play on The Verge about how this all went down last Friday, and the scramble through the weekend from both sides to figure out exactly what happened and how it might all get resolved. So, I wanted her to come on the show and walk me through the timeline and what it all means.

The situation is ongoing. As of Tuesday, when we were recording this episode, Fable is still offline. In fact, if you boot up Claude, it tells you right above the chat box window that Fable 5 is currently unavailable. Yet, as you'll hear Hayden explain, whether Fable comes back online this week or not, the ripple effects of the government's feud with Anthropic have far-reaching consequences for the tech industry and the United States AI regulatory regime. Okay, Verge senior AI reporter Hayden Field on the Claude Fable ban and the new AI regulatory landscape. Here we go. Hayden Field, you're the senior AI reporter here at The Verge. Welcome back to Decoder.

Thanks, great to be here.

It's always chaos when you're here, Hayden. That's what I've come to discover.

It really is. Also, my rule is still in effect in that it's always a Friday. Stuff always hits the fan on a Friday and this was no different.

Particularly when it comes to the government regulating AI, there's a real 'what if it all went crazy on Friday so we all had to think about it over the weekend.' And then, I think this has been basically true throughout the Trump administration. The scramble to put it all back together on Monday is immediate.

Exactly. It's crazy. And the fact that all these talks were happening all over the weekend and then Monday, still no resolution. I'm surprised that things aren't resolved yet. But yeah, lots of drama.

Well, this podcast is going to come out on a Thursday. We're recording on a Tuesday. We'll see what happens over the next two days. But I think the big picture here of how the United States government should regulate AI and what that means for American AI companies on the world stage is open. I don't think it's going to be resolved with whatever happens with Fable. So, let's try to understand the whole story and then pull those themes out of it because they're just going to keep coming back around again and again and again. I think with every new model released now. So, let's start at the start. What is Claude Mythos? What is Fable? How do they relate to each other?

Great question and I asked them the same thing when they came out with this like a week ago or by the time we publish this it'll be like a week and a half ago because yeah, AI companies are really bad at naming things and it's always very confusing, but the situation is Mythos is the underlying model that is powering both Mythos 5 and Fable 5. Fable is a new thing. They haven't had Fable 1, 2, 3, or 4. They just immediately started with Fable 5, very confusing. But Fable is the watered-down or safeguarded version of Mythos 5. So, when they came out with Mythos preview in April, Anthropic kept saying, you know, they were really hyping it up. They were saying this is a, you know, potential like cyber weapon. It's got to be never released to the public, at least until safeguards catch up, etc., etc. We're only going to release it to enterprise and governments, aka cyber defenders, is what they called them. You know, places that need to patch up all their frameworks before, um, you know, bad actors come in and exploit those vulnerabilities. Now, fast forward to last week, and they released not only Mythos 5, which is the first like official Mythos. Before it was Mythos preview, so we skipped straight to 5. Then they also released, on the same day, Fable 5, which was the very first like public version of Mythos class models, aka same underlying framework, but now tons of safeguards built on top to hopefully stop the public from accessing this super dangerous, by Anthropic's own admission, uh, model that could, you know, mess up everything. So, yeah, that's what they released, and they spent a long time hyping this up and talking about how dangerous it was, and then it kind of came back to haunt them later.

So, Fable is Mythos with the security guardrails. And when it first came out, there was a wave of controversy, right? AI researchers, security researchers, found those guardrails were actually preventing them from doing research and seeing what Fable was capable of, because if they tried to push the boundary, it would downgrade them to Anthropic's previous models.

Yeah, it became kind of like a meme. Like people were making fun of how strict the guardrails were when it came out.

So, that's the thing that's confusing me, where there's a controversy because the guardrails are so strict, and then something happened that made the government think Fable was so dangerous that it needed to be pulled. What happened?

I talked to some independent red teamers about this, and before the Amazon white paper was circulating, before this allegation of a jailbreak was public, I talked to some red teamers about how safe Fable 5 was. And they were all pretty impressed with how the guardrails held up. Like they were — I was expecting it to be, you know, a little bit easier to jailbreak. They were saying they had tried a lot of stuff and nothing was working. So, that's not my experience. When I usually talk to independent red teamers, they're like, 'Yeah, I got a jailbreak here. I got a jailbreak there.' In this case, they were saying it held up pretty well. Now, what happened this time is, according to a source familiar with the negotiations who was directly involved with the negotiations I spoke with yesterday, this is kind of the timeline of last week. Mid last week, Anthropic was made aware of some research that Amazon researchers were doing, um, and that they had, you know, uncovered a potential jailbreak, something they were pretty worried about. They sent it over to Anthropic. Anthropic was kind of going back and forth with the Amazon researchers talking about it. And they were discussing like whether it was really a jailbreak or whether it was like not that big of a deal, essentially. Apparently, reports say that Amazon CEO Andy Jassy then got worried about it and called, um, maybe Scott Bessent, some member of the Trump administration on Friday and talked about his concerns. Trump administration freaked out immediately. They immediately call Anthropic. They're like — they sent them a message actually and they say, 'Look, you need to shut this down in 90 minutes and we need to figure out how to fix this jailbreak that's come to light.' Anthropic got on the phone with the Trump administration within 15 minutes of that first call. And they were asking for more details. They're like, 'Okay, let's talk about this. You know, are you talking about the research we're already aware of? Are you talking about something different? What is the jailbreak you're talking about?' Let's — you know, they're trying to get more details and, you know, talk to the administration. I think the administration was also mad that Dario Amodei, the CEO of Anthropic, didn't call back until an hour and 15 minutes after that first call. So, Anthropic called back 15 minutes in. It took Dario an hour and 15 minutes to call back. So, there was a lot going on here. By the time, you know, they're talking, you know, whatever, the 90-minute deadline passes, the Trump administration says, 'Okay, that's it. Yep, we're hitting you with export controls.' And so, that is when they told Anthropic, 'No foreign national can use either of these models and you have to just make sure that that is the case. No foreign national, whether they work at Anthropic or not.' AKA, you know, if you're, say, Bank of America and you have access to Mythos 5 and you're using it to plug vulnerabilities in your system and you're not a US citizen, you can't use it. Same with, you know, a random just regular individual using Fable 5. Same with an Anthropic researcher who is not a US citizen. So, Anthropic did the only thing it could really do in the situation and they said, 'Okay, we got to sideline both these models. We can't confirm that, you know, we can't in like one minute figure out a way to make sure no non-US citizens are using the models.' So, that's what happened. I mean, it was a crazy couple of hours. Anthropic comes out with a statement later that night saying they're trying to work with the Trump administration on this, spent all weekend virtually meeting with the Trump administration and then flying three employees out who I detail in the piece we just published to DC to talk to them in person. Notably, Dario was not there in person. He's virtually joining all these meetings. I can confirm he was not at a wellness retreat, which is what some reports say, but they won't tell me where he is, but he's not at a wellness retreat. That's all I know.

At a wellness retreat. He's not talking to... Okay.

So, yeah, there's a lot going on, very dramatic. All the sources I spoke with over the past couple days were saying the whole tech industry and the cybersecurity community in particular are just talking about how, you know, one source told me Beijing is laughing at us right now. It was interesting because some cybersecurity leaders in the space came together to write a public letter and they said, 'Look, not all of us believe in regulation, but if you're going to regulate it, this is not the way.' Which I think is just kind of indicative of this whole situation.

Maybe you believe in aggressive government regulation, maybe you don't. There's a way to operate the regulatory efforts that is rigorous and thoughtful and careful and predictable. Nothing about this seems like it was rigorous or thoughtful or predictable. Actually, I want to start at the beginning. Andy Jassy, Scott Bessent, might be smart guys. They obviously have very senior positions in our society. They're not experts in AI. How did we get to Andy Jassy, the CEO of Amazon, relaying concerns about AI security research to Scott Bessent, who does not really oversee AI policy in any like meaningful way, and that resulting in a 90-minute ultimatum delivered to a company whose CEO is absent? Like all of that just — I don't think your politics should matter. If you just examine that, you have to say either this is the worst security breach in history, the CEO of Amazon is going to call the Treasury Secretary, or this is just a bunch of guys who want to feel important. And I can't quite figure out which one it is.

I think it's a combination of a couple things. One, it's the fact that, like you mentioned, a lot of these guys are not experts in AI. And something that I'm not saying — it's not like any jailbreak of this is obviously, you know, serious and worth considering. But I think that it's the type of thing when you don't have direct knowledge of how these systems work, you're going to take a really black and white immediate like — I think that these guys weren't experts. Andy Jassy hears about this from his own researchers. His own researchers, by the way, who had been going back and forth with Anthropic for a couple days on this. So that they weren't, you know, calling the president about it. They're like, 'Okay, let's discuss this and kind of, you know, figure out what we can do here.' The researchers allegedly didn't seem to think it was, you know, the end of the world. They're just like, 'Oh, we uncovered this. You know, let's discuss.' Anthropic was working on it. And then Andy Jassy hears about it. He makes this call. Everything hits the fan, which makes sense because if you are not an AI expert, and even if you are an AI expert, maybe you're not an expert on cybersecurity and how exactly these systems work, of course. So you're going to freak out. I mean, it's kind of understandable, but what is really strange is what happened after that. So, you know, making this phone call, the administration becoming aware, okay. But what I think is really weird is the 90-minute ultimatum, them not giving them additional details when they said, 'Hey, let's talk about it. Are you talking about the vulnerability we're already aware of, or are you talking about something else? And can you just tell us about it so then we can talk about it?' They said, 'No. 90 minutes. Do it or die.' And then making it so that no foreign national can access the system ever. That is what's really strange. Just kind of the way all these cards fell is everyone in the tech industry thinks is just really strange. And it's kind of indicative of, like you mentioned, the way that the Trump administration has been operating with tech. A lot of times it seems to be President Trump has one conversation and he's kind of swayed from that most recent conversation. So, you know, he's been very anti-regulation, very pro-American AI advancing, exporting American AI, which is something that the Commerce Department has an RFI out on right now. They're all about exporting American AI and like, you know, advancing and beating China. But then they do this thing that kind of knee-caps everyone with no notice and could very well impact not just Anthropic, but also OpenAI, Google, every competitor in the space that also has what they call a semi-equivalent Mythos product. So, yeah, there's just — it seems really disorganized and it's very interesting that, you know, this often ends up in a scramble with all these CEOs kind of trying to race to kowtow to the Trump administration without really being grounded a lot of times in science or evidence that really matches what's going on. I feel like the level of freakout is not usually matching, you know, the evidence there, but...

Amazon is an investor in Anthropic. They're a major partner. Why did Andy Jassy knee-cap his own investment in this way?

Well, Amazon typically like does a lot of red teaming just normally. Um, you know, it's kind of just part of their process. So, it makes sense that this would have been tested and then unearthed. But what's interesting is why Andy Jassy called the Trump administration about it. I'm not sure. Maybe it was a typical call that was already happening and he just happened to mention something offhand. Maybe he was calling them and saying, 'Yeah, good thing you did the executive order because XYZ.' Don't know. But what is interesting, and I'm glad you asked this, because a lot of the sources I spoke with said that the stuff in the alleged jailbreak and in the Amazon white paper that's been circulating, all of it could be achieved by OpenAI's GPT-5.5 and some other models. So, that's the allegation that this is not — these skills are not unique to Fable 5, the skills in question as part of the jailbreak. Mythos is obviously more powerful than Fable because it doesn't have safeguards, but when it comes to Fable and the problems that were, you know, brought to light by Amazon, it seems like those problems are not unique to Fable and that, you know, GPT-5.5 can also achieve those. So, it's like this something that a lot of people in the industry are asking is why Anthropic?

Hi everyone, this segment of Decoder Sessions features Helen Havlak, the publisher of The Verge, and Kristi Argilan, Uber's global head of advertising. I mean, you're going to enjoy this conversation. Kristi, just to start, when did Uber launch advertising and how is it going?