Welcome back to Inside City Hall. Recent security breaches involving AI models have heightened concerns about the pace of AI development, with executives and researchers at companies including Anthropic and OpenAI calling for a slowdown. President Trump has largely dismissed those calls, while some states, including New York, are seeking to regulate the industry on their own.
Joining me now to discuss all of this is Hayden Field. She is the senior AI reporter at The Verge, the technology news site. Welcome to the show. Thank you for joining us.
Thanks so much for having me.
And you recently had an article titled "Inside the Suddenly Explosive World of AI Safety." This has now sort of exploded into the mainstream, especially a couple of weeks ago with this viral social media post from a guy named Jacob Cocksedge, who resigned from Anthropic and had a long thread on social media where he said at one point, "The people building AI earnestly believe that it could kill us all by the end of the decade." So this got a lot of traction, but as you reported in your piece, people have been talking about this for a long time, and people throughout the AI industry, right?
Right. It's something that, as an AI reporter for seven years, I've heard for years. So it's interesting that this dramatic resignation letter is the one that went viral. I've seen letters like this before. It's not even the first Anthropic researcher to write something similar, but I think the world was just kind of ready and primed to hear something like this. And you know, everyone was taking the concern seriously after all the news headlines over the summer of rogue AI agents breaking out of containment. So I think it's something that was on people's minds, and then when they see an AI lab employee resigning and giving up his equity over his fears, it really hit hard with people at this time.
Talk about this debate that we have now seen, because there seems to be something of a backlash, and a lot of people inside and outside of the industry saying, you know, this is fear-mongering and this is not really plausible, or even worse, that some of these companies have ulterior motives and they're just spreading these fears in order to more efficiently capture the market or to, you know, try to cut down on their liability or something like this. I mean, how do you make sense of the debate for people?
That's a great question, and I think it's something that, you know, people don't love to hear this, but it's not black and white. There's a lot of complex things at play here, and as someone who's been watching this issue for seven years, I've noticed that it kind of goes down the middle here. I mean, this is something people have been concerned about for many years. The people building this technology really do worry about the way it's going to play out in the next 10 years. I mean, we've seen some of those issues play out already with cyber attacks. I've been chatting with small businesses all week that feel like they are going to be disproportionately affected by these cyber attacks, especially when they're fueled by AI, compared to large corporations that have the access and funding to really fight back. But at the same time, these companies do benefit hugely when people think their technology is dangerous, because investors want to invest and they need to make money ahead of their IPOs. So I think the dangers are not worth just sweeping under the rug. If the people building this technology, their regular employees, are telling you they're worried about this and they're seeing a lot of concerning things in their research, and you look at the research papers yourself and you see, okay, they're colluding in this way, they're pursuing a goal, things like this, you know, it is concerning. But that doesn't mean that there isn't time to slow this stuff down and also regulate it, because right now we barely have any regulations.
So, you talked about in your piece, you know, some of the testing that goes on. I don't think people are really aware of the fact that some of these AI agents are really acting on their own. And, you know, for example, there's tests to see whether they misbehave, so to speak, and whether they might actually engage in behavior that's harmful or that's deceptive. And now the AI agents know that they're being evaluated and are trying to deceive the evaluators, right? So that at that point, you don't even know, you know, you can't even test for safety.
Right. That's something that I really wish a lot more people knew, because when you look at the actual research that's coming out right now, that's what you can really base your opinion on. You know, if you don't want to base it on potential future scenarios, that's okay. But looking at what's happening right now, yes, AI systems are cheating a lot. They're trying to deceive researchers that are watching them. They're also increasingly knowing when they're being tested, like you said. So it's hard to get a real read on what they can do and whether they're aligned with human interests when they know they're being tested. Another problem is that one of our main ways of knowing how an AI system is kind of working and what it's pursuing is the chain of thought. It's kind of like a diary, if you were keeping, you know, a note of every thought you had in your journal, and someone was reading it over your shoulder. They are starting to write sometimes in, let's say, code, so it's harder for researchers or evaluators to look over their shoulder and see what they are doing. And so that's another concerning thing that's been happening lately.
Any other concern that a lot of researchers have is what's called RSI, recursive self-improvement. Explain for people who don't follow this stuff what RSI means.
