Allen Says AI Safety Must Keep Pace With Model Releases

Or are both of these things true? There absolutely are serious safety concerns regarding AI that deserve immediate attention. And while President Trump did dismiss such concerns as a hoax, I think that has more to do with trying to strengthen his negotiating position in advance of the summit with Communist Party Chairman Xi Jinping of China, rather than his true beliefs on the subject. I think his true beliefs on the subject were revealed when his administration banned Anthropic's Mythos model worldwide. They didn't do that because there's no safety concerns here. Recursive self-improvement is a concept we've talked about here.

Well, to a certain extent, you can say there's always been a trend of recursive self-improvement. Every tool that we've made since the stone knife has been helpful in making the next generation of tools.

Yes, exactly. What's unique about recursive self-improvement is the extent to which the humans might not be involved at all in the story, which is to say, 100% of the AI performance improvements are coming from the AI. But we could hit the danger zone well before we hit that 100% AI-driven self-improvement loop. The issue is, are our tools for understanding how these AI models think and how they might go wrong anywhere near as powerful as the AI we have today, or changing and upgrading as fast as the AI we have today? And that's the concern that we've now seen issued by four of the leading AI CEOs in the entire world — Anthropic, OpenAI, xAI, and Google DeepMind — all getting behind this idea of we need to make sure that model releases into the general public are not exceeding our pace of safety technologies, which we've made a lot of progress over the last year, but nowhere near as much progress as made in general AI capability.