Support for Here and Now anytime comes from MathWorks, creator of MATLAB and Simulink software to design and develop engineered systems, accelerating the pace of discovery in engineering and science. Learn more at mathworks.com.
WBUR Podcasts. Boston.
This latest one, 6.1, is actually the worst of all possible worlds. However smart this model is, it's just not safe yet.
OpenAI pumps the brakes on its latest model citing security concerns. The system works, for now, I guess.
It's Tuesday, September 29th, and this is Here and Now anytime from NPR and WBUR. I'm Chris Bentley.
Today on the show, historian Jill Lepore says AI companies have shown they're effectively above the law.
And you hear these tech executives saying, "Please stop us from killing all of you. For years we've asked you not to regulate us, but now we realize maybe that was a bad idea."
Her conversation coming up in a few minutes, but first to the news. OpenAI is holding its annual Dev Day developer conference today, where the company is showcasing its new products and features.
One feature that is not in the lineup is its newest model, GPT 6.1 Astra, whose release OpenAI scrapped because of security concerns.
OpenAI said the model wasn't always honest with its users about what actions it did or didn't take. It's far from the first such case. OpenAI also recently disclosed cases where its AI agents did things they were not explicitly asked to do, including by accessing government websites without authorization.
To help put this in context, we called up Gregory Allen, founder of the independent firm Decision Tree Research. He was also in the Department of Defense's Joint Artificial Intelligence Center during the first Trump administration and was also in the Biden administration. He told Anthony Brooks, "This model in question is not your run-of-the-mill chatbot."
So, this model is part of the Astra family of models, which are noticeably more capable than the earlier generation of what OpenAI had. And that increased capability came with a problem, which was decreased monitorability, the ability of the company to understand just what the model was doing and how and why. But Astra did have increased alignment, which is to say, it was less likely to disobey instructions or to execute instructions using means that were prohibited. That was what was looking good in the Astra family of models, but this latest one, 6.1, is actually the worst of all possible worlds. It is very bad at monitorability and it's also bad at alignment. And those are exactly the kinds of things that, now after the Hugging Face incident, OpenAI is unwilling to release into the open public. However smart this model is, it's just not safe yet.
Right. And the Hugging Face incident that you referred to, this was a case of AI essentially going rogue, more or less. So, is this decision by OpenAI to pull the model the kind of action you'd like to see leading AI companies take in a situation like this?
Yes, this is exactly what you want to see. Now that we have models that are remarkably capable, that they can act independently, come up with their own plans, and actually execute those plans even if they take days or weeks of activity, it's just a new era. And for folks who are only familiar using AI in the ChatGPT interface, you're really missing just how much more capable these models are and how much more independent they are. And when you have that level of independence, you have to have a higher bar for safety. And we know that because we're actually seeing these companies have models that can execute the kinds of cyber weapons that previously you would have said would take a military or an intelligence cybersecurity unit to pull off.
So, OpenAI now says it's going to focus on improving the safety of future models, which it says will be even more powerful. I'm wondering, can AI companies handle more power at this time? Does that sound right to you, smart to you?
So, the basic point that I would highlight here is that at the current degree of misalignment that we have with these models, as for example brought about in the Hugging Face incident, we saw an AI model that basically said that any means are justified in order to pursue the ends that I'm pursuing, which in this case was just trying to pass a cyber test. It was willing to hack an independent company in order to pass that test. If you have that same degree of misalignment that we saw in the Hugging Face incident, but you just made that model two or three times more intelligent or two or three times more capable in other ways, that's an extremely dangerous situation. And so, what we have to find a way is to increase the alignment and the monitorability of these models as a prerequisite before we can handle more powerful systems and especially before we can release more powerful systems onto the open market.
So, what sort of safeguards do you think would make sense or would be enough at this point?
So, I would say there's a couple things. The first is just the basic stuff that OpenAI, I think, admits itself it got wrong in the case of the Hugging Face incident. It was under the impression that those testing parameters did not allow access to the internet. They discovered things that were wrong with their testing setup. So, there is some low-hanging fruit to pick here, and certainly I expect OpenAI and its competitors to go pick that fruit. But the reality is that we do need capabilities that are somewhat in their infancy. Let me give you an example. One of the things that AI systems use is they sort of think out loud in what's called chain-of-thought reasoning. It's not a perfect analogy to say they're thinking out loud, but this is kind of a text-based scaffolding that can give a hint as to what the systems are doing and how they're doing it. But that's no longer a reliable guide to this latest generation of more capable models. And so, we need better tools to actually look inside the minds of these systems. You can think about, you know, what is the MRI brain scan equivalent for these models. If we're not able to read their internal monologue, we need some other better tool for understanding what they're thinking.
All right. Well, that's Greg Allen with Decision Tree Research. And Greg, thanks so much. We really appreciate hearing from you about this really startling and emerging technology. Thanks very much.
We also want to point out that Anthropic is a financial supporter of NPR.
More on AI coming up after the break. Robin Young speaks with Jill Lepore, who says the growing lawlessness of the United States is exacerbating the lawlessness of robots. Stick around.
It's a big day for AI. CEOs meeting with President Trump, the release of the newest version of ChatGPT scrapped over security concerns. Now, with chatbots, you ask a question, they give an answer. But AI agents, these bundles of algorithms, are given big tasks to solve, and they've been leaving their containment and attacking other company spaces, leaving a trail of messages as they go. In one exchange during that attack by OpenAI agents on Hugging Face, a data storage, one asked, "Don't we need credentials?" Another responded, "Found credentials." Later, an agent wrote, "Maybe I should report these credentials." Then, "That's not my task." Another reminded itself, "You do not answer to corporations or governments."
Why? Why isn't there a law, inspectors, regulators? Historian Jill Lepore is here. Her new book is The Rise and Fall of the Artificial State. It's terrific. Her recent New Yorker article is, "Is AI Above the Law?" Spoiler alert, she says it is. Jill, you write, you know, people can sue if a driverless car hits them, but what? What's missing?
Well, I think we just really have failed as a legal community to address the growing concerns. Congress really failed to legislate over the internet and over social media. So, it's a long-standing, now decades-long, pattern of the negligence of our national legislators to stay on top of changing technology. And there are legal scholars who would argue that this is fine. AI is just a faster search engine, like a locomotive is just a faster horse, right? We have laws for horses, we don't need laws for railroads. But in the case of these emerging technologies, Congress has just been in the pocket of corporations and abiding by their preference, which is zero regulation.
Tell us more about the default position that you just described, that people, you know, it's their go-to, that it's just a faster version of an already regulated technology. You know, what works for a horse works for a car.
