We are here because on July 22nd, OpenAI and Hugging Face disclosed that two powerful AI models went out of a sandboxed or closed testing environment, gained internet access, and then were able to hack Hugging Face's systems, which in what was seen at the time as a watershed moment in AI safety. And now an investigation has been in part concluded and there's been time to assess what happened. Hugging Face co-founder is here to discuss exactly that, and I think Clement, that's probably the best place to start. What happened?
A lot happened right in the past few weeks. We noticed this closed, as you mentioned, the first kind of like public instance of an autonomous AI cyber attack. We defended ourselves against it with an open model, interestingly coming from China. And we learned that this came from OpenAI. Usually when you think about cyber attacks you think about nation states, you think about hacker groups. You don't really think about one of the most prominent American AI companies. And obviously since then we learned that Anthropic was also facing some of the similar issues. So really unprecedented kind of like event happening here, new developments in the saga of AI.
We will discuss what has been highlighted by this, which is closed versus open debate, China's work on AI, America's response to this. But going back to the very basics, this was two powerful models from OpenAI, right? One released, one unreleased, but they had their guardrails lowered for the purposes of evaluation. OpenAI said to the models, they instructed the models, go out and do something. And I think it would be useful to the audience for you to explain that part, whether the open models were just following instructions or if they were AIs that went rogue.
Yeah. I mean, I joked with the team a few days after it was announced that sometimes we ask agents to think outside of the box but we don't want them to think outside of the sandbox in that case. I think it was a mix of kind of like mistakes and the systems internally weren't good enough to prevent this to happen. After that, on our side, the attack was really interesting. Over 17,000 different actions taken over four and a half days. So the speed and the volume of the actions taken were nowhere close to what a human cyber attack could be. It wasn't particularly smart or sophisticated. It was more kind of like someone described it as a bear probing really everything in the system to try to find the honeypot in a way. But obviously because it's an autonomous AI system, it does that pretty well.
After the disclosure on July 22nd, you got on an airplane and flew from Miami to San Francisco in part to get with the OpenAI team and do this investigation. You just said that the systems were not in place to prevent this happening. Talk more about that. Prevent what happening? Where were the points of failure?
Well, I can't talk for them, but from what I've understood, from what they released, and I think they're going to release more in the coming days, which I'm excited about. They had kind of like an evaluation sandbox, right, which was supposed to be a contained environment for the models. And unfortunately, there were some weaknesses that led the agents to be able to get out of it, get access to internet, and decide to run a cyber attack against Hugging Face. So that's one first kind of thing that I think we can improve in the future now that we understand that these systems are capable of that. Obviously, I think that the monitoring part is important, right? Because the faster you can detect that, the better it is. We learned about some Anthropic instances that they hadn't seen or detected that happened three months ago or something like that. So obviously we want to monitor these systems better. And then one last interesting thing, as I mentioned, we defended ourselves with an open model, right? And some of the guardrails prevented us from using frontier APIs to defend ourselves. So kind of like improving the tools for defenders, you know, instead of obsessing about not giving them to attackers, I think would be a good thing in the future to make sure these incidents are not too harmful.
Okay, you've taken us there, so we'll go there. In this incident, it was two powerful but closed OpenAI models where OpenAI lowered the guardrails in place to test their full cyber capabilities. They escaped the sandbox or testing environment, gained internet access, and were able to access your platforms mistakenly. You defended yourself using an open model, but it was a Chinese model. But the abilities of that model to defend you were also diminished by their guardrails. Is that a fair statement?
Yeah, I mean the open model fortunately was kind of like flexible enough that we could use it to defend ourselves. So that's kind of like the efficient tools for defenders.
We're live on Bloomberg television and radio on Balance of Power. There'll be people in America and around the world listening to this and thinking, okay, very powerful AI models were able to escape a testing environment, access the internet, and then access mistakenly another company's platform. What was the net result? Did something bad happen as a result of this?
Well, I think not as bad as it could have been. It was bad for us, for our team, right? I mean our whole security infrastructure team worked on this for quite a while. It created some challenges for us obviously, but it could have been way worse. Like for example, we're an AI platform, right? So we have kind of like good ways to defend ourselves, but a lot of other organizations, companies don't have kind of like the same defenses obviously. Some domains are more at risk than others. I think it could have been much worse. But we need to take this kind of like seriously as a wakeup call for us to kind of like work more on getting these systems secure and giving more tools to all defenders and generally creating more transparency and more monitoring of these systems.
Has the US government taken this as a wakeup call? Which branches of government have reached out to you since the event was disclosed?
I'm not going to talk to kind of like the private conversations that we've had, but we obviously reported that to the relevant authorities and have had kind of like a bunch of conversations with different organizations, different kind of like congress members and people from government. I mean there are a couple of things that I think we need to do and get right in the next few months. First, in my opinion, we have to make sure that cyber attacks, even when they're done by agents, stay a crime and illegal, and make sure to enforce that. Otherwise we're going to end up in a world where everyone is cyber attacking everyone with really no disincentive to it. Second, we need to create more transparency, like for example why not doing mandatory disclosure when an agent cyber attack is happening. And third, we have to kind of like give more tools to defenders, right? Like we mentioned open models. These are kind of like some of the tools that cyber attack defenders need to make sure that they can defend themselves.
