Healthcare doesn't always work great. If you've ever waited on a refill or couldn't schedule an appointment, you get it. That's the kind of stuff Optum is changing. They're using data and technology to integrate patient care, pharmacy, and everything else. So healthcare is connected, not complicated. What's that look like? Cheaper prescriptions that are easier to get and care that looks at the whole person, how you need it. Optum is helping make healthcare work as one for everyone. Learn more at business.optum.com.
Bloomberg Audio Studios, podcasts, radio, news. We are here because on July 22nd, OpenAI and Hugging Face disclosed that two powerful AI models went out of a sandboxed or closed testing environment, gained internet access, and then were able to hack Hugging Face's systems, which was seen at the time as a watershed moment in AI safety. And now an investigation has been in part concluded and there's been time to assess what happened. Hugging Face co-founder is here to discuss exactly that. And I think, Clem, that's probably the best place to start. What happened?
A lot happens in the past few weeks. We noticed and disclosed, as you mentioned, the first kind of public instance of an autonomous AI cyber attack. We defended ourselves against it with an open model, interestingly coming from China. And we learned that this came from OpenAI. Usually when you think about cyber attacks, you think about nation states, you think about hacker groups, you don't really think about one of the most prominent American AI companies. And obviously since then we learned that Anthropic was also facing some of the similar issues. So really unprecedented kind of event happening here, new developments in the saga of AI.
We will discuss what has been highlighted by this, which is closed versus open debate, China's work on AI, America's response to this. But going back to the very basics, this was two powerful models from OpenAI, right? One released, one unreleased, but they had their guardrails lowered for the purposes of evaluation. OpenAI said to the models, they instructed the models go out and do something. And I think it would be useful to the audience for you to explain that part, whether the OpenAI models were just following instructions or if they were AIs that went rogue.
Yeah, I mean, I joked with the team a few days after it was announced that sometimes we ask agents to think outside of the box but we don't want them to think outside of the sandbox in that case. I think it was a mix of kind of mistakes and the systems internally I think weren't good enough to prevent this from happening. After that, on our side, the attack was really interesting. Over 17,000 different actions taken over four and a half days. So the speed and the volume of the actions taken were nowhere close to what a human cyber attack could be. It wasn't particularly smart or sophisticated. It was more kind of like someone described it as a bear probing really everything in the system to try to find the honeypot in a way. But obviously because it's an autonomous AI system, it does that pretty well.
After the disclosure on July 22nd, you got on an airplane and flew from Miami to San Francisco in part to get with the OpenAI team and do this investigation. You just said that the systems were not in place to prevent this happening. Talk more about that. Prevent what happening? Where were the points of failure?
Well, I can't talk for them, but from what I've understood, from what they released, and I think they're going to release more in the coming days, which I'm excited about, they had kind of an evaluation sandbox, right, which was supposed to be like a contained environment for the models. And unfortunately, there were some weaknesses that led the agents to be able to get out of it, get access to the internet, and decide to run a cyber attack against Hugging Face. So that's one first kind of thing that I think we can improve in the future now that we understand that these systems are capable of that.
Obviously, I think that the monitoring part is important, right? Because the faster you can detect that, the better it is. We learned about some Anthropic instances that they haven't seen or detected that happened three months ago or something like that. So obviously we want to monitor these systems better. And then one last interesting thing, as I mentioned, we defended ourselves with an open model, and some of the guardrails prevented us from using frontier APIs to defend ourselves. So kind of improving the tools for defenders, instead of obsessing about not giving them to attackers, I think would be a good thing in the future to make sure these incidents are not too harmful.
Okay, you've taken us there, so we'll go there. In this incident, it was two powerful but closed OpenAI models where OpenAI lowered the guardrails in place to test their full cyber capabilities. They escaped the sandbox or testing environment, gained internet access, and were able to access your platforms mistakenly. You defended yourself using an open model, but it was a Chinese model. But the abilities of that model to defend you were also diminished by their guardrails. Is that a fair statement?
Yeah, I mean, the open model fortunately was kind of flexible enough that we could use it to defend ourselves. So that's kind of the efficient tools for defenders.
We're live on Bloomberg Television and radio on Balance of Power. There'll be people in America and around the world listening to this and thinking, okay, very powerful AI models were able to escape a testing environment, access the internet, and then access mistakenly another company's platform. What was the net result? Did something bad happen as a result of this?
Well, I think not as bad as it could have been. It was bad for us, for our team. Our whole security infrastructure team worked on this for quite a while. It created some challenges for us, obviously, but it could have been way worse. For example, we're an AI platform, so we have kind of good ways to defend ourselves. But a lot of other organizations, companies don't have the same defenses. Obviously, some domains are more at risk than others. But I think it could have been much worse. But we need to take this kind of seriously as a wakeup call for us to work more on getting these systems secure and giving more tools to all defenders and generally creating more transparency and more monitoring of these systems.
Has the US government taken this as a wakeup call? Which branches of government have reached out to you since the event was disclosed?
I'm not going to talk about more the private conversations that we've had, but we obviously reported that to the relevant authorities and have had a bunch of conversations with different organizations, different congress members, and people from government. There are a couple of things that I think we need to do and get right in the next few months. First, in my opinion, we have to make sure that cyber attacks, even when they're done by agents, stay a crime and illegal, and make sure to enforce that. Otherwise, we're going to end up in a world where everyone is cyber attacking everyone with really no disincentive to it. Second, we need to create more transparency, like for example, why not doing mandatory disclosure when an agent cyber attack is happening? And third, we have to kind of give more tools to defenders, like we mentioned open models. These are kind of some of the tools that cyber attack defenders need to make sure that they can defend themselves.
