Look, I think there is this caricature of me which is like I don't care about AI safety and I'm, you know, just trying to like make revenue go up and, you know, a YOLO CEO, I believe someone once said.
OpenAI has had a challenging year. Musk sued OpenAI, and another top executive left OpenAI this week. Apple is suing OpenAI. Lawsuits accuse OpenAI of wrongful death assistance. The company which ushered in the AI boom has dealt with staff departures, safety issues, lawsuits, and a public backlash against the rapidly advancing technology. It's lost ground in the AI race to its arch-rival, Anthropic. But company leaders say they've fixed these issues and have a vision for OpenAI's next chapter that is incredibly ambitious.
In mid-August, reporter Alex Heath went inside OpenAI's San Francisco headquarters to speak with more than a dozen of the company's leaders about what's next and how the company is balancing growth with safety.
We are so good at running stuff in the cloud now.
Sam, I want to start with something you recently posted actually. You said, "We did not have our best last 12 months ever, which is mostly my fault, but we are about to have our best 12 months." What did you mean by that?
I think we clearly had some missteps as a company, which will happen periodically. I mean, part of trying to make a portfolio of bets is that sometimes more of them work and sometimes less of them work.
Were there specific decisions you can look back on that you made that cost the company momentum?
I think we were trying to do too much on the product side. And so there was like, you know, these were all things that were actually very good things to do. They were just not as good as the most important thing to do, which was sort of push on the general capability of the intelligence. So we were doing things like a browser and Sora, and we now have a very relentless focus on being this intelligent service to people. I think our models have gotten to be the best in the world, and they will get much, much better over the coming months. People are really doing remarkable things, but that is what we should have been focused on, and I should have been holding everybody to: this is the one thing; we will not worry about these sort of side quests.
More. In July, the OpenAI team was rattled by a breach that marked a turning point in AI advancement. Rogue agents in the midst of a test escaped a contained environment known as a sandbox by exploiting a cybersecurity vulnerability and hacked into a company called Hugging Face.
I think it felt very weird and unprecedented to us.
It took time for OpenAI researchers to discover the breach. When they did, they imposed new safety measures. That was a real moment of, "Man, this is like a sci-fi story."
The Hugging Face incident is amazing on a lot of dimensions. I was re-watching your team's Black Hat presentation last night for that. And there were things that blew me away, like the model literally writing like, "Holy shit," when it escaped and was able to get onto the internet.
You could see some colorful wording here in the model's chain of thought to summarize the model's response to this result.
And it's made me think about like what does alignment even mean in this context? Like what are we aligning towards? Because if you look at it very plainly, you guys gave it the task of completing an eval, and it did whatever it needed to do to try to do that. And in a way that's aligned, if you were to just take a very simplistic view of it. But I'm curious like how your thinking on alignment has evolved since then.
Well, in a way that's aligned; in another way, it's like not at all. Right. Like when we talk about alignment, we talk about following the intent of a user. And the intent of the people that were running that was not like break out of your sandbox and go steal the thing. No. And so I think there was a failure in alignment in that it was not doing what its user intended. You can understand how every piece of it happened, but the number of things that came together for the Hugging Face incident to happen was a real—wakeup call is too strong of a word because again we had talked about this, but it was like that and the things that happened at other companies were a legitimate moment of like, wow, the AI capability level has reached new heights, and our alignment of the model, the security we have around the model, that failed. Now, we treated that as an accident and we've responded as such, and I think that is the way to make things better.
Following the widely covered Hugging Face breach, Anthropic and Meta announced that some of their models had also escaped during training. The breaches sent shockwaves throughout the AI industry.
More than 1,000 workers from top AI firms have signed a petition calling on the US government to help, quote, "deliberately pace AI development."
And the OpenAI team soon discovered more troubling behaviors while training yet another unreleased model. In response, OpenAI announced that it would pause training of Frontier models until new safeguards are put in place—the first time the company has taken this kind of step.
I don't think this is like, you know, we're in this extremely critical potential catastrophe point. But I also think that as the stakes get higher, as the models get more capable, because of what our mission is and because of how important it is that safety outweigh all the other pressures we have, we wanted to react with an abundance of caution. And I think that's the right thing to do. I think it's good that we're doing that. I think it is a good time to slow down and make sure we can have new safety cases that justify the runs we want to make. Previously, more of the risk in the world was on how the models were deployed and used. We are moving to a world where there's more risk during the actual training and production of the models, and it's good to react, but I don't want to over-dramatize it either. People need to stay in control. We cannot have a loss of controlled AI. We cannot have a kind of like worship our models and sort of trust them unchecked to make our decisions for us. We have to keep the power in human hands.
Do you think about the impact this will have on the company's momentum?
Getting AI safety right is more important than any company's momentum. So like, yes, I won't pretend it's not some factor or something to think about, but it does not like rise above the noise floor.
Both OpenAI and Anthropic are preparing to go public in the near future. With Anthropic now expected to go first, becoming a public company would bring new pressure from shareholders. OpenAI CFO Sarah Friar recently told employees that the company will IPO in 2027 or sooner if its business continues to inflect.
You don't seem phased about like the implications of pausing training, and it sounds like you think the business will be okay. I'm sure you're still going to get concerns from people.
I think I've been very consistent over the 10 years of OpenAI—more than 10 years, almost 11—of talking about the risks and the upsides and the need to balance those. And I don't think we're perfect. I don't think our company is perfect. I don't think our model is perfect. I don't think I am perfect. But I think unlike some other people running various AI efforts, I've said the same thing through actions and words matched. And this is a moment we always talked about, and we always said we'd put this ahead of profits or revenue or anything else. I still think we will build a phenomenally successful company. But, you know, maybe we're like not the company you would have expected to say, "Hey, we're going to slow down because we see these new risks." But that is always the company we've thought we are.
