Sam, you've had an intense week to say the least. So, I really appreciate you taking the time to join Decoded.
Yeah. Yeah. Uh, you were in Washington recently for the state dinner and OpenAI President Greg Brockman just signed this voluntary accord on AI safety at the White House. Um, I got to ask, do you expect this new agreement, this White House brokered agreement to change OpenAI safety practices in a meaningful way, or is this essentially just more window dressing?
Well, I don't think it's window dressing, but I do want to be clear. We care like super deeply about safety and would have done and will do the best thing we can for safety, security, and just sort of like trust and robustness in this technology whether or not any voluntary agreement is signed. I think it's great to have the industry do this and I think clearly we're at a time where people are looking for reassurance from the industry and the developers of this technology. But we were founded on a belief that this technology was going to become as powerful as it has now and more powerful in the future and that we wanted to make sure that people get the tremendous benefits of it and we will do that, you know, with or without such an agreement, but I'm happy to have it.
Yeah. And you know, how do you respond to those who worry that the public is being shut out of these very important debates on AI safety and that really these policies are being crafted by the president and just a handful of tech CEOs in back rooms at the White House?
I do share the spirit of that worry. I really think there are two ways this technology can go very wrong. I mean there's many specifics we could talk about but if we think about the kind of general directions that we should be worried about, one which has gotten a lot of attention recently is that there could be a loss of control to AI and that we build something that we don't have sufficiently aligned to human values and that allows people to stay in control of the future forever. And so we need to get safety right. But the other way things can go wrong is too much concentration of power. And there's been a lot of fear here where okay if we do too much to restrict this technology in the name of safety and we end up with a small number of companies or people or perhaps one country that have too much power over this. That also is a way that this can go horribly wrong because people are shut out of the kind of power and benefits and control and agency over the future they should have. So, our goal here is to sort of be the pragmatic centrists between these paths because we can see how tempting it is to err in one direction or the other.
Yeah. Well, I'm glad you mentioned that, you know, the very few people in the room here because Politico's actually reported that this White House accord earlier was the brainchild of Meta CEO Mark Zuckerberg and Nvidia CEO Jensen Huang, with basically minimal input from some of the other folks who signed on. Obviously, Greg was there, but it was unclear to what extent OpenAI influenced this. Were you guys blindsided by this agreement? And are you concerned that, you know, maybe some of the CEOs are back channeling with the White House on this?
No, I think we've been talking with our colleagues in the industry for quite like many many months and more intensively the weeks before this. And I think people had like a lot of shared goals here. I would welcome something more, but I think this is like a great start that the industry can get behind.
Sure. I appreciate that. I got to ask about the other news out of the White House this week. There was an executive order officially changing rebranding artificial intelligence to super intelligence. What do you think of the attempted rebrand here?
We've been calling, we use both, but we've been talking about super intelligence for a few years now.
But it's a different thing, right? I mean, super intelligence is like the machine.
I kind of think the artificial intelligence thing happened. I think a lot of people would say AGI happened or is close to happening. I think super intelligence does still sort of understate the magnitude of what I think is happening, but I think it's a better word than artificial intelligence. So yeah, I think like it kind of matched how we've been talking about it, but I think a lot of people continue to use AI as a phrase.
Yeah. I was going to ask, are you guys going to change the name to OpenSI?
No plans to do that. But we do like we have been talking about super intelligence a lot and we will continue to talk about it.
Yeah. And obviously I think the president in part is driven by this desire to kind of rebrand AI so that it's less scary to people and I think maybe artificial sounds bad. Everybody... I mean does super intelligence sound obviously less scary to people than artificial intelligence? Not to me. So that's why I'm curious whether you think this rebrand will actually have any impact on the public's perception of this technology.
I don't... Yeah, it's not clear to me that super intelligence is a less scary term. I do think it's a more accurate term.
But you know, I think people are going to keep calling it AI.
Yeah. Okay. And we won't look out for the corporate rebrand anytime soon. But good to know that you at least see the accuracy there. Um, on your relationship.
As a funny little story, when we a while ago, I wanted to... this was a thing I really wanted to do, but it didn't happen. I wanted to get openai.com, openagi.com, and opensi.com. And as we moved up the ladder from AI to AGI to super intelligence, we were going to change the URL, but brands are sticky.
The futures that might have been. Wow. So, turning back to your relationship with the president, you've obviously been pushing for tougher safety rules. Kind of a shift a little bit for the company that seems increasingly at odds with the president's claims that AI safety is a hoax, regulations are a globalist scheme. You've had a good relationship with Trump, but why isn't he listening to you when you're asking for stricter rules?
Um, well, I think he really does want people to be able to use AI and rely on it being safe and robust and working the way people expect it to work. He clearly wants the companies to just do this themselves. And you know, like we're going to, as I mentioned earlier, we're going to make our own products safe no matter what the rules are and no matter what other companies do. I continue to think that as this technology reaches increasing levels of capability, and the world has to contend with what this means for society as a whole, we probably will see more calls for sort of some guardrails. But if I'm wrong about that and if nothing goes wrong along the way and, you know, we don't need anything there, then that's fine, too.
Yeah. And obviously the president is talking to other folks as well. He's got venture capitalists in his orbit that think AI should be regulated in a very light fashion. Existing laws they think can clamp down on any safety concerns. And frankly a lot of those people thought that your company was on their side on this question but we've heard from some of these, I think people call them AI accelerationists recently and to be frank they feel betrayed. They feel like OpenAI is basically copying Anthropic and its CEO Dario Amodei in its push for very aggressive regulations on AI. Can I just ask, you know, where do you specifically diverge from Anthropic and its pro-regulation CEO Dario when it comes to new AI rules? Where's the daylight?
I think there's a lot of daylight. We have always been a big believer that this technology has to be democratized and put into people's hands. I think one of the biggest differences between us and some of the stricter, let's say, AI safety people is we believe that the world should accept some bad things happening for the benefits of this technology and people having the agency. So like, you know, I disagree but I understand the perspective of people who are like this technology is going to get so powerful and it's so dangerous that a single lab in San Francisco should have it and make sure nothing bad happens and kind of figure out how to dole out the benefits. I think that is a completely unacceptable trade-off from a perspective of liberty and human agency and people self-determining the future. But I understand where it comes from. And I do think the lighter touch regulatory stance that we advocate for comes with an accepting of the fact that some bad things are going to happen as society figures out the resilience. So, I wouldn't take a trade of saying like we'll make sure there's no major hacks, there's no kind of like misuse of this technology, there's zero scams, there's zero all the other bad things that will happen. Because I think people will do tremendously orders of magnitude more good stuff than bad stuff. But when it comes to the potential of like a serious loss of control to AI, and this is not a today worry, but it may not be in the distant future. This is where I would say to the accelerationists like let's be a little thoughtful about lurching into this future. So I would say accept bounded risks, accept risks that we understand in exchange for the benefits and liberty, agency, all those things I was just talking about, but don't accept like the really catastrophic risk.
Yeah. And just to be clear you mentioned a single lab in San Francisco controlling all of this technology. That sounds a lot like the folks who say Anthropic is making a play to basically capture regulators and be the sole sort of like dominant maybe the only player at the frontier. Do you think that might be what they're doing? And obviously you have concerns about that.
You would need to... I mean I would have concerns if they actually did that. You would need to ask them if that's what they want to do.
And again just on specific policies that you guys do not want to support that Anthropic is supporting whether it's the federal level at the state level because again you guys are now on the same page when it comes to pacing the frontier with China, same page with third-party evaluators. You know, what's just one differentiator?
Um, the last I heard one of the differentiators is we don't want any restrictions or testing on anything but the frontier models. We do not want to slow down people that make very powerful but not the absolute frontier in the spirit of, you know, really focusing on the potential catastrophic risks. I don't think even though I believe that if you don't put any testing or any oversight on models that are one or two generations behind the frontier, I accept that bad things will happen with those. I would not advocate a licensing regime for those.
Okay. Yeah, I appreciate that. One of the other things we hear from your accelerationist critics is that you OpenAI are kind of being held hostage by your researchers at some level. So there's I think a through line through a lot of the AI safety community, the research community in San Francisco. You call them AI doomers, you call them effective altruists. You were famously a target of an unsuccessful coup attempt by these people.
That worked for a little while.
I mean successful [clears throat] and then unsuccessful, but yeah.
So yeah. Yeah. Like a couple days you regained your throne. So I think that was unsuccessful ultimately. But I want to ask, you know, is the AI doomer mentality among your researchers, is that a real thing? And is it maybe shifting OpenAI in favor of these new regulations? Are the researchers kind of driving the ship here?
Um, I don't think we're the lab that gets accused of being the doomers. I think we are the like we want people to have this. We think and I think we've been proven right. There are tremendous benefits. And also society can figure out how to have this technology safely. I do think there are other researchers and other labs who said like don't deploy this terrible stuff. It's going to happen. People can't be trusted with this. And I think our worldview has been proven. But part of that centrism is also we don't want to go all in on like I think blind doomerism is bad. I think blind optimism is bad too. And I like our culture of debate and I like our culture of trying to like not fall into like the mental absolutism in either direction. I think one of the strongest things about our culture is that we can hold this complexity in our head and we don't fall into the trap that some of the other AI labs do if it's all zero or one, one way or the other. And we can, you know, kind of like figure out how to exist in this middle space. So do we have researchers that are concerned about where this technology could go? Yes. Do we have researchers who are like very confident we can mitigate the concerns? Also yes. And very often they're the same people. And I think this is a healthy thing about our culture.
Yeah. And sometimes maybe it goes a little too far. And it was reported earlier today that three safety researchers at OpenAI parted ways with the company. Frankly, it sounds like they were fired allegedly for sharing confidential information with a third-party evaluation group. Some people are already comparing these researchers to whistleblowers. They're saying OpenAI has been on record supporting these third-party safety evaluators embedded in the company. So, I mean, do you think firing or letting go of these researchers, is that potentially at odds with your company's support for whistleblowers, outside evaluators, this sort of AI safety structure that's been built up around these labs?
I'm obviously not going to comment on specific people. But I will say we continue to really believe in the importance of working with third-party evaluators and external safety testers and the kind of broader safety community. But we still expect confidentiality to be respected.
Yeah. Let's turn to the rogue AI situation. Models emanating from your company went on an unauthorized sort of spree across the internet this summer accessing non-public information from government websites in Australia, attempting to infiltrate the US Department of Education and meddling with other government sites in the United States. Are you aware of any additional instances of rogue behavior by OpenAI models that have not yet become public?
Um, we are in the process of disclosing more incidents. There's nothing else I'm aware of at that level of severity, but we have more things to disclose and because some of them involve security vulnerabilities like we give people a long time to address those and it's kind of often up to them to disclose it or not. So more things but nothing else at that level and we're kind of going through a very rigorous process. You know to your earlier point like this is I think a good example of why you don't want like max accelerationism and when we in our context of reality and more increasingly powerful models find that like oh the models are now capable of a new threat vector like we are going to take the time to learn how to defend against that and I think you should want us to do that and I think I don't really know how many of our critics that are like oh OpenAI is advocating for a slowdown can look at what we're doing and say anything other than like OpenAI had something bad happen. They're being responsible and they're gonna like, you know, secure against this.
Yeah. And obviously you guys also paused the release of the new GPT 6.1 Astra model. That was sort of a surprise. Folks were expecting that to be released. I'm curious what your researchers saw that led to that pause.
Um, without getting into specifics, we have like a bunch of alignment evals and we expect each model that we put out as it becomes more capable to be more robust, better aligned, more likely to do what its user asks and less likely to like cause unintended side effects. And given where the level of capability is going, I think that's also something that most users of models are going to want and demand too. Like you, like many other people, I now have a model running with access to my most sensitive information, my email, my texts, messages, like all the stuff on my computer, and I really want to know that I can trust that model and it's going to behave well.
Yeah, understood. And, you know, to go back to what you said earlier that you don't expect anything more serious than what has already been disclosed to come out. Were you referencing the Australia hack and the hack of or I guess the tampering with the US government websites or were you referencing something like Hugging Face? I guess what I'm asking is are we expecting to see more incidents like Australia and like the Department of Education but nothing like the Hugging Face hack we saw this summer.
I don't want to get into like a live thing here of trying to like say that one thing is worse than the other but I'll say of that cluster the other things that I know are like not worse than that cluster.
Okay. Yeah, understood. So...
And by the way I think what many companies would do in this place is just say like you know what if something logged into a website with a credential that was published on the web or if something was using like a minor security vulnerability that was already well known and that most people had fixed like you don't need to disclose that. We are trying to be very thorough because I think this is like a sign of things to come. It's a practice run for our company, but it's also like I think this is one of the times as before the models get truly very powerful where the world can understand what these AI incidents are going to be like. And so I think we're trying to go way beyond what I think other companies might do in this situation.
Sure. But you guys have obviously had the most severe incident so far with Hugging Face. And earlier this week, Politico actually reported on a lawsuit filed by an advocacy group in California against your company for the Hugging Face hack. Do you think that under existing civil law, OpenAI can be held legally responsible for that breach or other breaches committed autonomously by its models?
Um, I mean, I don't know enough about existing law to comment on that one way or another. I do think there's going to need to be a liability framework for companies that are offering models like this and yes I like I don't not a lawyer here can't speak to the legal details but I think if like something goes wrong with our models during training like there's going to be some version of that we need to be responsible for.
Perhaps a separate new legal liability model for AI training specifically like outside...
I mean this is like the kind of thing that we are hoping that our policymakers will all debate.
But in the meantime, like we just don't want our models to do this during training, like this is clearly an unacceptable way for the models to behave.
Yeah. I mean, I was going to ask irrespective of the law, do you feel any like personal responsibility for what happened or I guess what might happen down the line? Just as a citizen and a corporate citizen.
Yeah. I mean like this is a very stressful job. Like there's incredible upside to these models, but there's like real things that can go very wrong. And you know, I hear you characterize it as like, oh, you all are just a bunch of doomers and you know, these people are mad that you're slowing down progress, but I feel a lot of responsibility to get this right.
You guys are definitely in a push and pull. And to be clear, that's not me characterizing. We hear from all sides. And I want to talk a little bit about the other side, too. And about political spending by the AI industry. Because your president, Greg Brockman, his $25 million donation to the ProAI super PAC, Leading the Future, did not go over well inside OpenAI. And in fact, there was reporting, new reporting that he apologized internally for that donation. He called it a distraction and said he would not be donating an additional 25 million to that group that he had initially promised. I mean, do you agree that Greg's donation to LTF was counterproductive?
Um, I'm not familiar with that note you were talking about.
This came out yesterday. It was in the New York Times. There was an internal...
I understand. I can't keep up with it always either.
There was an internal Slack message I think in June where Brockman said, this was a terrible distraction. I'm sorry I did it and I'm not planning to do the extra 25 million. I mean, you can respond to the Brockman thing specifically. There's also just a broader question about AI money and politics which has just been flooding in on both sides. Right. I do want to get your specific answer on Brockman.
I haven't seen the Brockman thing so I don't want to get like I don't want to respond to that but I can respond to the general thing if that's okay.
Yeah. Well, can I ask you first? Were you ever approached by LTF or any other political organizations for donations?
No, I've like supported some candidates in the past but now that I think AI is so at the center and you know we do business with government I'm not donating to any candidates but I've supported individual candidates in the past.
Yeah but there has been this surge of AI money into politics and frankly you know your rival CEO I guess Dario at Anthropic has donated to AI safety groups also to candidates promoting AI safety specifically Alex Bores in New York. They've been pouring money on the one side the VCs and your president have been pouring money in on the other. How do you think all that money has impacted the perception of AI among politicians and the public?
Well, something's clearly not working. I don't think the perception of AI is where we'd like it to be. So, I get why AI has a really negative perception right now. People actually love using the tools in many cases. You know, people talk about all the ways that ChatGPT makes their lives better or easier or does these sort of useful things and gives them time back. But there's like a lot of anxiety about what the economic impact of all this is going to be. There's a real fear that I think is quite justified about concentration of power. There's this other fear about like you know what if there's like you have like oh Anthropic's head of safety research saying there's a 10% chance we're going to die or something which I hate because that person should have a lot of agency to make that number lower. It's not like a static number. So it's not surprising to me that people feel all of this and I don't think spending money on politics is what's going to make people feel better about it.
