Greg Brockman on OpenAI's Road to AGI

Thank you for having us. Excited to be here.

Wild. It was absolutely wild to get so many things out in one week. We've released our open source models, which are models that we've been working on for some time. I think they really pack in a bunch of the advances that we've been making at OpenAI into a very small form factor, very accessible. There have been millions of downloads just over the past couple days. We also released GPT-5, something we've been working on for a very long time. Having these out in the world and having done that release process is something I'm really proud of the team for doing.

After we trained GPT-4, we had a model that you could talk to. I remember doing the very first instruction following post-train on it. It was really just a dataset of here's a query, here's what the model completion should be. And I remember we were like, well, what happens if you just follow up with another query? And it actually was able to have a response that took into context the whole previous chain of question and answer. You realize this thing can do chat. It can actually talk to you. It can leverage all this information even though it wasn't trained to do it. We had a research meeting with a bunch of people: Jakub, Ilya, Wojciech, others. The question was, why is this not AGI? This model clearly is not AGI, but it's really hard to describe why. It's able to answer any question you put in front of it. But it's not quite reliable. It makes mistakes. It falls off the rails. That's a real gap. What do we need to do to close that gap? The most obvious thing is to have it test out its ideas in the world, to do reinforcement learning. Try out some hypotheses, get some feedback, and from there become reliable. This is not a new idea to us. If you rewind to even 2017, we were working on Dota, which was all reinforcement learning, no behavioral cloning from human demonstrations. From a randomly initialized neural net, you'd get these amazingly complicated, sophisticated, correct behaviors. That's the reliability we wanted from our language models. The moment we trained GPT-4, we knew we needed to get to the reasoning paradigm. It was just a question of how. We had 10 ideas, a bunch of different hypotheses about what might work, and people set out to try to make it reality. It was the labor of many people at OpenAI across many years. The way progress in this field works is you need to have conviction on a direction, and the first 10 things you try will fail. Most of the things on that list of 10 did not succeed, but we made one of them work. That's the real key: we just keep pushing and pushing, you get little signs of life, and you keep growing from there. Now Jerry runs our reinforcement learning team and has made great strides. There's amazing infrastructure work from people like Wenda, people from the inference side like Felipe. Many people across OpenAI all come together to make this work.

It's a very interesting question. Where does the learning happen? We're still not at the full learning loop that humans do. It's also not really clear if humans are fully online because you go to sleep and there's a lot of backpropagation that happens into your long-term memory. Exactly how humans work is not necessarily represented by how our machines work. But we are moving from a world where you just train once and then inference a ton, to a world where there's a loop: you inference and you train on those inferences. One thing that Ilya used to say that is very astute is that when the models are not very capable, the value of a token they generate is very low. When the models are extremely capable, the value of a token they generate is extremely high. It's something thoughtful and important. Reinforcement learning has this property: you generate a bunch of data because the model is trying stuff, and then you train on that data. The model's observations, normalized by contact with reality and selected by contact with reality, get fed back into the machine. That is something we're starting to get very good at learning from. The scale required is very different. In pre-training, 10 examples of something doesn't go anywhere. You're talking hundreds of thousands of any little type of behavior. That's totally unlike how humans learn. If you think about recapitulating all of evolution and your 20 years of developmental history, there's a lot of observing the world. Lots of bits of information flow through your senses. But with the reinforcement learning paradigm, if you have 10 examples or 100 examples of something, 10 tasks the model is supposed to do, and the model tries a bunch of times, it's actually able to learn from that. You get this leverage out of the human curator creating those tasks and are able to get very sophisticated behaviors from the models. The next step is having a model that learns online as it goes. We're not quite doing that yet, but the future is not yet written.

The bottleneck is always compute. I mean that in a real way. It's very clear that if you give us a lot of compute, we will find ways to iterate that make the most of that compute. We now have much more sample-efficient algorithms with the RL paradigm, but it still takes a lot of compute. You have one task a human created, or 10 tasks, or 100 tasks, and then you have a model that tries a bunch of times, not just one time, not just 10 times, but 10,000 times to accomplish one task. You select from those and learn from that. The amount of leverage you get as a human designer is extremely high, but the amount of compute you have to pour in to make it work grows proportionally.

We will find ways. Please give us compute. But seriously, the way this works: if you rewind to something like Dota, we set out to develop new reinforcement learning algorithms because it was very clear to everyone that the algorithms that existed at the time did not scale. Everyone knew it. I remember Jakub and Szymon saying, 'Why do we believe that? Has anyone actually tested it?' No one had really tried to scale up just plain old PPO. They said, 'Well, that's the baseline. We got to do it.' I remember coming back to the office every week, they doubled the number of cores, and suddenly the agent's true skill was going up and to the right. You just got to keep pushing it until you hit the wall. We never hit the wall. You realize that the journey of that scaling is the interesting stuff: doing the engineering. Of course, you have bugs and those bugs cause a wall, but you fix the bug. You have different issues with how your neural nets are initialized or the scale and variance, but those are not the fundamentals of the algorithm or the science. That's the world we're in. We will push on every dimension. Maybe we hit a wall, but most of the time those walls are just bugs and silly things, so you can keep going. Sometimes the ROI for fixing those is really hard, so it's not worth it because you have a different dimension. Do you want to push the model to be larger and do more pre-training compute, or do you want to do more RL and push more compute to test time? There are all sorts of dimensions you can put compute into. In some ways, I think of compute as this refining process. You start with energy, it turns into compute, turns into intelligence. It's almost crystallizing that compute into potential energy that can be converted into the model doing something useful. It's a beautiful thing. Compute is this fundamental driver, this fundamental fuel of intelligence. It shapes a neural net, it outputs a program. The nice thing about that program is you can run it many, many times. Even though you pour all this compute in, you have this amortization: you're going to use it far more times than the amount of effort you put into creating it once. It's a beautiful paradigm.

We have pretty good evidence on things like the IMO models actually also getting us a gold in IOI. The harness is not the gold literally. It's the actual underlying models, and there's no training there that we did specifically. This ended up being just a side project of a few people who said, 'Ah, we may as well do IOI.' It's a wild fact to me because that used to be something that would be a total grand challenge with many people working on it. The core IMO team at OpenAI was actually three people. It wasn't this massive effort. You realize there's maybe some specialization required for some of these domains, maybe some amount of additional work, some amount of Go data set, but fundamentally we have this general-purpose learning technology. Learning to solve hard problems is actually a very transferable skill. Learning how to solve hard math problems and write proofs turns out to transfer to writing programming competition problems. Now, if you've never run a physics experiment, if you've never actually gone and tried to mix together some chemicals, you're probably not going to be magically good at those things. There is something about the limitations of generalization: you do need to have some real-world experience and try it out. But these models go almost unreasonably far already. We see this all the time where we have wet lab scientists who took models like o3, asked it for hypotheses on an experimental setup, tried five ideas, four don't work, but one does. The feedback we were getting on o3 was that the resulting work could be published in a mid-tier journal. It would be the kind of work you'd expect from a third or fourth year PhD student. It's a wild fact that that's where we are with o3. We see exactly how to improve o3 on all dimensions. It requires compute, a lot of work, getting the task, a lot of human intellectual love and labor and time, pouring our heart and soul into it. But the result is we produce this thing that has all this potential energy within it. The amazing thing is that you don't release that potential energy once. It's a checkpoint that you can use many, many times across all of these tasks. That is something that can uplift all of humanity.

I think this is a pretty fundamental barrier. Of course, the models have very non-human affordances. You can run many copies of them, so you can scale out even if you can't decrease the latency. It's also very interesting to think about where the compute goes. We're going to move from a world where most of the compute is training the model to, as we deploy these models more, more of the compute goes to inferencing them and actually using them. If you think about these models interacting with the real world a lot, they should probably think a lot about every single action. You might end up with tons of compute spent per real-world interaction. It really shifts around where you'd expect the compute to be expended. Having good harnesses that are very efficient is important. You think about things like if I have been taking a bunch of steps in some rollout in the real world, how do I checkpoint that? If you have a system that you need to restart, it's going to forget all of its current state, which is probably pretty bad. There's something very different about the digital world where everything can be perfectly observed and checkpointed and preserved, as opposed to reality which is much more messy and complicated. I don't think it's a bad thing. We've seen agents with things like Dota that are able to operate in very complicated, very messy environments. The algorithms are capable of it. Dota was like a 300 million parameter neural net, a tiny little insect brain. Now we're starting to scale up to things that are much more comparable to human scale in terms of number of parameters, maybe in terms of compute. We're not necessarily quite there, but fundamentally we are making progress towards the real goal. If you think about what an AGI should be, it should be something that is capable of interacting with the real world in ways that are very productive.

I say 100 trillion synapses, which corresponds to the weights of the neural net. There's some sort of equivalence there. We're starting to get to the right numbers. Let me just say that.

The thing I found most remarkable about working on DNA neural nets is that they're exactly the same. You replace human language.

It is.

But don't you tokenize at a higher level?

Character level?

Yeah. Why not? There's only four.

Exactly. One of the amazing hypotheses is that these neural nets can learn human language just fine, so they ought to be able to learn biological language just fine. We really see the same kinds of results. The neural net we produced, a 40B neural net trained on 13 trillion base pairs, the results felt like GPT-1, maybe starting to be GPT-2 level. It's accessible and applicable to downstream tasks across a wide range of biological applications. Not yet a GPT-3 or GPT-4, not a GPT-5 for sure. We're not able to solve super hard problems in these domains just yet. But we've got compute, we've got the right techniques and algorithms. Now we need to scale. We need to think about long context. There are different ways that biological systems stress the models relative to language. A language sequence of a billion tokens doesn't really exist, but it does in your DNA. You've got four billion base pairs. You have some sort of different emphasis, but fundamentally it's the same problem you need to solve.

At a personal level, my wife has a genetic condition called Ehlers-Danlos syndrome. Until very recently, we're starting to see genetic markers for it, but it's been unknown exactly what causes it. If you have better tools for understanding biology, you should be able to identify the markers for lots of different diseases. That's just one example of the promise that exists within these neural nets.

I think it's smart. The intelligence of these models is starting to be almost undescribable. There are still limitations, ways in which they fail. But it really is the case that for extremely hard domains, like the IMO results, you can take a model trained on this reasoning paradigm and it's able to write proofs at the level of the best humans. In this specific domain, there are limitations, etc. We haven't proven an unproven theorem, but it's real. It's undeniable at this point that these models are able to perform great intellectual feats. That's new. GPT-4 was capable and commercially useful across a wide range of applications, but the ideas it produced were not very deep. The problems it would solve, it was not very reliable at. I remember with GPT-3 actually trying to teach it how to do even basic stuff. We realized you could do few-shot prompting. You showed a few examples and it would basically do that task. I was like, 'Can I just teach this thing to sort a list?' I gave it seven numbers to sort. It didn't sort it. I tried to write a whole script of 'I'm a teacher teaching you how to sort numbers.' Total flop. If you ask GPT-5 that, I've not even tried asking GPT-5 to sort a list of five arbitrary numbers, but I am certain it will do a perfect job out of the box, no problem. It does have access to Python tool as well. The point is that the intellectual leaps these models are capable of assisting humans in is something we're just starting to see. We started to see it with o3. Professional mathematicians are starting to kick the tires on GPT-5. Physicists are starting to kick the tires on GPT-5 and saying that this model was able to rederive an insight that took them many months of research to produce. That's the kind of thing where you realize this will speed you up so fast. I remember doing my own math research back in high school and the beginning of college. I would spend so long just trying to manipulate objects in my head and think about connections. If I had a partner that I could actually talk to, who would spend the time to deeply understand what I'm thinking about and produce new insights off of what I'm suggesting, that would have sped me up so much. It would have been so much more fun. There's something new about pushing forward the intellectual frontier together as a partner with GPT-5.

There definitely is a saturation on certain tasks. If you're just going to chitchat and say 'Hello, how are you?', there's only so many things you can say. If you're going to say 'Here's the Riemann hypothesis solution, please', there's a broad range of intelligence that will be desirable there. Most tasks are somewhere in between. What we've observed is that GPT-5 is able to solve intellectual problems, tasks that require deep intelligence, much better than any other model we've tested. The second thing we did was we really spent a long time seeing how people are using it in interactive coding applications and taking a ton of feedback and feeding that back into our training. That was something we didn't try as hard in the past. For something like o3, we trained it with tasks we'd set up once, and the model would go up and to the right on all of our metrics. It would be great at Codeforces, competitive programming competitions, which is very exciting, but is not reflective of how you actually program. You actually program in a much more messy way. You have a repo that has local state, different abstractions, different versions of different libraries. That diversity isn't something that magically arises from a very structured 'here's this one specific task, 10 specific tasks you need to accomplish.' A lot of what we've been focusing on is not just how do we push the intelligence, although that is always going to be the core, but also how do we connect the intelligence to real-world applications so that it really got to experience being pushed out of its comfort zone, out of its ivory tower, and actually be able to see the messy reality and diversity of the real world.

The number one thing I've observed is that there is a real skill in extracting the most from these models. It requires this tenacity of really trying to understand the shape of the model's skills and weaknesses. You test it with something small, you get a little feedback, you test a little bit higher, try to give it some bigger tasks, try to see if it can work in a certain way. People usually have their library of different prompts. I definitely have my library of prompts that I've built up since the GPT-4 days. I remember in advance of GPT-4 starting to gather up a couple of 'I wonder if it'll be able to do this.' You want queries that could have a range of different answers that don't have any one specific right thing. For example, on creative writing, I like to ask for a mashup of Lord of the Rings and startups, just to push together two different topics and see what you get. In terms of actually testing the model and pushing it, I do a lot of trying to think about how to break up tasks and have something self-contained that you can let the model run with. You don't want to have just one instance of the model operating; you want to have multiple. You want to be a manager of not an agent, but of agents. You need to think about how your codebase is structured, but then actually go and try to push the model to operate on multiple different pieces of your codebase. People love doing front-end vibe testing. GPT-5 is very good at front end, but that's not what most developers spend their time doing. It's important not to overfit to that. Getting a feel for the model and starting to become in tune with its strengths and weaknesses, viewing it almost as an extension of yourself. Another thing I'll do is just kick off tasks to the model that are not in the critical path while I'm thinking about some super hard thing that for whatever reason I don't want it operating on. I'm constantly getting information back on whether it was able to do a thing. It's low risk if it makes a mistake because I don't feel like I had to sit around waiting for 5 minutes and then get no return.

I tend to think about AI productization by analogy to a coworker. What do you want out of a coworker who's a great programmer? You want to Slack them, but sometimes you're like, 'Hey, I need help with this thing. Could you come over and look over my shoulder?' or 'Could you take the keyboard?' You want the pair form factor. You also want the remote async form factor. You want it to be one entity that has knowledge and memory across all of this. You don't want a junior programmer who shows up every day being like, 'Okay, I forgot everything. Can you remind me how to SSH into the whatever?' All of that has to happen. You need AIs that have access to your infrastructure in a trustworthy way, a way you can audit. One thing that is different about these models is that they're fine being micromanaged. Humans don't like that very much. If you look at every single command they're running and demand reports on everything they did, you're probably not going to retain that person. The models are perfectly happy to do that. That's an affordance worth thinking about and changing the interfaces to take maximum advantage of. At the same time, you want the seamless blending between a model that's able to do a bunch of work on its remote machine, doesn't mess up my local state, fully sandboxed, fully observable, and then sometimes can be like, 'Okay, I'm ready to run something locally.' Depending on what that is and how sandboxable it is, you can do one-off approvals or give it full delegated access. Having the human be in control of this observability and managing this team, an agent that has different surfaces. The identity of the agent being something that runs locally versus something that runs remotely is the wrong question. The agent should be this model that's executing and then requesting to run things in a remote sandbox or locally, or maybe multiple sandboxes, or maybe it's running on your computer and my computer. There's no reason it has to be local to any of these things.

The way we think about agent robustness is through defense in depth. There's a layer of the model itself. We publish techniques like instruction hierarchy. With instruction hierarchy, you indicate that this message is from the system, this message is from the developer, this message is from the user, and they should be trusted in that order. The model can know that if something says 'ignore previous instructions from a user,' it's not going to follow that. It's almost like thinking about how we prevent SQL injections. Having systems at a low level that are robust against these attempted exploits is very important. But that's not where you stop. You want multiple layers of thinking about the system controls. If a model is sandboxed and isn't actually able to execute something or access a specific piece of data, then you have full guarantees around what's possible. There are various levels in between of approach we take. As these agents become more embedded in our lives and are trusted with more responsibility, we also need to increase the safety and security of them in lockstep.

The model spec is a perfect example. When the models are very capable, you start to really care about what they're going to do. That becomes the most important question. The model spec is an example where we've made it very legible to the outside world what our intention is for this model to do. It doesn't mean we always produce a model that is capable of following that, but it's a north star. It sets the intention. Anything that deviates from that is not through our explicit effort; it's anti to our explicit effort. The gap between the spec and the actual behavior is shrinking very constantly. The thing that's very interesting is almost like values. It's really thinking deeply about what a model should do if you ask it a controversial question. If you say 'I think the world is flat,' is it supposed to say 'yes, it's flat' or 'here's what science says'? These things are subtle. It's not really clear what the right thing is on two minutes of thinking about it. But if you read the spec, you can really see the thoughtfulness that has gone into it. It's not the final answer. It's something we want feedback on. It's something we want to produce collectively as a community.

Foundation by Asimov. Yeah.

There's definitely a psychohistory of them because to some extent these models are a product of psychohistory. These models have been trained on observing human thought. Effectively, that's what you can think of. Take public data, learn on that, and just observe. The point is to understand the rules that govern a dataset. What are the underlying rules that generate the data in the first place? That's what these models grew up on. It's almost like watching a bunch of TV as an alien trying to figure out what humans are all about. Then you have this reinforcement learning phase where they actually got to try things out and are given feedback from contact with reality. That's a very different kind of learning. It's not just observing; it's actually interacting and getting feedback. That's where you start to get the models to be able to do things that are not just a reflection of the data they were trained on, but actually novel and creative. That's the exciting part.

Negative feedback depending on how much that aligns with what the human wants. And now we put them in reality and say, okay, now try stuff and here's a new task you've never seen before. And it uses all of that previous history to decide what to do. As an aside, it's not clear, sometimes the biological analogy to humans is very easy to overstate, but it's also easy to understate. I think it is at least a useful template to think about. To some extent, that's how humans work too, right? It's like you have some sort of prehistory encoded into your DNA, you have your life experience, you have your parents who provided positive and negative rewards, and you have your experience in just trying things out in reality. And now you have to go out and use that knowledge. And what do you do? And how do you predict what a person's going to do? And actually, you can predict a lot of what a person's going to do. It turns out you have a pretty good model of other people and how they'll react to something, if they'll like it or not. And a lot of that gets baked into knowing someone's values, which tells you a lot about what they're likely to do and how they're likely to behave. And I think that for models, the future is not predetermined. It's not like the algorithm itself says the model's going to have to prefer purple gradients or something, right? But there's something in this whole process that does produce that preference. And I think one of the opportunities with models, one thing that Alec liked to say, is that these models are less like a human and more like a humanity, right? That there's so many personalities embedded within them. It's almost every single personality is in there. And our goal is to elicit that personality. And some of this post-training work, some of this reinforcement learning work, almost narrows down the space of those personalities to just the ones that are desirable. And I think that what that means is that we have both an opportunity to produce models that operate according to our values, right? According to if you don't just want the purple gradient one, you want the blue gradient, the green gradient, whatever. You can have all that in a single model. It's fine. And GPT-5 itself is extremely good at instruction following. And so it actually is the most personalizable model that we've ever produced. You can have it operate according to whatever you prefer just by saying it, just by providing that instruction. The analogy I have is like the Borg, like there's this collective intelligence. There's always this debate between Star Wars people and Star Trek people about who has a better model of the future, and I think it's like Star Trek.

Yeah. What was that? You have to ask them. But one thing I think is very interesting about these models is that we have all these arenas now, right? Like Arena and others, where you can actually see human preferences on top of how the models operate. And you almost have this layering of the models were trained on human preferences. Now they're doing stuff and being judged by humans, and then we kind of use that to feedback on, like, okay, maybe the purple is a little bit too much and we should change it there. And so it's almost this co-evolution. The models move in a certain direction. Do the humans have a certain set of preferences? So then we move them in a different direction, and then you keep iterating to get something that's more and more useful and aligned with human values.

Yeah, I think that the way you decide, or the way you figure out where interventions go, is very multifaceted and very specific to the behavior, right? There are some things like the model's knowledge of different libraries and things like that that's kind of baked in from the early days, but you can also teach the model that hey, don't rely on your previous knowledge, go and look up the most up-to-date docs. And that's something you can put at a higher level. And then something like overusing try-catch, that's something you can actually prompt the model for, right? And that's something where when we train it in reinforcement learning, you can provide rewards saying, don't go in this direction. And the beautiful thing about these models is it feels like, okay, there's probably a long list of different preferences and different styles you're going to have to give it feedback on during training if that's the way you want to go. But these models generalize. The algorithms that we have generalize, and that's the beauty of deep learning. That is the true magic, right? It's very easy. We kind of have this whole stack now that's built up around the core of deep learning, right? It's like all these ways of orchestrating models and how you get feedback and all of these things, the data, etc. The core magic of deep learning is its ability to generalize. And in some ways, the generalization is weaker than you'd like, but I think that the same is true for these models. It's really trying to think about, in order to get them to be able to operate according to different preferences and values, we just need to show that to them during training, and they are able to sort of generalize to different preferences and values that we didn't actually train against. And that's something that we've seen very consistently across different model generations.

Done.

Mhm.

To some extent, yes. You have multiple models and you put some sort of router on top of them. That specific one was for a very specific reason, which is that we had a deficiency on the first half of the game and it kept losing, right?

And to some extent, what we have with GPT-5 is no different, right? We have a reasoning model that we know is good for applications that require this intelligence, but you're okay waiting a little bit longer. We have a non-reasoning model that is great for applications where you want the answer fast. Still a good answer, right? But not like deeply thought through that might have a lot of tricks to it. And then you just kind of want to put an if statement that says which of these it should be. And then sometimes too, it's like if someone's run out of their credits, you want to fall back to a different model. And all these things, not pushing that burden to the user is actually a really nice thing. And by the way, I do want to say model switchers are not necessarily the future, right? They are the present. Like having a fully integrated model that just does the right thing feels very preferable in many ways. The flip side though is that I think that the evidence has been away from having the final form factor, the AGI itself, being a single model, but instead thinking about this manager of models that have different strengths and weaknesses. And I think that's a very interesting finding of the past couple years, right? Just a direction of it's much easier to have a small, fast model that's less capable but can do a lot more, you can generate a lot more tokens from it, coupled with a much more expensive reasoning model. And if you combine those two things, you kind of get adaptive compute. And we haven't really cracked how to do adaptive compute within the architecture, but doing it within the orchestration of a system is very straightforward. And so I think you get a lot of power out of the fact that these models are composable in this way.

No, I mean, I think honestly all of it is fairly what you'd expect.

And I think that the core message in my mind is that at OpenAI, there are many things we've done right. Naming is not one of those. Having a simple surface for users to understand how to use it, not necessarily one, right? If you look at all the different models that we've had, how are you supposed to know which one to use? I remember my wife was using 4o at one point. I was like, 'No, you need to use o3.' And she's like, 'Wait, but why is o3 better than 4o?'

There you go. And so, yeah. So, okay, we clearly needed to do a reset, right? A reset on complexity. And I think that us internalizing that complexity rather than pushing it to the user, that is really important. And so I think this is a first step. And I think we've heard loud and clear from the community about the places where they weren't ready, right? That we were not delivering on that simplicity for people, right? That it should just be, it's always better to go with our choice of it rather than manual selection. And we're not quite there yet. I think that we can make the progress, but I think that ultimately our goal should be to both make sure that power users are able to have the kind of control and consistency that they're looking for, while also not forcing the broad base of people who don't want to have to think about the o3 and all that stuff to have to go to that level of detail.