Sam, Jakub, and Wojciech on the future of OpenAI with audience Q&A

Thanks Sam. At the core we are a research laboratory focused on understanding a technology called deep learning. And so a particular focus of ours is understanding what happens as you scale up training deep learning systems. And one consequence we discuss a lot there is AGI, artificial general intelligence. But we find that in some way even this maybe understates a bit the magnitude of the possible progress and change here. And so in particular we believe that it is possible that deep learning systems are less than a decade away from super intelligence. So systems that are smarter than all of us on a large number of critical axes. And this is of course a serious thing, right? There's a lot of implications of this to grapple with. And one particular focusing impact of this technology and the technologies leading up to it, and something that we organize our entire research program around, is the potential to accelerate scientific discovery, to accelerate the development of new technology. We believe that this will be perhaps the most significant long-term impact of AI development, and it will fundamentally change the pace of progress on developing new technologies. And so thinking about how far along we are towards these goals, one good way to think about progress is to look at the time horizon that it would take people to accomplish the task that the models can perform. And so this is something that has been extending rapidly over the past few years. So where the current generation of models is at right now is about five hours. So you can see this by looking at the models matching the best people in competition such as the International Olympiad in Informatics, and we believe that this horizon will continue to extend rapidly. This is in part as a result of algorithmic innovation and in part just scaling deep learning further, and in particular scaling along this new axis, in-context compute also called test-time compute, where we really see orders and orders of magnitude to go. So this is roughly how much time the model spends thinking, and if you look at how much time the model's currently spent thinking about problems, and if you think about how much compute, how much time you would like to spend on problems that really matter such as scientific breakthroughs, you should be okay using entire data centers. So there is really quite a way to go there. And anticipating this progress, we of course make plans around it internally and we want to provide some transparency around our thinking there. And so we want to take this maybe somewhat unusual step of sharing our internal goals and goal timelines towards these very powerful systems. And you know, these particular dates we absolutely may be quite wrong about them, but this is how we currently think, this is currently how we plan and organize. So as a research organization that is working on automating research, naturally we are thinking about how does this impact our own work and how will AI systems that accelerate development of future AI systems look like? How can they empower research like alignment? And so we are making plans around getting to quite capable AI research interns that can meaningfully accelerate our researchers by expanding a significant amount of compute by September of next year. So we believe that is actually quite close. And then we look towards getting a system capable of autonomously delivering on larger research projects and a meaningful fully automated AI researcher by March of 2028. And so of course as we look towards these very capable systems, we think a lot about safety and alignment. In fact, a lot of our work both on deployments and safety, but also just on understanding deep learning and development capability side, we can think of preparation for these very capable models. Safety is a multifaceted problem and so the way we generally structure our thinking are these five layers ranging from factors that are most internal to the model to ones that are most external. And so at the core, what we believe is the most important long-term safety question for super intelligence is value alignment. So to put this, value alignment you can think of as what is really the thing that the AI fundamentally cares about? Can it adhere to some high level principles? What will it do if it's given unclear and conflicting objectives? Does it laugh at humanity? And the reason we believe that this high level objectives or principles driving the AI are so important is that as we get to the systems that are thinking for very long, as they become very smart, as they tackle problems that are at the edge or perhaps beyond human ability, really getting to complete specifications becomes quite difficult. And so we have to rely on this deeper alignment. Then there's goal alignment. Does the agent interact with people? How does it interact with people? How does it follow instructions? Then reliability. Can the AI correctly calibrate its predictions? Can it be reliable on easy tasks, express uncertainty on hard ones, can it deal with environments that are a little bit unfamiliar? Then we have adversarial robustness which is very related to reliability but it's about adversarial settings. So can the AI withstand targeted attacks from human or AI adversaries? And then the outer layer is systemic safety which are guarantees about the behavior of the overall system that do not rely on the AI's intelligence or alignment. So for example this can be security or what data does the AI have access to or what devices it can use. And so we invest in multiple research directions across these domains. And we have seen quite a lot of progress also come from just the general development and improving understanding of deep learning as a whole. And I want to take a slightly deeper technical dive here and talk about a particular direction. Value alignment is a hard problem, right? It's definitely not solved yet. However, there is a new promising tool that aids our study of it, and that is chain of thought faithfulness. It's something we invest in very heavily. Starting from our first reasoning models, we've been pursuing this new direction in interpretability. And the idea is to keep parts of the model's internal reasoning free from supervision. So don't look at it during training and thus let it remain representative of the model's internal process. So we refrain from guiding the model to think good thoughts and so let it remain a bit more faithful to what it actually thinks, right? And this is not guaranteed to work, of course, right? We cannot make mathematical proofs about deep learning. And so this is something we study. But there are two reasons to be optimistic. One reason is that we have seen very promising empirical results. This is a technology we employed a lot internally. We use this to understand how our models train, how their propensities evolve over training. Also we have had successful external collaborations on investigating the model's propensity scheme for example. And secondly, it is scalable in the sense that explicitly we make the scalable objective not adversarial to our ability to monitor the model. And of course an objective not being adversarial to the ability to monitor the model is only half the battle. And you know ideally you want it to help with monitoring the model. And so this is something we're researching quite heavily. But one important thing to underscore about chain of thought faithfulness is it's somewhat fragile. It really requires drawing this clean boundary and having this clear abstraction and having restraint in what ways you can access the chain of thought. And this is something that is present at OpenAI from algorithm design to the way we design our products. So if you look at the chain of thought summaries in ChatGPT, if we didn't have the chain of thought summarizer, if we just make the chain of thought fully visible at all times, that would make it part of the overall experience. Over time it will be very difficult to not subject it to any supervision. And so long term we believe that by preserving some amount of this controlled privacy for the models, we can retain the ability to understand their inner process and we believe this can be a very impactful technique as we move towards these very capable, long-running systems. And I'll hand back to Sam.

Hello, glad to be here. Thanks for being here. So the term AI resilience is a little bit broader than what we historically thought about AI safety. In the case of resilience, we think that advanced AI comes with risks and disruptions and we would like to have an ecosystem of organizations that can help to solve a number of these problems. So let me give you an example to better illustrate it. We all believe that AI will advance in biology and as it advances in biology, there is a risk that some bad actor could use AI to create manmade pandemics. So on the safety level, the mitigation would be to make sure that the models block the queries that have to do with virology. However, if you consider the entire AI industry, it's very likely that even if OpenAI blocks it, someone could use different models out there and still produce pathogens. And in the case of resilience, we don't want just to block it but also have a rapid response if the problem would occur. So when I think about the risks and disruptions, there are just many: mental health is one of them, bio is another one, job displacement might be another one. And we think that we need the ecosystem. Maybe a good analogy that I like is cybersecurity. So at the beginning of the internet, it was a place that people didn't feel comfortable putting their credit card numbers because it was so easy to get hacked. And when there was a virus, people were giving each other a call to disconnect the computer from the internet. And we got a long way. At the moment, there's an entire infrastructure of cybersecurity companies protecting the critical infrastructure, governments, corporations, and individual users to such an extent that people are willing to put the most personal data online, to have life savings be online. So cybersecurity got really far and we think that something analogous will be present for AI, that there will be an AI resilience layer. And I'm really excited that the nonprofit will help out to stimulate it, to create such an ecosystem.

One thing that we are quite hopeful about in terms of what we optimize for in products like ChatGPT or Sora is thinking about optimizing for the very long term, which is naturally very aligned with how we think in general about extending the horizon on which the models can work productively. And so we believe that quite a lot of development is possible there and we can eventually get the models that really optimize for long-term satisfaction and well-being instead of just short-term signals.

So I think in some number of years we'll look back at these years and we'll say this was kind of the transition period when AGI happened. I think one way we thought about it, as some said early on, we thought about AGI kind of emotionally as this thing that is the ultimate solution of all the problems and it's this single point for which there is before and after. And I think we found that it's a bit more continuous than that. And so in particular for various kinds of benchmarks that seemed like the obvious milestones towards AGI, I think we now think of them as indicating roughly how far away we are in years. So if you look at a succession of milestones such as computers beating humans at chess and then at Go and then computers being able to speak in natural language and computers being able to solve math problems, they clearly get closer together.

And maybe one other thing to mention, right? I think one kind of counterintuitive thing here is that obviously we're working with a pretty complicated technology, we're trying to understand all these algorithms, and maybe initially we imagined that AGI is the moment once you have figured out all the answers and it's the final thing. And I think now we increasingly realize that there is some curve of intelligence, maybe a multi-dimensional one, and humans are somewhere on it. And as you scale deep learning, as you develop these new algorithms, eventually you inch closer to that point and eventually will surpass it, and already have surpassed on multiple axes. And that doesn't actually mean you have solved all the problems around it, which is something we need to seriously think about.

I think we have quite strong expectations for our next models. So I think we expect quite rapid progress over the next couple of months and a year.

One thing to mention on the chain of thought faithfulness that I talked about earlier, we actually have started talking about establishing industry standards and we started some joint investigations with researchers from Google and Anthropic and some other labs. And yeah, that's something I'm very excited about and I think that is an example of something where we can really benefit from collaborating across multiple labs.

Yeah. So definitely there is a problem where we aim to lay out the high-level policies and guidelines for the model in the spec that we develop for ChatGPT. But the space of situations you can find yourself in is enormous. And at some point, establishing the right boundaries really becomes a tough intelligence problem. And so we are seeing improved results on this metric from reasoning models and from expanding more reasoning on thinking about this software questions and trade-offs. Of course, this is a bit more difficult to train for than math problems, for example. And so this is something that we're researching quite heavily.

Yeah, we are seeing quite a lot of ability as we get to the new frontiers of intelligence to reduce the cost for that quite quickly. And so especially with reasoning models, we've seen that actually quite cheap models when allowed some additional test-time compute can become much more capable. Yeah, and so this is something that I expect will continue. And so as we talk about getting to these new frontiers and automating research and so forth, I expect that the cost of a lot of that will keep falling quite a lot too.

So I would say as a research lab, we haven't set out to build a chatbot originally, although I think we've since come to appreciate how aligned this product is with our overall mission. And we of course expect ChatGPT to continue to become better and be this way for people to interact with increasingly advanced AI. But we do anticipate that eventually AI systems will be capable of creating valuable artifacts, of actually pushing scientific progress forward as we were discussing, and I believe that will be the real lasting legacy of AI.

Next year I think is definitely what I expect.

I think it is definitely possible for current systems that if you compare a model like 4.5 to a model like o3, I would expect that there will be trade-offs there. I think there are definitely transitory as we figure out our way around these technologies, and so again I expect this will get better.

So I think already we are at a point where a lot of the gap that stops present models from being able to perform a lot of intellectual jobs is more about integrations and interfaces than maybe raw intellectual capability. And so I think we definitely have to think about that. Automation of a lot of jobs is something that will be happening over the next years. And I think this is a big thing for us to collectively think about: what are the jobs that will replace those and what are the new pursuits that we'll all engage on?

I expect this is a quite philosophical question. I think it can go in many directions, but some things I expect: the high-level goal setting, picking what pursuits we're chasing, that is something will remain human. And I think that is something that a lot of people will derive meaning from. I think also just the ability to understand so much more about the world, the incredible variety of new knowledge and also entertainment, but also just intelligence that will be in the world, I think will provide quite a lot of meaning and fulfillment for people.

I think in some ways maybe that's more of a question for you. With GPT-5, with previous models like GPT-4, GPT-3, we've kind of kept very tight connection of how we're training new models and what are the products that we ship. And as I was just saying, I think right now there's a lot to do on the integration side. So for example with GPT-5, it's the first time we really bring reasoning models as our flagship main model. And so we're not coupling these releases and these products as tightly to our research program anymore.