Wewnętrzny monolog naszych maszyn! Moja droga. | Jakub Pachocki Time100, Open AI

Our company is built on the belief that in the not-too-distant future, AI will be as intelligent or more intelligent than humans. I believe this development has no end, and where we are now is, in a sense, the beginning of a new paradigm that also scales. Wojtek Zaremba wrote to me, and I went to OpenAI. It really shook my world. We will reach a point where models will train even better models.

Yes. Since the invention of atomic bombs, the world hasn't been in such an unstable place as this revolution could cause, so it's something that stresses me out. The next step is AI that is truly capable of building something in the real world.

In this field, we are the best in the world.

Right now I'm Chief Scientist. So I lead our company's research program, deciding which directions we will pursue, how we will develop the AI we are working on.

Our company is built on the belief that in the not-too-distant future, AI will be as intelligent or more intelligent than humans. And I think that 5 or 7 years ago, it was more like, 'maybe there will be such a moment.' Now it seems to me that we see it's more continuous, right? GPT is already smarter than humans in some ways, and much dumber in many ways, and I think it will just keep developing. But yes, a lot of our focus is on creating the strongest AI in the world, but also understanding how it works, how people will collaborate with it, and what impact it will have on people.

Well, this has been the subject of my work and research for a long time, it just evolves a bit. When I was in high school with Professor Szubartowski, it was about what I can program a computer to do. That was always a huge draw for me in programming tasks—there's a clear input to a program and it has to produce an output, and at first glance, it's not obvious how the computer should do it. And the fact that I can somehow figure it out and show that the computer can do it very quickly was fascinating. That's why I later went into theoretical computer science and did a PhD in theoretical computer science, where you prove what computers can theoretically compute and what they can't. It's a very long-term science that really touches the boundaries of mathematics—what functions can be computed, which cannot.

I didn't abandon mathematics, because deep learning is also mathematics, just a bit different. It's more applied. How is it that Kuba Pachocki, from such theoretical mathematics—looking at your work at Harvard, it was about theoretical mathematics—and suddenly you focused on practice, on deep learning, on applied mathematics? When I started my PhD around 2013-2014, my view of the field was that there are algorithms we can prove work, and algorithms we can prove work even if they aren't very effective or practical right now, are worse than heuristic methods, but they have the property that if the computer gets bigger, they will scale. And if you look at the last few decades of scaling, and if you were looking into the future from that moment, you could conclude that what computers can do now isn't that important. What's important is what computers will be able to do in 50 years, and we should build a theory to understand the limits of computer capabilities and how to think about it, and really expand our understanding of algorithms. And what also confirmed me in this belief was that a lot of the most interesting things computers could do were based on theoretical algorithms. For example, an optimization algorithm that was used a lot in practice and still is—linear programming, or specifically maximum flow, which was always one of the most magical algorithms for me in programming contests. When I was doing my PhD, new fantastic methods based on linear algebra for solving these problems were emerging. Professor Aleksander Mądry, whom I didn't know then, was working at MIT and making amazing progress.

I'll come back to him later. So I was thinking about building an understanding of what computers can do that will be important in a few years, and there were already some early results in deep learning. I mean, deep learning results had been around for a long time, but they were definitely starting to be good for vision, for image recognition. But it wasn't clear to me then whether this was something that would scale, whether it would last, because the theoretical foundations of deep learning are much worse. Unlike classical theoretical computer science, in training we can't prove that we're actually finding the optimal solution, that we're finding the optimal parameters of the network we're optimizing. We just build a network that looks reasonable, optimize it for some time, and get some result. So it was easy to imagine that this was something that works a bit, but it's unclear if it will continue to work. What made me fundamentally rethink my approach to the field were the results of DeepMind's AlphaGo. There were the fantastic results of Deep Blue, inspiring that computers can play chess. But okay, chess is a game with a relatively small board, few moves, and it's easy to imagine that modern computers can thoroughly search all possibilities. Maybe not all, but a large part.

But in the game of Go, it's a very large board, stones can be placed anywhere, there are very many possible moves. And there was a general consensus in the field that this game would take many, many more years, that algorithms like Deep Blue didn't have much of a chance. And what DeepMind showed then was that if instead of writing a bunch of heuristics by hand for how to search all possible moves in the game, we use a neural network that learns from human games or its own games, we can play better than humans in most cases. It really shook my world, because it's much easier to imagine how from playing games you can get to solving programming contests or further, writing scientific papers and really advancing science, than from just image recognition. From then on, I became more and more interested. Soon after, I finished my PhD and started talking to several potential companies that wanted to conduct deep learning research. At Harvard, I was already moving more towards machine learning. I had theoretical papers there, but more related to the topic. And during my time at Harvard, I also spent a lot of time—I got to know Olek Mądry better, and we spent a lot of time together. He had amazing results on graph flows, in the same field I had worked in before, but he had come to similar conclusions as me. So we talked a lot about how everything would change and how we could both—we were both a bit dissatisfied that deep learning had such weak theoretical foundations and we didn't quite understand how it would scale and how to think about it. We wondered if we could build a better theory and a better way to think about all this. And in particular, for some time then, Wojtek Zaremba was writing to me, who I think had a plan to simply recruit people who did well in programming contests. For a while, he wrote to me from Facebook AI Research, and then from OpenAI. And ultimately, half a year after my PhD, half a year after finishing, I decided to jump into this field with both feet and went to OpenAI.

Something that is... We try to simultaneously push the development of AI and the field in a good direction, but on the other hand, we don't publish many scientific papers, so I don't know if such a statement would be fair.

Yes. It's a bit more than a reflection. It's something we actively think about a lot—what are the consequences? The path I took to AI, from interest in programming contests, what computers can do, to AI, isn't very unique. Many fantastic people at our company also did programming contests in the past. So a few years ago when someone asked me when I thought we should really start, when we would be close to artificial intelligence, I said when AI starts solving, gets a gold medal, or starts solving all tasks on a mathematical or programming olympiad. That's happening. It's happening now. Yes, we are at that point now. So the next step is AI that is truly capable of building something in the real world—whether it can build useful programs, or write books, or conduct research and discover new things. And I expect we'll see this before the end of next year.

It's worth mentioning that this test consists of two parts. One of them is much harder than the other. So we're probably close—these models already perform excellently on the part that resembles math competitions a bit more, not as involved in scientific research. But I think it will still take a while before they really solve the entire exam. But yes, it's happening. We see a fairly direct path to this. And we're definitely thinking a lot about what it means that we'll have models that not only are nice to talk to and might help you remember something, but are better than us. And we can take a cluster of GPUs, run a bunch of copies of the model on it, and we'll basically have a research laboratory. They'll just think on their own and produce scientific results. I keep coming back to research work and scientific results because I think that's in a sense the most drastic consequence—because scientific results lead to new technologies, they lead in particular to accelerating AI development, but also to changes in medicine. And this could start accelerating at a pace we're not used to.

If we want to solve really difficult problems, many theorems are discovered in order to prove other, harder theorems. So these aren't completely different problems. If models are able to solve difficult tasks that require many months of work, that's also largely creative work—you need to formulate new hypotheses and build understanding. We wouldn't be able to train the models we train today without many years of work building foundations and trying to understand what models from years ago were doing.

Absolutely. Yes, especially now when so much is happening. We're building a very important technology that already has great significance for people and will probably have even greater significance. Of course, there's also the whole active market, competition, business—and to remember in all of this what the mission is: that we want to build artificial intelligence that will truly change the world for the better. The assumption is that artificial intelligence will be created, so we want it to work for the good of humanity, not against us. To remember this, and to remember that we want to continue building understanding of how these models work, how to make them work beneficially. It's not that different from having to find time amid the noise of competitions and school and everything else, give yourself a bit of space, and focus on what's really important.

I try to focus on what we fundamentally want to achieve. Our company wasn't built on the assumption that we need to build a big business bigger than Google. Of course, we've had significant success, GPT is developing, there's competition, we need to think about how we'll fund future research, and we need to create value. But I have fantastic colleagues who also think a lot about these trade-offs, and I'm able to focus largely on research work and on how what we're working on now leads to artificial intelligence that can conduct research, and how such intelligence affects the world and what we can do to make this whole path go well.

Yes.

The observation that language models like GPT-2 or GPT-3 perform better if you ask them to think for a while in context—that was already known long ago. It was one of the first observations that really inspired people with these early models. So it's not a new idea that we want the model to think for a long time in context. What took some time was perfecting the technology so it could really learn. The basis of these technologies, which isn't a huge surprise either, is reinforcement learning. You train based on what the model does itself—instead of copying how humans think, we look at how the model thinks and reinforce the parts of its thinking that lead to better results. This is in principle the same method that AlphaGo used to teach neural networks how to play Go without looking at how humans play. It's also the same method we used—this was my first big project at OpenAI—to teach a neural network how to play the computer game Dota 2. OpenAI was very interested in reinforcement learning from the beginning. We thought about how we could reach a point where these networks learn from their own behavior. Combining these original ideas with the fantastic GPT models leads to models like O1.

In a sense, supervision is still over conclusions. The difference is that we train the model not only to produce conclusions, but we reinforce its own chains of thinking that lead to correct conclusions.

So we can pause for a moment on the fact that we now have this new paradigm, because that in itself is significant. I think it's easy to take it for granted, but the last time we had a new paradigm was maybe GPT-1 or GPT-2. And the difference between those models and GPT-4, which ChatGPT was based on, is enormous. And where we are now is in a sense the beginning of a new paradigm that also scales. So we can really develop along this same axis, and this certainly isn't everything—I'll say what else we can do—but there's still quite a lot left on this axis, so I think we'll see much smarter models based on similar methodology. What these models currently don't do is have long-term memory in a sense—neither O1 nor ChatGPT. There may be some simple memory mechanisms, but it's not like with a human, where you can talk every day and they build real context of everything happening, learn about you, learn about the world around them. Right now these models mostly don't have that state. Especially if we want to conduct very long-term research, if we want these models to function in the real world, that's one of the things that still needs to be solved. Another important thing is that right now, a model like O1 is just some chat where you can talk with text. But of course it could do much more. First, if it can use a computer—which will probably happen soon—and in the future, if it can function in the real world, there's also robotics.

Also, but I think the technologies we have—I think you can see how spatial understanding emerges from them, it's not that difficult a part. We already have video generators, ways to watch video, so I think these models will be able to reason spatially. However, I think the harder problem, or at least one that will take more time, is giving them a robotic body that really functions well.

Well, first of all, if we look at models like Sora, video-generating models, they to some extent understand the laws of physics—they can generate video that mostly, at least partially, models some laws of physics. But if you wanted to integrate robotics with artificial intelligence, that's not enough. It's not enough to understand space, you also need to understand the effect that a specific robotic action, a robotic arm, will cause in the physical world.

I wonder differently—whether it's able to create its own physics, or rather whether it's able to fill in the gaps in our understanding of reality. Because we don't know many things about the world around us. So these models—if we look at GPT-4 and O1—they try to predict what happens in the real world. And if we try to predict really well what someone would write about what happened in the real world, we need to be able to model the laws of physics. What we're modeling might be a physics question, or a scientific paper about physics, or a book where someone describes an event and its consequences. So even without ever looking at images and never having a body, you can learn quite a lot about physics.

That's a very interesting question. Generally speaking, yes, I believe this development has no end. And in particular, one thing that is a bit counterintuitive—we always talk about AGI, artificial general intelligence. It's in a sense the holy grail for the entire field—intelligence that is truly intelligent, truly as smart as humans. And what's a bit unclear and counterintuitive is actually how smart humans are. If you think about it this way, you can imagine that true artificial intelligence is when we've understood 100% how deep learning works, built great data centers in space, fixed all the shortcomings of our methods, and then built artificial intelligence and solved all the world's problems. But the truth is probably more that intelligence develops somehow, and humans are somewhere on some level, at some point. And you don't need to solve all those problems—at some point these models will just be smarter than us on practically every axis, and that doesn't mean we'll understand how they work very well, and it doesn't mean it will be in 50 years—it could be very soon, and that's a bit hard to internalize, I think.

Well, certainly today we would consider strong artificial intelligence differently than in 5, 10, or 15 years, because we will also develop. Yes, I think it's already clearly visible that it's not a point where models are suddenly smarter than humans—it's more that there are things humans do better, things models do better, and this will develop. People will increasingly work with these models, but at some point we'll reach a place where we'll use large GPU clusters to conduct truly open-ended scientific research, say in medicine. And I think when we reach that point, we'll probably still argue about whether it's AGI or not—maybe they're just parrots—but it will be a somewhat different world than now.

O1 already works largely on synthetic data. There's the GPT paradigm where a language model can learn from a huge amount of text that humans wrote. And humans wrote a finite amount of text, and at some point it stops scaling. But things like AlphaZero or Dota didn't have that problem. And I think we see many ways in which current methods are much less limited by data.

Definitely. I think that's a direct consequence of models that can—those recursive loops. Of course, it will be gradual. I think this year some simple, easy-to-understand part—for example, some internal implementations in infrastructure—will be assisted by thinking models. But it will still take some time before we reach models that conduct entire research programs. But I think that's the expected consequence. And of course, if we reach that point, we need very good ways to supervise such models, which is non-trivial because they'll be much better at thinking about these problems than we are. A few years ago, this problem wasn't clear at all—how to approach supervising models smarter than us. One thing I think is very promising in models like O1 is that they have their internal monologue, their internal chains of thought, which unlike what ChatGPT outputs, don't try to please the user. If an O1 model thinks we're completely wrong and it makes no sense, it might not tell us directly because it's trained to be nice to users, but internally it will think what it wants. And we can actually peek at that.

Yes, it's a very interesting problem. It's something now, especially in the context of DeepSeek, there's definitely a conversation about it. They give fairly direct user access to how the model thinks. In our first release of O1, we didn't give such access. What we're also thinking about is that if we make products where part of the product is full access to what the model thinks, then ultimately it will be subject to the same pressures as what ChatGPT outputs—users will want the model to think clearly and not think about how to deceive us. But the problem is that if we supervise models not to think about deceiving us, they might think about deceiving us in ways that aren't obvious to us, which would be much worse.

I think they weren't trained much for that. I expect it will improve significantly.

Maybe not necessarily all the senses. I agree that right now models are pretty bad at telling jokes, although from our newest model I sometimes get a quite funny snippet. But I agree that it's a clear gap, especially compared to how great they are at math and such things. But it's not something they're well-trained on right now. But I really expect this to change quite soon, so I definitely wouldn't bet that humor will be the last thing. I think humor will come quite soon, and it will be something very interesting and a bit of a different feeling. Especially if we reach models that can produce interesting jokes as well as they can now solve math problems. That would be quite cool and amazing. But I think it will happen quite soon.

There are definitely several things I think about a lot. What we were just talking about—in the longer term, when these models are conducting research, building new versions of themselves, how will that work, how will we supervise it—that's a big problem we need to solve. There's also a more short-term problem—even if models aren't that smart yet, but can perform a substantial part of valuable work, that in itself is a real technological revolution. Since the invention of atomic bombs, the world hasn't been in such an unstable place as this revolution could cause. So it's something that stresses me out.

Rather in the morning. For some time now, I've been dividing my time—every other week, it was Sam's idea—for one week I sit in the office all day, meet with people, and we talk about how to run projects, discuss research directions, and I try to help people create space and guide everything in some direction. Then for a week I try to focus on one specific technical problem, maybe a research direction I'm working on, or collaborate with one team to really be able to dive deep.

It varies. Most often I work a lot with Szymon during that time—we've been very close collaborators since the beginning of my career at OpenAI—or with other employees from the company. Because the problem is that there's a lot of context—we're building on several layers of understanding of new research directions. If I just jumped from meeting to meeting all the time, at some point I wouldn't be able to really think deeply about it. And this way—Sam himself suggested this approach to build some space for me, and it's been working so far.

I think the lesson I've learned many times in my life—maybe first from Professor Szubartowski, but also later—is that if something really interests you, and maybe it's not exactly what you're working on right now, or maybe you're not even that good at it yet, but it really fascinates you—you can do it. It's not that you can't. Sometimes it's hard to realize that you just can—sometimes you build an image of what you do, that 'I'm a student in school and I do this' or 'I'm a programmer building a specific type of system, so I obviously won't work on that because it's something else and someone else already does it better.' Understanding that you actually can do it and nothing fundamentally prevents you is incredibly powerful. And of course, here in Silicon Valley, there's an atmosphere—it's a bit of a mecca for people who want to do something on their own, and people here don't really feel such barriers. But even here, I often have to remind myself of this.