This is totally backwards from how you're supposed to do a startup, right? You're supposed to have a problem and we had no idea what the problem was.
Is there a world where the AI becomes the manager and it, you know, gives you ideas and gives you some tasks to do?
This was probably the hardest project that I've ever done, because it felt totally doomed, right? It's like, I know like every instinct, every builder instinct of mine.
Did it actually feel doomed?
Oh, I felt totally doomed.
Greg in 2010 dropped out of MIT to become our first engineer and went on to become Stripe's CTO. In 2015 after he left, he co-founded OpenAI. OpenAI is really cooking at the moment and Greg is one of the most productive people I know. Cheers. Wow, you really have some old school photos. Some deep cut GDB trivia. So yeah. We figure we've gotta put people at ease, make people feel at home. Very nice. Thank you. Okay. Well, I just dive straight into all the questions I have, which is a lot. All right. If you were not working in AI, how could one have known that something was about to start working? People were telling you that AI was the future in the 1970s, in the 1980s, the 1990s. Then very quickly, in the late 2010s everything started happening.
Well, I was someone who was not in the field and so, I remember very much what it was like. 2013, 2014, it felt like every day on Hacker News there'd be a new 'deep learning for X' article. And I remember being like, 'What is deep learning?' I knew, like, one person in the field, and I asked them to introduce me to more people in the field, and I just kept getting introduced to a bunch of my smartest friends from college. Now, if you actually look at the work that was being done... 2012, basically, image recognition, for the first time, you could solve with the neural net much better than anything else. It just blew all these traditional computer vision approaches out of the water. It's like this learned system is able to outperform 40 years worth of, 'Let's write down all the rules and try to handcraft the algorithm for the task.' And it's very easy to then be like, 'Okay. Well, this approach, sure it works for computer vision, but it's never gonna work for machine translation.' In 2014, suddenly, you're getting great results in machine translation. I think that this pattern was applied in subfield after subfield.
One thing I've been wondering about is so many different things are finally working at the same time. We have LLMs, which are obviously amazing, but then we also separately have image models really working. We also have text-to-speech and speech-to-text working way better than they were before. What's the common factor behind everything starting to work at the same time?
Well, it's deep learning, right? I think deep learning is the core- We've had deep learning for a long time. Why didn't deep learning work in the 1980s? If you look at the number of orders of magnitude of compute that we've gone through from 1940 to today, I mean, it's just astounding.
You think they're all explained by compute scale-ups applied to the right algorithms?
Of course, the type of algorithm changes, and some of those results aren't even deep learning. But I think that fundamentally, it is about compute, and you need an algorithm that is scalable that can actually absorb that compute.
Was OpenAI the first company to take the scaling hypothesis really seriously?
I think that claiming the first is always difficult, but I think that is clear that we sort of succeeded much more wildly sooner than anyone else. I think that we had real conviction behind what we needed to do. Some people think that OpenAI set out to prove the scale hypothesis, whereas it was almost the other way around, that the scale hypothesis is what we observed as the thing that was working for us, and we really saw it for the first time, actually, during our Dota 2 project. We started out with 16 cores to train a little agent- on Jakub and Szymon who were leading the project from an ML perspective, on their desktop. Then, they scaled to 32 cores. It felt like every week I'd come back to the office and they'd scaled it by another 2x and we had 2x performance- just so clear, you just need to keep going. Where does this thing peter out? It just never did. Founders get too much credit, because you have an initial product that's a pretty reasonable idea Then you listen to the customers, and you follow what's working.
So you were saying that was kind of OpenAI with the scaling hypothesis, where you started trying to make Dota AIs work and you noticed that adding more compute worked really well, and you said, 'Where else will just throwing more compute at it yield benefits?'
I think that's to the first order correct. I think one thing that distinguishes OpenAI from the typical startup, is we did everything in reverse, right?
It's like, you're supposed to have a problem to solve. No one cares about the technology. Form the entity upfront.
Exactly, yes. And for us, we really chased the technology without any idea of how it would be applied. A lot of pursuing the technology really is you have to let reality hit you cold, hard in the face. There's just no other way to achieve results. You can't will it into existence. You can't convince people that this is the thing. It's like, you have to actually, make the system work. We have to just sort of figure out what is the right frontier, what are the problems, what are the things that are on the edge of working, and to really double down on those.
What else do you take away from the Dota work? How else does it... Because you could have just started with LLMs and you know, we could have skipped that period in the wilderness, but it sounds like it was somewhat formative for the OpenAI organization.
I think Dota had many lessons, one of which actually was a management lesson for me. I remember when we started out the project, I tried to set a list of milestones, right? It's like, 'Okay, this date we're gonna beat this player. This date, we're gonna beat this player.'
It did not work at all. I remember our first milestone came and
Exactly. And so you realize that you cannot control the outcome, right? You cannot set outcome-based milestones. What you can do is you can control the inputs of we're gonna try these experiments by this date. We're going to implement this feature by this date. And that is what actually worked. And I remember it was one of those things that was just a story that I could not have written in any better, if we'd intended to. We beat our in-house best player and then we were playing as a semi-pro and he was just trouncing us, trouncing us. Then, suddenly we're starting to get pretty good. So we showed up at the international, this tournament blind. First day, we had three players that we played against. We went 3-0, 3-0, and then 2-1. We're like, 'Oh no. We lost. What happened?' It turned out that this pro that we're playing against, he had used an item we'd never trained against. And we were like, 'Oh no, we're totally going to be hosed.' So, what do we do? Well, we just need to change the training. And so people stayed up all night to get this done. They added this extra item in there. 4am, they finally get the job running. That Wednesday, we're supposed to play against the number two and the number one player in the world, and our semi-pro plays against it. He's like, 'This bot is totally broken.' And we're like, 'Oh no, we like clearly had a bug. Something terrible has happened.' He was like, 'Look, it's taking all this damage it doesn't need to. I'm gonna go kill it.' He goes to kill it, he loses. And he was like, 'That was weird.' He'd realized that what had happened was it had learned a baiting strategy. And then we realized, 'Well, we have a super-bot, but it's so bad at the beginning, because it's trying to do the baiting. So, what if we just stitch the two bots we have together?' And then that bot was just undefeatable, and we played against this number one player and won. To me this is like the story of how deep learning works, right? Is it's like you kind of can't control where you're gonna go. You can control everything that goes in. You can put these metrics in these measurements and you can have sort of the evaluations. And being able to gauge where you're at is almost as important as being able to make the forward progress. But if you get all those elements right, then you can do true magic.
You're also describing something that worked really well for an organization where it was motivating to stay up all night. If the prize was impossibly far away, it wouldn't have been as motivating. But the fact that there was a near-term reward function and you were able to show concrete progress...
I think so. I think some of my favorite engineering stories have the same character. I remember you and I staying up all night to get our ISO 8583 integration. There's something about staying up all night for like critical projects that actually have important history in all startups.
I'm glad to hear that tradition is alive and well at OpenAI. So, Dota has fallen, Chess has fallen, Go has fallen. We've passed the Turing test, I think by anyone's measure. People comment on how there was little fanfare when we did, but we seem to have pretty clearly done so. What's a good new Turing test?
Well, I'll tell you two things. One, is that, if you look at the strict version of the Turing test, I would actually claim we haven't done it yet. No one's really gone that extra mile to say, 'Can we actually have an AI that like is fully indistinguishable from a human?' It's not clear if it's even a good task, right? But I think that the right question to your point is like, well, what is the milestone that we should be chasing in terms of capability? I remember talking to one of our board members in 2018, and he said, you know, 'Look, I get that you were all excited about near-term AGI, but it just doesn't feel like it's on track.' I asked, 'Well, what do you mean?' He said, 'In a world with near-term AGI, you would expect massive economic value to be delivered by AI already, and where is it?' In 2018, I think that was a very fair criticism, and clearly, that's starting to change now.
It feels like one thing that may really change the AI market is personalization. Up to quite recently, when you asked ChatGPT a question, it was like walking into a shop off the street. They've never met you before. They know nothing about you, whatever. That's obviously, not ideal for this, you know, close part of your digital life. I'm curious how you're thinking about personalization from a product point of view, because it feels to me like the most meaningful change since the chat interface.
Two and a half years ago. I mean, I think it's absolutely critical, and I think it is very rightly considered to be kind of a next frontier. I'm someone who always, when I just Google something, I go into Incognito Mode, because I don't even want my computer to remember that history. And I always used to go for temporary chats on ChatGPT. But now my usage has totally reversed. I want ChatGPT to remember everything. I want it to remember my interactions, because it's useful.
Okay. So you guys figured out, from a product point of view how to make the memory actually work better. It's the product point of view, but also, really the research point of view. And I presume there's a flip flop between the product and research where, when you find something that's useful from a product point of view, then the product people say, 'I'm just a product person, you researchers go actually make this good.' Then that kind of kicks off more research. Is that how it works?
To some extent that's a failure mode in our mind. I think that we really don't want to have that kind of silo. We really want to blur the lines and have people cross-collaborate. And so, it's a very different mindset from how you would traditionally build a product versus how you do research. Part of what had happened, actually, was that we had GPT-3, we knew we needed to build a product in order to be able to continue to raise funding, and we were like, 'Well, what product do we build?' And we wrote down a list of, like, 100 different products. We could do a medical thing. Then you're like, 'Okay. Well, now we have to sell to hospitals. We're gonna have to hire doctors.' You realize you give up on the G in AGI, right? You're gonna like go for a specific thing. Someone had the idea of saying, 'Well, why don't we just make an API and let people figure it out?' And, again, this is totally backwards from how you're supposed to do a startup, right? You're supposed to have a problem, and we had no idea what the problem was. Yeah, yeah, yeah, we're gonna back into the problem. And so this actually felt like this was probably the hardest project that I've ever done, because it felt totally doomed, right? It's like, I know like every instinct, every builder instinct of mine says-
Did it actually feel doomed?
Oh, it felt totally doomed.
It wasn't just like open-ended or something?
But you were still doing it.
Yeah. I mean, it's like, at some point, if you have... There was definitely no other path.
It was the only shot we had. I remember someone also saying like, 'I can't imagine anyone paying for samples from this model.' And I was like, 'You might be right.' I'm still trying to imagine it myself. Yes. It was just not clear, were we above threshold or below threshold? We showed it to people, and people were interested. But that's very different from people being like, 'I will build my company on top of this,'
Yes. So, what was the first use case to get any traction?
What was that again? There you go. AI Dungeon was a text-based adventure game. Oh sure, yeah, yeah, yeah, yeah. Okay. But that was real revenue or that was non-zero revenue?
It was enough. And, in fact, I believe they were our first paying user. And that gets you confused for, you're like, 'Ah, clearly the future of OpenAI is gaming' you know?
I know, back to our roots.
Exactly. It's interesting too, because we had dreamed of all of these applications, like medicine and all these things, and you start with the gaming application. But we could see signs of life on so many other things. I think in many ways GPT-3 was like the world's best demo machine, right? When we released the API, people were coming with all these cool things you could do, but making them reliable was so hard. It really wasn't until the next generation of GPT-4, until we started to figure out how to do post training well that then you were actually able to build real businesses on top of these things.
Bill Gates was saying recently that GPT-4 was the best demo he'd ever seen since Xerox Park. You know this quote?
Yes. He said it to me the night that he saw it. Yeah. That's high praise.
I wanna touch the medicine thing, because you've mentioned it. Like you said your family has personal stories. You've talked about getting very valuable diagnostic help. We, ourselves, actually, it's much more minor in our family, but we managed to fix a cat thanks to debugging it with an LLM. I think that's an interesting example, because so many people that I know have had some kind of experience like this. Maybe it's because you actually don't get that much time from a doctor. Are there other examples like this medicine application where you're seeing a lot of success, that many people have similar stories, but we just hear less about?
Yeah, I think it's a great question. And by the way, I think like medicine is an example of one where I kind of thought it was gonna be one of the last domains that we would successfully be able to add value in, but it turns out that the bar is so low, you just need to exceed WebMD. I think that we have seen other areas that are like a real common theme. One that's very interesting right now is the life coach, life advice kind of application, where you just talk your AI. That's actually really taking off.
Education is another area that just like clearly is really having an impact. There are studies coming out now that actually show that people are able to learn better through the use of these tools. That's to be expected, right? It is the Bloom 2 sigma effect in a product.
Yes. And that, for example is like why, Sal Khan, started Khan Academy, to think about if you can give personalized tutoring to everyone. We showed him GPT-4. He's like, 'This is the thing. We need to become a GPT-4 app.'
I think that there are these really amazing applications that are affecting everyone's daily lives. Obviously, programming is another one, that people are seeing all across the board in a professional context. We're heading to a world where, like, if you want to do productive work, and you don't have access to computer, you're going to be hampered. Similarly, not having access to AI, it's heading in the same direction.
Speaking of not having access to AI, I will posit that these days, it feels like AI product development is mostly OS limited. Is that how you feel? Are we stuck at the moment?
I do feel a little of stuckage, but not to worry, it was overcomeable. But yeah, I think it is true. Two years ago we released plug-ins in ChatGPT. Do you remember those?
That was trying to make it so anyone could write apps that then ChatGPT could access. The models were just not that good, right? That we limited to like three plug-ins at a time. You could have only so many functions and stuff. It just wasn't that reliable. Now we're in a world where MCP basically is really taking off and is a way to hook up your AI to different tools, and very much like kind of trying to take that same type of idea and really make it work. Now, the world that we're in is very similar, where there's certain interfaces we don't have. Being able to access your phone and all those APIs. There's a question of, is the model above threshold to actually use them or not? My observation has been that basically, I think that there is maybe a lag of, like, six months of different interfaces that are hard to access. But once we have a model that's good enough, we will find a way. People will find a way. I think that we're in a world where I have every expectation that we will get the future that has been promised. It's just gonna take some work.
