Hello. I'm Andrew Mayne, and this is the OpenAI podcast. My guests today are Mark Chen, who is the chief research officer at OpenAI, and Nick Turley, who is the head of ChatGPT. We're gonna be talking about the early viral days of ChatGPT. We're gonna talk about ImageGen, how OpenAI looks at code and tools like Codex, what kind of skills they think that we might need for the future, and we're gonna find out how ChatGPT got its totally normal name. Even half of research doesn't know what those three letters stand for.
You know, you're gonna have an intelligence in your pocket that it can be your tutor, it can be your adviser, it can be your software engineer.
There's a real decision the night before. Do we actually launch this thing?
First off, how did OpenAI decide on that awesome name?
It was gonna be 'Chat with GPT-3.5', and we had a late night decision to simplify.
Wait. Wait. Could you say that again? Say that name again?
It was gonna be 'Chat with GPT-3.5'.
Which rolls off the tongue even more nicely.
And you said that was a late night decision, meaning like weeks before you finally decided what to call it.
Right. Right. No. Weeks before we hadn't started on the project yet, I think. But, you know, I think we realized that that would be hard to pronounce and came up with a great name instead.
So that was the night before?
Roughly. Yeah. Might have been the day before. Was all kind of a blur at that point.
I would imagine a lot of that was a blur. And I remember being in a meeting where we talked about the low key research preview, which like really was. Like, we really thought like, oh, this is because it was the 3.5. 3.5 was a model that had been out for months. And from a capabilities point of view, when you just look at the evals, you're like, yeah, it's the same thing, but we just put the interface in here and made it so you didn't have to prompt as much. And then ChatGPT comes out. And when was the first sign that this thing was blowing up?
I mean, I'm curious for everyone has their slightly own recollection of that era because it was a very confusing time. But for me, day one was sort of, you know, is the dashboard broken? Classic like, the logging can't be right. Day two was like, oh, weird. I guess Japanese Reddit users discovered this thing. Maybe it's like a local phenomenon. Day three was like, okay, it's going viral, but it's definitely gonna die off. And then by day four, you're like, okay, it's gonna change the world.
Mark, did you have any expectation about that?
No. Honestly, I mean, we've had so many launches, so many previews over time, and, yeah, this one really was something else. Right? The takeoff ramp was huge, and, yeah, my parents just stopped asking me to go work for Google.
Wait. So wait. Wait. Wait a second. Up until ChatGPT, your parents were asking, like, what you're doing here?
Yeah. No. I mean, yeah, they just never heard of OpenAI. Right. I think for many years thought AGI was this pie in the sky thing, and I wasn't having a serious job. So it was a real revelation for them.
Yeah. What was your job title at the time?
I think just member of technical staff.
Member of technical staff. Yeah. And then that blew up, and now you're head of research?
Yeah. Actually, on the GPT name, I think even half of research doesn't know what those three letters stand for. It's kind of funny. You know, like, half of them think it's generative pretrained. Half of them think it's generative pre-trained transformer.
Okay. Alright. Yeah. Those people, they don't know the name of it. Yeah. It's weird how just a silly name like that all of a sudden becomes a thing. But you see that with like, you know, Google, Yahoo, Kleenex, things like that. Xerox. And sometimes they were some of those were names by intention, and this was really just a silly sort of name. For me, the moment that I felt like after watching the launch, watching it accelerate, I knew what was gonna happen, and then what it did was when it was on South Park. And remember that when South Park made fun of the name and...
That was the first time I'd watched South Park in, oh, let's just say a while. And that episode, I still think it's magic. Yeah. It was obviously profound to watch and see, you know, something you helped make show up in pop culture. But there's the punch line in the end where it's like, oh, this was co-written by ChatGPT.
I think they took that off, though. I think in later episodes, because it used to say, I think, by, like, Trey Parker and, like, And then no, it was. And then I think later I think they may have pulled that off at some point. I don't remember.
Oh, I strongly feel that you shouldn't have to give credit to it. It's always necessary that you're using the...
If I had to give credit to ChatGPT for every aspect of my life. Well, might as well just say 'ChatGPT maybe with Andrew.' True.
Did you use it for prep for your interviews?
You know, one of my co-producers, Justin, probably uses it. I haven't asked him yet because I'd like to think that he's handcrafting every single question that we're thinking about here, but I am sure. You say it was a bit of a blur. I'll tell you, like, standout moment for me at the launch of ChatGPT was I don't know if you remember this, but the Christmas party. And we'd had several weeks of ChatGPT out there, and Sam Altman went up and said, 'Hey, it's been exciting to watch this, but the Internet being the Internet and I think we all felt this way, it's gonna die down.' Spoiler alert. It did not die down, and it just kept accelerating. What were the things you had to do internally to sort of keep this thing up and running as more people wanted to use it?
We had quite a few constraints. And for those of you who remember, you know, I think you guys remember ChatGPT was down all the time in the beginning, and that was yeah. We'd said, hey, this is a research preview. No guarantees. Maybe it goes down, but the minute you had people loving and using this thing, that didn't feel super good, so people were certainly working around the clock to keep the site up. I remember, you know, we obviously ran out of GPUs. We ran out of database connections. We had, you know, we're getting rate limited in some of our providers. Nothing was really set up to run a product. So in the beginning, we built this thing. We called it the fail whale, and it would just tell you kind of nicely that the thing was down and made a little poem, I think it was generated by GPT-3, about being down and sort of tongue in cheek. And that got us through the winter break because we did want people to have some sort of a holiday. And then when we came back, we were like, okay, this is clearly not viable. You can't just go down all the time. And eventually, we got to something we could serve everyone.
Yeah. And I think, you know, the demand really speaks to the generality of ChatGPT. Right? We had this thesis that ChatGPT embodied what we wanted in AGI just because it was so general. And I think, you know, you're seeing that demand ramp just because people are realizing, you know, any use case that I want to give or to throw to the model, it can handle.
We were kind of known as the company working on AGI. And I think prior to ChatGPT, the API was certainly the first time we had a public offering where people could go use it and do it, but then it was more for developers and stuff. And I think that as long as people were sort of thinking AGI, that seemed to be the point at which people thought these models would be useful. But we saw GPT-3, we saw that that was useful, and then we saw that we could do other things that were useful. Was everybody at OpenAI on board with ChatGPT being useful or being ready to launch?
Yeah. I don't think so. You know? Even the night before, I mean, there's this very famous story at OpenAI of, you know, Ilya taking 10 cracks at the model, you know, 10 tough questions. And my recollection is maybe only on five of them, he got answers that he thought were acceptable. And so there's a real decision the night before. Do we actually launch this thing? Is the world actually gonna respond to this? And I think it just speaks to when you build these models in house, you so rapidly adapt to the capabilities. And it's hard for you to kinda put yourself in the shoes of someone who hasn't kind of been in this model training loop and see that there is real magic there.
Yeah. Yeah. Yeah. I think to build on the, like, the conspiracy internally about, you know, is this thing good enough to launch, I think, is humbling, right, because it's just a reminder of how wrong we all are when it comes to AI. It's why, you know, frequent contact with reality is so important.
Could you elaborate more on that contact with reality? What does that mean?
Yeah. I mean, when you think about iterative deployment, one way I like to frame it is, you know, there's no point everyone agrees where it's suddenly useful. Right? And I think usefulness is this big spectrum. And so, you know, there's not one capability level or one bar that you meet, and suddenly, you know, the model is useful for everyone.
Were there any hard decisions about what to include or what to focus on?
We were very, very principled on ChatGPT to not balloon the scope. We were adamant to get feedback and data as quickly as we could.
I'm always in Slack telling you things about the way that didn't make sense. Add this.
I remember actually there was a lot of controversy about the UI side. For example, we didn't launch with history, even though we thought people would probably want that, and guess what? That was the first request. I also think there's always the question, can we train an even better model with two weeks more time? I'm glad we didn't because we, I think, got a ton of feedback as we did. So, yeah, there was a ton of the scope discussions, the holidays were coming up, so I think we had this natural forcing function for getting something out.
Yeah, there was this habit of things that if it's gonna come after a certain point in November, it's not gonna come out until February. There's a sort of window where things would fall on either side.
Well, would be the classic method in a big tech company. I think we're definitely a bit more flexible on the ownership.
I felt like one of the big impacts was once people are out using it, it felt like the rate of these things improving was tremendous. I don't know if that was something that we really had in the calculus. We could certainly think about training on larger site more data, scaling compute, but then the idea of actually having the signal you would get from that many people using it.
Yeah. I think over time, feedback really has become an integral part of how we build the product. And it's also become an integral part of safety. And so you always feel the time cost of losing out on feedback. You know, you can deliberate in a vacuum. Right? Are they gonna respond to this better? Are they gonna respond to that better? But it's just not a substitute for just bringing it out there. Right? I think our philosophy is let the models have contact with the world. And if you need to revert something, that's fine. But I think there's really no substitute for this fast feedback, and it's become one of the big levers for how we improve model performance too.
It's sort of funny. Like, I feel like we started with shipping these models in a way that is more similar to hardware where you make, like, one launch. Very rarely, and it has to be right, and, you know, you're not gonna update the thing, and then you're gonna work on the next big project. It's capital intensive, and the timelines are long. And over time, and I think ChatGPT was kind of the beginning, it's looked more like software to me, where you make these frequent updates. You have kind of a constant pace the world can adopt. Something doesn't work, you pull it back, and you sort of lower the stakes in doing that, and you increase the empiricism. And of course, just operationally too, you can innovate faster in a way that is more and more in touch with what users want.
Yeah. One of the examples we had of that was the model becoming too obsequious or sycophantic. Could you explain what happened there? That was where people all of a sudden say, hey. It's telling me I've got a 190 IQ. I'm the most handsome person in the world, which I had no problem with personally. But other people did. And what was going on there?
Yeah. So I think one important thing is we rely on user feedback to move the models. Right? And it's this very complicated mix of reward models, which we use in a procedure we call RLHF, using human feedback to use RL to improve the models.
Could you give me just a brief example of what that would mean?
Yeah. Yeah. So I think one way to think about it is when a user enjoys a conversation, you know, they provide some positive signal.
Yeah. A thumbs up, for instance. And we train the model to prefer to respond in a way that would elicit more thumbs up. Right? And this may be obvious in retrospect, but stuff like that, if balanced incorrectly, can lead to the model being more sycophantic. Right? You can imagine users might want that kind of that feeling of, you know, a model saying good things about them, but I don't think it's a very good long term outcome. And, actually, when we look at kind of our response to and the rollout that resulted there, I think there were a lot of good points about it. You know, this was something that was flagged just by a small fraction of our power users. It wasn't, you know, something that a lot of people who generally use the models noticed. And I think we really picked that out fairly early. We responded to it, I think, with the appropriate level of gravity. And, yeah, I think it just shows that, you know, we really do take these issues quite seriously, and we wanna intercept them very early.
Yeah. It felt like there was maybe forty eight hours since the model came out, and then Joanne Zhang had a response explaining exactly what happened. And I think that that's the hard part. How do you navigate that? Because the problem with social media is you're basically monetized by engagement time. You wanna keep people on there longer so you can show them more ads. And certainly, the more people use ChatGPT, obviously there's a cost to OpenAI. The idea is maybe use it once and stay around forever, but that's not practical. How do you weigh that? The idea of making people happy with what they're getting versus making the model be broadly more useful than just pleasing?
I feel very lucky in this regard because we have a product that's very utilitarian. People use it to either achieve things that they do know how to do but don't feel like doing faster or with less effort, or they're using it to do things that they couldn't do at all. You know? First example is maybe, you know, writing an email that you've been dreading. Second example might be, you know, running a data analysis that you didn't actually know how to do in Excel. You know? True story. So, you know, those are very utilitarian things. Fundamentally, as you improve, you actually spend less time on the product. Right? Because, you know, ideally, it takes less turns back and forth, or maybe you actually delegate to the AIs so you're not in the product at all. So for us, you know, time spent, it's very much not the thing we optimize for. You know, we do care about your long term retention because we do think that's a sign of value. If you're coming back three months later, that clearly means we did something right. But what that means is, you know, I always say, show me the incentive, and I'll show the outcome. We have, I think, the right fundamental incentives to build something great. That doesn't mean we'll always get it right. The sycophancy events were really, really important and good learning for us, and I'm proud of how we acted on it. But fundamentally, I think we have the right setup to build something awesome.
So that brings up the challenge. I wonder how you navigate that is that one of the things early on when ChatGPT came out, was the allegation that it's woke, it's woke, and people are trying to promote some sort of agenda from it. My argument always been, you train a model on corporate speak, average news and a lot of academia. That's gonna kind of follow into that. And I remember Elon Musk was very critical about it. And then when he trained the first version of Grok, it did the same thing. And then he's like, oh, yeah. When you trained it on this sort of thing, it did that. And internally at OpenAI, there were discussions about how do we make the model not try to push you, not try to steer you. Could you go a little bit how you try to make that work?
Yeah. So I think at its core, it's a measurement problem. Right? And I think it's actually bad to downplay these kind of concerns because they are very important things. Right? And we need to make sure that the model, the default behavior that you get is something that's centered that, you know, doesn't reflect bias on the political spectrum or in many other, you know, axis of bias. And at the same time, you know, you do want to allow the user the capability to, you know, if you wanted to talk to a reflection of something with more conservative values to be able to steer that a little bit. Right? Or liberal values. Right? And so I think the thing is you wanna make sure that defaults are meaningful and they're centered, and that's a measurement problem. And you also want to give ability some flexibility, right, within bounds to steer the model to be a persona that you wanted to talk to.
I think that's right. I think, you know, in addition to neutral defaults, abilities to bring your own values to some extent, I think, you know, being transparent about the whole thing is, I think, really, really important. I'm not a fan of, you know, secret system messages that, you know, try to, like, you know, hack the model into saying or not saying something. What we've tried to do is publish our specs. So you can go look at, you know, if you're getting certain model behavior, is that a bug? You know, is it a violation of our own stated spec, or is it actually in the spec, in which case you know who to criticize and who to yell at? Or is it just under specified in the spec, in which case that allows us to improve it and add more specificity into that document? So by sort of publishing the rules of the AI that it's supposed to be following, I think that's an important step to have more people contribute to the conversation than just the people inside of OpenAI.
So we're talking about the system prompt, the part of the instruction that the model gets before the user puts the input.
Well, I think it's beyond that.
System Prompt is one way to steer the model, but it, you know, it goes much deeper into that. Right? You know? Yeah.
We have a very large document that outlines across a bunch of different behavior categories how we expect the model to behave. And just to give you an example here. Right? You can imagine if there's someone who comes in with just, like, an incorrect belief, just a factually incorrect kind of a point of view. How should the model interact with that user? Right? And should it reject that point of view outright, or should it collaborate with the user on kind of figuring out what's true together? And, you know, we take that latter point of view, and I think there are a lot of very subtle decisions like this, which we put a lot of time in.
Yeah. That's a hard one because I think some things you can test for and you can try to figure out in advance, but when you're trying to figure out how an entire culture is gonna adopt something that's challenging, like if I was somebody who's convinced that the world was flat, you know, like how much should the model push back against me? And some people are like, oh, it should push that back all the way, but it's okay. What if you're one religion or not another?
Yeah. Turns out rational people and well many people can disagree on how the model should behave in these instances. And you're not always gonna get it right, but you can be transparent about what approach we took. You can allow users to customize it, and I think this is our approach. I'm sure there's ways we can improve on it, but I think transparent and the open about how we're trying to tackle it, we can get feedback.
How are you thinking about as people start to use these models more and more, regardless of whether or not that's some dial you're trying to turn, it's just the more useful it becomes, the more people want to use it. There was a time when nobody wanted a cell phone, and now we can't get away from them. And how are you thinking about relationships people are forming with their systems?
Obviously, I mentioned this earlier. This is a technology you have to study. It's not designed in a static way to do x, y, z. It's highly empirical. So, you know, as people adopt in the way that they use the product, it's something that we need to go understand and act on as well. I've been observing this trend with interest where I think, you know, increasing number of people, especially Gen Z and, you know, younger populations are coming to ChatGPT as a thought partner, and I think in many cases, that's really helpful and beneficial because you've got someone to brainstorm on a relationship question, you've got someone to brainstorm on a professional question, or something else. But in some cases, it can be harmful as well, and I think detecting those scenarios and first and foremost, having the right model behavior is very, very important to us. Actively monitoring it, and in some ways, it's one of those problems we're gonna have to grapple with because with any technology that becomes ubiquitous, it's gonna be dual use. People are gonna use it for all this awesome stuff, and people are gonna use it in ways that we wish they didn't. And we have some responsibility to make sure that we handle that with the appropriate gravity.
I find myself having longer conversations with it. I like the memory function. I like the fact you can turn it off if you don't want. And I think about, like, you know, what's this gonna be two years from now or three years from now when it has a much longer memory, much more context with this? I like the idea to have these sort of like, you know, Memento anonymous modes too, where it's not gonna store this. But I kind of wonder how much you've been thinking about two years, three years down the road. What's that going to be like when ChatGPT knows way more about you?
Yeah. I mean, think memory is just such a powerful feature. In fact, it's one of the most requested features when we talk to people externally. It's like, this is the thing I really wanna pay more for. And I think, you know, you liken it to if you've ever kind of had a personal assistant, you know, you...
Build up contacts more...
But, you know, it's yeah. It's just like it's kind of in any kind of relationship that you have with a person. Right? You build up contacts with them over time. And I think just the more they know about you, right, the richer the relationship, the more, you know, it can also help you. Right? You can work together to collaborate on tasks together.
I do become self conscious of the fact that it knows everything about me when I'm grumpy, and I've argued with it recently, by the way.
That's good. Yeah. You should be able to argue with it. You understand a lot about yourself and having a thing to argue with, and I think you spare others of that experience, which can also be beneficial.
Don't argue on math and science. You're not gonna win those.
Yeah. No. Think increasingly, very unlikely. Yeah. Yeah. Think memory's cool. And to Mark's point, it's been part of our vision for a long time because we said we were gonna build a super assistant before we really knew what that meant. ChatGPT was sort of the early demonstration to that idea. But if you kind of think about real world intelligences, even they are not particularly useful on their first day, and I think being able to solve that problem or begin to solve that problem has been profound. To your earlier question, though, it really does feel like if you fast forward a year or two, ChatGPT or things like it are gonna be your most valuable account by far. It's gonna know so much about you, and that's why I think giving people ways to talk about this thing in private is very important. We make this temp chat thing very, it's literally on the home screen because we think it's increasingly important to talk about stuff sort of off the record too. It's an interesting question, and I think privacy and AI is gonna be an interesting one for the next coming...
I wanna switch gears, talk about another release, which again, kinda caught people by surprise and blew up, was ImageGen. And I was here for DALL-E, DALL-E 2, and then DALL-E 3 came out. I thought DALL-E 3, I thought, a very capable model, but it seemed like it preferred a certain kind of image, and a lot of the utility and the capabilities for variable binding was sort of kind of hidden away. And then ImageGen was kind of just this breakthrough moment that it caught me off guard. How did you guys feel about the launch of that?
Yeah. Honestly, it caught me off guard too. And this is really props to the research team. You know? Gabe, in particular, did a ton of work here. Kenji, many others on it. Did phenomenal work. And I think it really spoke to this thesis that when you get a model just good enough that in one shot, it can generate an image that fits your prompt, that's gonna create immense value. And I think we never quite had that before, right, that you just get the perfect generation oftentimes on the first try. And I think that's something very powerful. You know? Like, people don't wanna pick the best out of a grid. I think, yeah, you just got very good prompt following and, you know, this great style transfer too. Right? Yeah. This ability to kind of put images as context for the models to modify and to change and the fidelity that you could do that with, I think that was really powerful for people.
I think this image experience, it was just kind of another mini ChatGPT moment all over again, where, you know, you have kind of this you've been staring at this for a while, you're like, yeah, it's gonna be cool. Think people are gonna like it, but you kinda, you know, you're launching like 20 different things, and then suddenly, the world is going crazy in a way that you kinda only find out by shipping. Like, I remember distinctly, you know, we had, 5% of the Indian Internet population tried ImageGen over the weekend. And I was like, wow, we're reaching new types of users who we wouldn't even have thought, you know, who might not have thought of using ChatGPT. That's really cool. And to Mark's point, I think a lot of this is because there's this discontinuity where something suddenly works so well and truly the way you expected, where I think it blows people's minds. I think we're gonna have those moments in other modalities too. I think voice, you know, it hasn't quite passed the Turing test yet, but I think the minute it does, people are gonna, I think, find that immensely powerful and valuable, you know, video is gonna have its own moment where it starts meeting the expectations that users have. So I'm really excited about the future because I think there's so many of these magical moments coming that are really gonna transform people's lives. And also, you change sort of ChatGPT's relevance for people because, you know, there's I've always felt like there's text people and there's image people, and some of them are a little bit different, and now they're all using the product and discovering the value across the board.
The moment when it launched, I think it kind of illustrated the problem that had been with image models before. And when DALL-E came out, was super exciting because you're like, I'm doing pictures of space monkeys and all these sorts of things. The moment you try to do a really complex image, and that's the phrase I brought up before, which is variable binding, you start to see these things drop off. And that was when I realized, oh, there's gonna be a challenge for other image systems that don't have kind of the scale and the compute of, like, a GPT-4 under the hood. And now, was it just was it basically that, like taking, like, a GPT-4 scale model and say, now you do images that made the breakthrough?
Well, I think there are a lot of different parts of research that made this such a big success. Right? I think with a complicated multistep pipeline, it's never just one thing. Right? It's, like, very good post training. It's very good training. And I think it's just all of that coming together. Right? Variable binding definitely was one thing that we paid a lot of attention to. I think one thing about the ImageGen launch is a launch that was very deep. I think people, you know, they started by working on, you know, creating anime versions of themselves. But you realize when you play with it more, you know, the infographics, they work. Oh, yeah. Like, you actually create charts. Comic book panels.
