OpenAI Chief Research Officer Mark Chen is here to talk about the release of GPT-4.5, the company's largest and best model yet, which is coming out today. We'll dive in right after this. Welcome to Big Technology Podcast, a show for cool-headed, nuanced conversation of the tech world and beyond. We're joined today by Mark Chen, the Chief Research Officer at OpenAI, who's here to talk about the company's newest release, GPT-4.5. Yes, it's finally here and it is debuting today. Mark, great to see you, welcome to the show.
Thank you so much for having me on.
Thanks for being here. This is, in four and a half years of the show, our first OpenAI interview, so hopefully the first of many. We appreciate you jumping into the water like this, and it's on big news with the release of GPT-4.5.
Yeah, so GPT-4.5 really signifies the latest milestone in our predictable scaling paradigm. Previous models that have fit this paradigm have been GPT-3, 3.5, 4, and now this is the latest thing. It signifies an order of magnitude improvement over the last models, kind of commensurate with the jump from 3.5 to 4.
I think the question that most of our listeners are going to be asking, and certainly we asked on our show in the past couple months, is why isn't this GPT-5? I mean, what is it going to take to get to GPT-5?
Yeah, well, I think GPT-5, whenever we make these naming decisions, we try to keep with a sense of what the trends are. Again, when it comes to predictable scaling, going from 3 to 3.5, you can kind of predict out what an order of magnitude of improvements in the amount of compute that you train the model with, in terms of efficiency improvements, will buy you. And we find this model kind of aligns with what 4.5 would be, so we want to name it what it is.
Okay, but there's been so much talk about when GPT-5 is going to come. Correct me if I'm wrong, but I think there's been a longer wait between GPT-4 and 4.5 than there has been between, let's say, GPT-3.5 and 4. And I don't know, is this because we're seeing a lot of hype from OpenAI folks on Twitter about what's coming next? Or maybe this is probably the most impatient industry in the world and the most impatient users in the world, but it seems to me like the expectations for GPT-5 are built up pretty high. And so I'm curious, from your perspective, do you think it's going to be hard to meet those expectations whenever that GPT-5 model does come out?
Well, I don't think so. And one of the fundamental reasons is because we now have two different axes on which we can scale. So GPT-4.5, this is our latest scaling experiment along the axis of unsupervised learning, but there's also reasoning. And when you ask about why there seems to be a little bit bigger of a gap in release time between 4 and 4.5, we've been really largely focused on developing the reasoning paradigm as well. So I think our research program is really an exploratory research program. We're looking into all avenues of how we can scale our models, and over the last one and a half, two years, we've really found a new, very exciting paradigm through reasoning which we're also scaling. And so I think GPT-5 really could be the culmination of a lot of the things coming together.
Okay, so you talk about how there's been a lot of work toward reasoning. We of course have seen that with o1. There's a lot of buzz about DeepSeek, and now we're talking about again one of the more traditional scaled-up large language models with GPT-4.5. So the big question here, I think that was on a lot of people's mind when it came to this upcoming release—we thought it was going to be 4.55 anyway, it doesn't matter—the big question is: can AI models continue to scale when you add more compute, more data, and more power to them? It seems like you have an answer to this, so I'm curious to hear your point of view on whether what you've learned about the scaling wall, given your development of this model, and whether we're going to hit it, whether we're already seeing some diminishing returns from scaling.
Yeah, I really kind of have a different framing around scaling. When it comes to unsupervised learning, you want to put more ingredients like compute, algorithmic efficiencies, and more data. And GPT-4.5 really is proof that we can continue the scaling paradigm. And this paradigm is not the antithesis of reasoning as well. You need knowledge in order to build reasoning on top of. A model can't kind of go in blind and just learn reasoning from scratch. So we find these two paradigms to be fairly complementary, and we think they have feedback loops on each other. So yeah, GPT-4.5 again, it is smart in different ways from the ways that reasoning models are smart. When you look at the model today, it has a lot more world knowledge. When we look at comparisons against GPT-4o, you see that for everyday use cases, people prefer it by a margin of 60%. For productivity and knowledge work against GPT-4o, there's almost a 70% preference rate. So people are really responding to this model, and it's this knowledge that we can leverage for our reasoning models in the future.
So what are the examples? Like you talk about everyday knowledge work, what are some of the examples that you would use GPT-4.5 for that you would prefer it over a reasoning model?
Yeah, so I wouldn't say it's a different profile from a reasoning model. With a larger model, what you're doing is it takes more time to kind of process and think through the query, but it's also giving you an immediate response back. So this is very similar to what a GPT-4 would have done for you. Whereas I think with something like o1, you get a model where you give a query and it can think for several minutes. And I think these are fundamentally kind of different trade-offs. You have a model that immediately comes back to you, doesn't do much thinking, but comes up with a better answer versus a model that thinks for a while and then comes up with an answer. And we find that in a lot of areas, like creative writing for instance—again, this is stuff that we want to test over the next one or two months—but we find that there are areas like creative writing where this model outshines reasoning models.
Okay, so writing, any other use cases?
Yeah, so there's writing, I think some coding use cases as well. We also find that there are some particular scientific domains where this outshines in terms of the amount of knowledge that it can display.
Okay, and I'm going to come back to benchmarks in a moment, but I want to keep on this scaling question because I think there's been a lot of conversation about it in public, and it's great to be speaking with you from OpenAI to sort of get to the bottom of what's happening. So the first is the question that folks have is: do you end up at this size—and you don't talk about the size of the models, which is fair—but they're big, right? This is the largest model that OpenAI has ever released, GPT-4.5. So I'm actually curious to hear, at this size, does adding similar amounts of compute, similar amounts of data get you the same returns that you did, or are we already starting to see the returns of adding these resources tail off?
No, no, we are seeing the same returns. And I do want to stress that GPT-4.5 is the next point on this unsupervised learning paradigm. And we're very rigorous about how we do this. We make projections based on all the models we've trained before on what performance to expect. And in this case, we put together the scaling machinery, and this is the point that lies at that next order of magnitude.
So what's it been like getting here? I mean, again, we talked—okay, so there was a period of time that was longer than the last interval, and part of that was focused on reasoning. But there's also been some reports that OpenAI had to start and stop a couple times to get this to work, and really had to fight through some thorny issues to get it to be this step change as you're saying. So talk a little bit about the process, and maybe you can confirm or deny some of the things that we've heard about having to start and stop again and retrain to get here.
Actually, so I think it's interesting that this gets attributed to this model, because actually in developing all of our foundation models, they are all experiments. I think running all the foundation models often times does involve stopping at certain points, just kind of analyzing what's going on, and then restarting the runs. And I don't think that this is a characteristic of GPT-4.5. It's something that we've done with GPT-4, with o-series models, and they are largely experiments. We want to go in, diagnose them in the middle, and if we want to make some interventions, we should make interventions. But I wouldn't characterize this as something that we do for GPT-4.5 that we don't do for other models.
We've already talked a little bit about reasoning versus these traditional GPT models, but it makes me think of DeepSeek. And I think you already gave a pretty compelling answer as to what you would use one of these models for versus a reasoning model. But there's another thing that DeepSeek did that is worth discussing, which is that they made their models much more efficient. And it's kind of interesting, like when I talked to you about all right, so you need data, you need compute, you need power, you're like, yeah, and you need model optimizations, which is something that people often overlook. And just going back to DeepSeek for a moment, the model optimization, the fact that they went from basically querying the entire knowledge base to mixture of experts where they're able to sort of route the queries to certain parts of the model instead of lighting it all up, is credited with helping them get more efficient. So I just want to turn it over to you, without commenting on what they did or if you want, but I'm actually more curious what OpenAI is doing on that front and whether you did similar optimizations with GPT-4.5. And are you able to run these large models more efficiently, and if so, how?
Yeah, so I would say the process of making a model efficient to serve, I often see as fairly decoupled from developing the core capability of the model. And we see a lot of work being done on the inference stack. I think that's something that DeepSeek did very well, and it's also something that we push on a lot. We care about serving these models at cheap cost to all users, and we push on that quite a bit. So I think this is irrespective of GPT-4, reasoning models, we're always applying that pressure to be able to serve them more cheaply. And I think we've done a good job of that over time. Like the costs have dropped many orders of magnitude since we first launched GPT-4.
So are there like—I mean, maybe tell me if this is too—but the move towards, for instance, mixture of experts, is that more of a reasoning thing or can you apply that in GPT?
Yeah, so that is an architectural element of language models. I think pretty much all large language models today utilize mixture of experts, and it's something that applies equally to efficiency wins in foundation models like GPT-4, 4.5, as it does to reasoning models.
