Hello again, last section of the day. Let me introduce Mark. Mark is our head of research at OpenAI, looking after the research effort like model development. Mark was also the lead on o1. He's been at OpenAI for quite a few years. You want to say a few words about yourself?
Sure, yeah, so it's nice to meet you all. I've been at OpenAI for over six years now, and it's crazy just kind of having it go from a 20-person startup to really this kind of international corporation. And it's great to be back in Singapore. I was here about a year and a half ago, met with a lot of the people on the ground, and I feel like kind of the technical depth of the people we met with was beyond anywhere else that we visited. So yeah, really, you guys are in a great spot.
I agree. Every single meeting for me today and yesterday, I was pretty impressed by the depth for sure. Awesome, we have 30 minutes. I'm excited. I don't get to put Mark on the hot seat with like hard questions every day, but you know, here we are. You guys left a bunch of good questions in the box, and so yeah, we'll try to get to as many as we can. Let's start. Actually, starting like what excites you about being in Singapore for the event? How do you see Singapore leading in AI?
Yeah, I mean, it's really just how deep the technical depth that you all have. So I remember coming here right and meeting the former prime minister, and you know, I gave him a coding demo and learned that, you know, he actually codes himself and, you know, he's done quite a bit of that. And, you know, we found that pattern throughout, you know, all the people in government, in business, they just kind of knew the technical details in and out. So that's the thing that makes me most excited.
I have exactly the same anecdote. I was in a meeting with a regulator agency this morning. I was prepared to talk about regulation high level, like, you know, how does TCP work. We started talking about reinforcement learning like from the get-go. Chair, how does it work? The level of depth and pragmatism, I would say, is pretty high for sure. What was the last time something in AI research surprised you, as in, wow, this is sci-fi becoming real kind of way?
Ah, good question. You're starting with the spicy ones. Yeah, I think for me, you know, I got my start in AI research in image generation, and I just feel like kind of visual stuff is always very visceral to me. So it's very compelling, right? Like you can see it immediately, you don't have to kind of read a paragraph or something. So, you know, just the whole wave of image generation, video generation improvements, like that's just been wow.
Yeah, yeah, for me that would be speech-to-speech. Like the first time having like a natural-sounding conversation with an AI for like a few minutes and just seeing that, you know, it's actually like very intuitive. Like, you know, there is nothing like weird about it, like just what I would expect. That was my biggest mind-blown I think the past like few months.
Yeah, I mean, one thing to add on that too is, you know, I used to be a competitive programmer. One other big moment was just watching these models slowly like catch up to and even get better than I am, which is very scary.
He's pretty good. That's what's even most scary. Okay, very easy question, Mark. How close are we to AGI?
Yeah, I mean, that's a very hard-to-answer question just because people define AGI so differently, right? I think when you look at it from an economic lens, you already see our products providing a lot of economic value, right? Clearly, OpenAI is one of the most valuable tech companies, and we are providing billions of dollars of value to real users today. I think there's a different definition too of, you know, are we able to perform well on benchmarks that capture kind of intelligence or the ability to kind of do general tasks. And it's interesting to see just like two years ago those kind of tasks that were the frontier tasks for AIs were grade school math problems, and today they're the hardest PhD problems. So I do think, you know, we're in the regime where these models are able to solve some of the most difficult exams that humanity has crafted.
How do we benchmark models once they solve PhD-level problems? Like, what's the next?
Yeah, that's a very good question. So I do think increasingly we have to lean on utility, right? We have a product out there, we're trying to provide utility and value to the world. And I do think, you know, when you saturate all the benchmarks, like really what you're looking at is like, do I provide value to the end users? That makes sense.
And how has, sorry, I'm drilling down, how has like your mental model of like benchmark versus vibes like evolved over the past few years doing all research?
What do you mean by that?
Benchmark versus vibes, like, you know, like qualitative sense, like you use the model, it just feels smarter, it feels better, versus, you know, having quantitative like...
Yeah, yeah, I mean, I do think they're actually quite highly correlated. And really, AGI is kind of a conversation, right? It's like you put a model in the world and then someone says, oh well, you know, I know it beats the benchmarks today, but it's not quite capturing what I feel like an AGI is. So then they contribute a benchmark to you, right, where you fail, and then you create another, you know, version of the model. And so I do think, you know, this is an iterative process. And I do think it's very highly correlated with vibes.
That makes sense. That makes sense. Switching to safety, what is the most exciting safety development for you in the past year?
Also a very good question. So I actually think o1 is probably one of the biggest safety improvements that we've made in the last year. And it's surprising because often times this is framed as a capabilities improvement, but it's also fundamentally a safety improvement as well. And what I mean by that is you can imagine trying to jailbreak a model, for instance, right? And you can think of the old GPT systems as they have to respond immediately, right? And so maybe they're more susceptible to being tried. But when you have a reasoner, the model can kind of reflect on, hey, is this prompt trying to get me to do something that's not consistent with what I'm trying or what I should be doing? And that extra amount of time that it gets to think and reflect makes it more robust to a lot of these safety attacks.
Was that something that you and your team expected?
Yeah, no, it very much was. And I think when we think about reasoning, it's something that's very broad-based, right? It's not just something that you use for math, for coding. It's something that's very transferable, right? The same reasoning that you may use to do well in coding may be applicable to like kind of how you negotiate or how, you know, play a very difficult game.
Yeah, that makes sense. And talking about benchmarks, do you observe like the same challenge on safety benchmarks?
Yeah, so I think the good thing is, you know, safety kind of mimics this kind of adversarial attack framework. There, you know, the attacks are fairly strong. So I do think we have a ways to go there. I will not claim, you know, our models are perfectly robust. And I think, you know, there's a very rich set of tasks there to improve on.
Makes sense. What are the key enablers to move us from level one today to level five of super AI? If you want to remind people, like, what is level one to five?
Yeah, yeah, so essentially we put out a framework at OpenAI over the last couple of months where we define kind of levels of AGI. And this is progressing from, you know, basic reasoners to more agentic systems to, you know, models that can take actions in the world and eventually kind of more autonomous systems, right, fully autonomous systems. And I do think really like robustness and reasoning is the key there, right? I think the reason you can't rely on a lot of the agentic systems today is they're just not reliable enough. And I think that's also what underlies our bet on reasoning. We've invested so heavily because we think that reasoning is what's going to drive reliability and robustness in the future.
Makes sense. Would you say that we achieved level two already? You know, are we getting there?
Yeah, no, I do think we are moving from kind of like one to two, right? Moving towards more agentic systems. Again, I think today, you know, a lot of the agentic systems do require human supervision. But increasingly, right, like you can see, you're more hands-off, right? You let the model do its thing, and we're building more trust in AI systems.
Awesome. Let's double-click on synthetic data. First, if you could remind people what it is, and then is there any good practice to generate synthetic data to train models?
Yeah, very good question as well. You can really tell the technical breadth of this crowd. Yeah, so again, synthetic data is data that's not kind of produced by a human, data that's produced by a model. And often times you see the power of synthetic data in kind of like data-poor or like low-data-quality style datasets. So one example where we've leveraged synthetic data is in training things like DALL-E 3. If you look at the DALL-E 3 paper, you know, one core problem of training image generation models is that when you look at captioned images on the internet, there typically is low linkage between the caption and the image that it's describing, right? Like you could have a photo of a hot air balloon and the caption instead of describing the balloon is more like, oh, this was the best vacation I've ever had, right? And I think in those cases you can really leverage synthetic data, right? Like you train something that's able to produce, you know, a high-fidelity caption for your image and you go and kind of regenerate captions for your entire dataset. And we've shown, you know, approaches like this to work very well. And you can think of like other areas in which you have like a poor quality on one side of your dataset.
That makes sense. Maybe related, is AI hitting a wall?
Yeah, yeah, that's a very relevant question today, right? Like, you might have seen some articles recently, you know, saying like, you know, a lot of these large foundation labs are hitting some pre-training walls, or even Ilya has come out and said kind of explicitly that, you know, we may be hitting some walls in terms of pre-training. But I do think our perspective internally is that, you know, we have two live paradigms today. It's an even more rich environment than we had in the past. Like, there's this test-time scaling paradigm that we've explored with the O-series of models, and it's really, really taking off. I don't see the same barriers there to scaling reasoning models. And, you know, we actually think that, you know, I've been at OpenAI since GPT-1, and every time you go from, you know, one to two, two to three, three to four, there are technical challenges, right? And often multiple technical challenges. I do think even in the four-plus world, we have a handle on exactly what technical challenges we need to solve, and they're very concrete. I don't think they're anything that, you know, we really just don't have a handle on.
