to ensure AGI benefits all of humanity. We think that humans should be firmly in the driver's seat, should have control over the future, and this technology should amplify human agency.
When I look at the world we're going to be in where software in some ways is becoming like the barrier to entry is becoming so cheap. You don't just want software, right? You want software that is reliable, that is secure, that you can believe in. And so cyber spend, security spend should actually vastly outstrip software spend. I think that people have not yet really internalized how much the physics around cyber security are going to change. I think there's an opportunity to end up with a much more secure world.
Hi listeners, welcome back to No Priors. Today I talk with Greg Brockman, the co-founder and president of OpenAI. We discuss OpenAI's latest math results, Astra, and why Greg believes AI can now generate genuinely new knowledge. We get into drug discovery dots and his case for why security spending should actually vastly outstrip software spending, which is quite a claim. Here's our conversation with Greg Brockman.
Greg, thanks so much for doing this.
Thanks for joining us.
So, every week in AI feels like a year, and a lot's happened this week for OpenAI. You rolled out GPT-6 to over a billion people, which is pretty amazing. There's some really exciting news that you have on the math side. There's all sorts of things undoubtedly that are happening internally. What are you most excited about that's happening right now, both at OpenAI and in the broader AI world?
Look, I think that in AI, like I have this belief about how OpenAI operates, that there's this cycling quote of it never gets easier, you just go faster. And I think that's very much what happens is that there's just so much activity and that every time you think you're at kind of peak sort of, you know, impact and how things are happening, you realize that there's actually another level that's possible. And it's not just about any one company. It's not just about any one technology. It's really about we as a world and society are all coming together to adopt this technology, get the most out of it to improve everyone's lives. And so I think to me the math results this week represent a real milestone, right? It's a real moment where if you think about like the fundamental driver of human quality of life improving over the past thousand years is technology, right? It's scientific advancement. And I think that we now have the ability to massively accelerate scientific discovery. And there's always been this question of is AI actually able to come up with new knowledge? And at this point, it's just impossible to deny, right? It is definitively yes, possible. Now, there's still more to go. I think we're not yet coming up with whole brand new theories, but we are able to resolve these hundred-year-old math problems. And to me, the thing that's very exciting is it's not just about the sort of, you know, just the machine just sort of gives you an answer, right? It's really what can a human equipped with these tools achieve that was unthinkable before. And I think it's almost like the leverage of humans to do whatever you want to do. It's going way, way up, right? It's now that you can, if you have a math problem you want to go solve, you can solve it. But what about taking that same capability, applying it to biology? What about if you're a small business owner and you want to be able to operate your business? Well, like that level of intelligence that's tuned for that domain, it's just going to mean that almost this sort of unthinkable results that are aligned with your goals, with your desires is now going to be within the reach of everyone. And that to me is so exciting.
The rate of progress has been pretty amazing too because if I look at it, you know, you used to have problems with arithmetic, right? In early models, I remember people complain, oh, why can't it do basic math? And now you're solving these advanced theorems. As these things continue to advance and this starts to really impact areas of math, physics, materials, etc., and various applications, is there some need if these models get so smart to focus on how to communicate that effectively to people or how do you think about the actual communication side of this as you do a very advanced theorem? Because if you look at any field of math, only a very small number of people can ever really adjudicate a theorem that's produced by a person.
Right. And so you kind of have this specialization that happens.
Well, and this, by the way, is something that has always been like, I came from a math background, like I thought I was going to be a mathematician and I was very inspired by mathematicians like Gödel or Gauss, right? These people who are working on like a hundred, four hundred year time horizon, right? And I think there's something deeply beautiful about mathematics being about sort of understanding the structure of the universe. And so it's not just about, and there's always these questions about is math invented or discovered. And that in my mind it's something that the beauty and the meaning of mathematics, it's not something that is objective in some sense. It's in the eye of the beholder. It's something that appeals to us. And there is some math that's like kind of physical laws and it's about simplicity and, you know, compression theory, you know, it's just complexity theory and all these things like that are maybe more physics, but there's something that is also importantly human about it. And so to me it's not just about, I think a lot of the conversation about mathematics is not just about here's a proof, no one understands it, great, we resolve the problem. It's really about what are the insights that are derived. And the thing that I think we will see, but I think that there's promising signs of life on, is that there's beautiful insights within the work that these AIs have done. And I think what we're seeing right now is that, you know, we published about almost 400 very important open problems that are now resolved and the mathematical community is going through all of that to look for insights. And if you read some of the things that people have posted, you know, domain experts in these particular problems and they say that it's like that there's these almost like sort of very surprising ways of applying ideas or incredibly like, wait, why did they use that result over here? And that what's coming out is not these incomprehensible theorems or proofs, which by the way, there is a history of incomprehensible proofs in mathematics, like four-color theorem is a perfect example of this, that it's like, okay, like we just had the machine check a bunch of cases.
Like there's no beauty in that. And so I think we're into a different phase of machine discovery. I have some optimism that like as models also get to better theory of mind, like the ability to figure out like what we as humans know and how to get us to follow them on the path to new understanding will also be paved, right? And so like for example, like I'm a math person, but at some point in school I stopped learning math and then I did Math Academy and then I did like ChatGPT to help me learn new math. And so I just feel like the expectation for, you know, the rising tide of what knowledge like you can expect people to understand will also be there.
I think that this is a really important point and one of maybe the underappreciated benefits that I expect from and hope for and we're starting to see from this technology is helping people know more, understand more, understand each other better, understand the world better. And I think that there are technical mechanisms that can underlie the, you have an AI that's able to go and solve these problems, see all the connection between domains, but how does it actually communicate to you? How do you understand it? And actually there's even connections to AI safety and alignment on having AI that we're able to understand what they're doing. And so there are techniques that we've been talking about since 2017 such as debate and iterative amplification which are kind of two sides of a similar coin where you really think about if you have AIs that are solving problems that are just too hard for humans to check. And even for example, summarizing a book is an example of a task that's really hard to check, right? It's like it's really hard to know is the summary any good unless you've read the whole book. And so if an AI just gives you a summary of a book and says this is the summary, is it a good summary or not a good summary? It's pretty hard unless you have another AI who is trying to prove that the summary is a bad summary. And so then if that AI comes back and says, hey, like here's this like paragraph from the book that I found. It's got clearly a very important plot point and that does not represent this summary. You don't have to read the whole book to be like, oh yeah, that's actually a very good argument. And so you realize that there are ways to train the AI that are actually very aligned with having humans be able to have oversight, to be able to be in the driver's seat, which is a fundamental value, a fundamental thing that is important for the future from OpenAI's perspective. But also are very much aligned with what you're saying of can you make the results of an AI be something that means you learn something that is really tuned for the human rather than just sort of knowledge for, you know, some abstract knowledge sake.
For most people, advancements in math theory are still somewhat abstract, right? So that there's like, wow, this is absolutely happening today. And then also, you know, AI for science, like there's a claim or a belief that it could solve all disease in some number of future years, right? How do you look forward like one step in terms of what we might see in advancements in between?
Well, I think that we've had a number of years and iterations of this technology to see the shape of what to expect. And I think that it's kind of like when you have an AI that is capable in a certain dimension and if you understand why it got capable in that dimension, then all other fields of human endeavor that have similar properties, you should also expect similar results. And so in my mind, like why is mathematics something that we've managed to really transform in the way that we have had such incredible results? And that part of it is that our AI is now at a certain level of capability and raw intelligence. Part of it is that you could think of it as that, well, mathematics is checkable, but that's not quite true anymore, right? Like we really started with very checkable problems of here's an arithmetic problem. There's like some numeric answer and so you have an AI that you can train to just like check if it got the answer right because we can very much see is the answer that the AI is proposing the same as the actual answer. But for a proof, very different. Proofs are very hard to check. Is this actually accurate or not? Formalized proof. Okay, yes, that is checkable. But natural language proof, which is what we're really producing here, much, much more difficult. But the thing that actually I think is the core insight is really about data, right? It's that you can get lots of data in mathematics. You get lots of data in theoretical physics. Can you get lots of data in biology?
That's obviously something very data bottleneck, very difficult. Now fortunately we also have been working on lots of techniques to be able to work with less data. And if you think about what that means is like, okay, we now have smart models that if there's something where there's a lot of data and this is the fundamental physics of deep learning, right? Compute, data, algorithms equals capability equals results. We're able to do it in these hard domains where people have spent hundreds of years of our smartest people all trying to solve the problem and now the AI can solve it. Amazing. When you transfer that over to biology and clearly what we want is drug discovery, all these things that are like fundamental changes in biology and our understanding of it to improve quality of life, that's what we want to apply. The question will be, and that is not yet resolved, do we have the data? Do we have the ability to get the data? And so I think that what will happen is that we will take these and we're already doing this where we take these models, these same techniques and apply them in the biological domain. And there are some areas that will fall quickly, right? That we'll just see massive acceleration of things that used to be incredibly painful now incredibly easy. But there will be other bottlenecks and there'll be a question of, well, how do we alleviate those bottlenecks? And some of these, by the way, are not even really about deep biological theory. For example, regulatory filings, huge bottleneck, right? It takes like a year to write up your whole sort of case for clinical trial and all these things. And it's just like you can write those things with an AI much more easily. And so I think that we're going to have to look at every step of the drug discovery process and there are going to be places where the fundamental data just does not exist. And so then innovations on wet labs, innovation on how you actually do simulations, all of this will become incredibly important. But so I think that we will see these breakthroughs coming, but it's important to know there are things that are shared with mathematics, there's about scientific discovery and things that are very different in terms of where the data comes from.
There's specific, you mentioned biology and I think that's a great example where a lot of the early research, AlphaFold and things like that, were really focused on designing a molecule or understanding how something binds to each other. And to your point, it takes 15 years to develop a drug. It's $1.5 billion. $50 million or so tends to be pre-clinical mouse studies, the molecule, and then $1.4 billion plus tends to be the clinical trial. And so your point is, and it's also, you know, two, three years versus another 12 years. And so your point is really well taken in terms of a lot of the early emphasis of these models has been can the thing mine versus what are all the other things we have to worry about downstream to actually turn it into a productive thing for human endeavors. How do you at OpenAI currently stack rank the different areas of scientific endeavor in terms of where you're focused or where you want to generate more data or more results or more capabilities that others can benefit from?
Look, I think that drug discovery is the most clear and obvious area if you get to pick one that you want to fully revolutionize and just places where you're going to really uplift everyone's quality of life that is just like so purely important and maybe the top one application of AI that everyone's been kind of thinking about for a long time. We clearly got to do it. And to your point, like there's this whole pipeline, it's just so long. And I think we're going to find that humans are incredibly bad at biology, incredibly bad at drug discovery. And if you think about it, that there's this like property where for humans, natural language, we're very well suited to it. Like it's something that, you know, very clearly it's very human. It's like very well tuned to how we have interacted with the world. Whereas biology, it's totally foreign. But for a neural net, I don't really think there's any reason to believe it should be better at natural language than it should be better, you know, compared to biology. And so the fact we're seeing such incredible results with neural nets in natural language, you're just like from first principles, you should be able to really be able to crack the language of biology.
And so to me, that is number one. It's like first principles, it should work in terms of what we're seeing. We're so close on so many axes.
Yeah, it's really exciting.
Yeah. And I think you can massively increase the parallelism. I think you can increase, I think you can decrease so many of these steps in terms of their duration. And I think, by the way, that there are many dimensions to drug discovery, for example, and of one diseases, there's even like lots of people who the drug to cure their particular condition exists. It's just that it's just out there somewhere in the literature that there's a paper. And if you can find that one paper that is relevant to your conditions, then you'll be able to cure the person. And this there's plenty of precedent for it and it can massively scale up those kinds of operation. So I think that impact in the short run is actually very possible. But material science, another example, and I think there's a long tail of other domains where we should just see real changes in kind of what is possible for humanity to accomplish.
It seems, just we think about why a drug might take 15 years to go to market. Part of it is you're doing like successively more expensive trials that have a, you know, different set of safety considerations resulting in large scale human trials. And then these trials today are, they're ordained by regulation, right, for good reason. It feels like one of the limiting factors, if you believe the technology will work and parts of the data can be acquired or generated, is going to be convincing regulators that we should change the expectations around what drugs are accepted and what the steps are.
I think there is a moment here where, and there's more work to be done to reach this point, but if you can really prove that the physics of drug discovery have changed, then of course it should trickle through to every part of the system.
And if you think about, I mean, even just take a very different example. Think about self-driving cars, right? That we have so much of our how roads operate and safety and all these things that are really tuned for human drivers. And if you have, if you reach a point where self-driving cars are 10 times or 100 times safer than human ones, right, that you've reached that point where AI is actually able to deliver the results in a domain that just are so unambiguously the thing that you want. Of course, you should then say like, okay, how do we rethink the whole transportation system? Maybe you should have regulation that says you only are allowed to have self-driving cars in certain areas like that. There are a lot of potential sort of changes to the way that you think about what is even trying to come out of a system that needs to trickle back into regulation as well. And of course, regulation is not uniform across the world, right? That there's very different regulations in Australia versus the US when it comes to clinical trials and that you're already seeing a lot of people moving their trials to places where the regulatory environment makes more sense for how things are, how they want to pursue and, you know, what people think is efficacious for them. And so I think that exactly how things should change, I think will depend a lot on what we actually can prove, right? Is it the case you actually reach the point that you can get better results from a neural net than from actually running a clinical trial? Like we're not there. But you could imagine we will get there.
But if they were more predictive on efficacy or tox or something, then obviously you should change it.
That of course, and I think that of areas of areas where I think society should and I hope will be very sort of leaned into the what is possible has fundamentally changed and let's very quickly then reorient around it. And again, you have to do it, you have to really think through all the consequences, right? And all of the implications. And that one of the, you know, if you're thinking about long-term studies of people and like question of like, you know, is this neural net actually in terms of what's predicting the stimulation actually accurate for long-term outcomes, it's pretty hard to not have at least had, you know, one long time tick to correspond to that. But I think that we will see like 80% of what is possible come to fruition very quickly.
You're a little ways out from the Astro release and Dev Day. Now, is there anything in terms of how people internally or externally are using some of the new products or capabilities that surprises you?
Well, I think Dots has really changed how many of us work and the thing that Dots represents is...
Can you describe Dots briefly?
So Dots is a proactive agent. It's always on, always working. You stop working, it keeps working. And the kind of thing that my Dots does, right? So I have a dot that's looking out for work that I was supposed to do, like messages I was supposed to send, things I said I would do that I haven't gotten to yet. And I always wake up in the morning, there's like a list of things that I was like, oh yeah, this is my to-do list. And it often notices things that I never even asked it to notice. If there's like a conflict on my calendar in terms of scheduling, it'll point that out. If it's like, hey, you actually need to leave this meeting earlier in order to get to the other thing. And so it's like so contextually aware. It knows what I'm doing with my day. It understands the projects we're working on. It understands, I sort of, you know, it knows what's in my inbox. It knows what's in my Slack. And it's able to integrate across these. And it knows the questions I've been asking about. So it knows what areas are most top of mind for me. And I think that having like in some ways what Dots is, it's a change to the form factor, not the capability, right? It's like it's an Astro level model. It's got its own cloud computer. But that the change in that form factor in a workplace setting with this level of intelligence, it's a game changer.
And do you have multiple dots running or you just have one that's sort of covering all your background tasks or anything?
I've got one. I've got one. It's a little owl is the icon. You can ask to change the icon. And it's just sort of doing everything across my work life.
OpenAI obviously has research efforts and then a consumer and an enterprise and developer business platform business today. Do you think that there's going to be a separation between, let's say, assistants that help you at work like Dots, which is how I primarily think of it, and then like consumer assistants like the Instincts, Muses of the world?
There's some, I think these things are going to converge, no question. And I think that there are some pressures that are very different in different environments. So for a mass market consumer product, really figuring out how to scale and to provide things for free or at very low cost is very important. Whereas in the workplace and if you're productivity, if you're busy executive, individual contributor, whatever your role is, then it is much easier to say peak intelligence and really as long as you're getting returns to paying more, it's worthwhile to throw more compute. And so I think that that is the main distinction is really around cost. And we've seen this in ChatGPT, right? That ChatGPT we put in a lot of effort to have the largest scale free tier and bringing things like memory to all of our free users was an incredible endeavor, like a real huge engineering effort. And it's just very easy to implement the same functionality at much higher costs for a very small number of people. And so I think that what we're seeing in the personal assistant workspace is that there are going to be these different market segments where different amounts of compute are going to be expended fundamentally. And so sometimes when there's different levels of compute, it then means the product can look a little different, right? That it can be that there's just more functionality that's embedded in one versus the other. And there's of course different use cases. You think about things like, you know, payments and travel and things like that are much more for the consumer, but I don't think there's any fundamental difference, right? It's like really just like an AI that's doing things and it just happens to be like slight sort of pressure in one way or another. And so one of the strengths of OpenAI is the fact that we have 1.2 billion consumers on ChatGPT and we have also a leading coding and enterprise business and no one else has both in house. And so a lot of our strategy is to say let's unify these so we're not sort of competing on two fronts, it's really competing on one front. And I think that this is also the general direction of travel of AI is to blur the distinction between what is consumer, what is enterprise. We're going to have this renaissance of entrepreneurs, like we're already seeing it. I mean, I imagine you're seeing this very concretely. Like I know people who are quitting their job to go start companies who say, I never would have done it if I didn't have an AI that I knew I could rely on. It could take care of so much of the normal barriers to entry. And so I think that we're going to see just like the distinctions have held for so long, I think they're going to change.
So one of the things that traditionally has separated the enterprise and consumer world is how you think about UI. Obviously there's your point on cost and, you know, underlying ability to deliver something for somebody at a certain level of fidelity, but the other part of it is just the user interface. And often the user interface in an enterprise setting or how you access it is very different from a consumer setting. To your point, you're looking towards unifying these layers. You've also been thinking quite a bit about like dynamic UIs and that's always been kind of one of the things people have talked about quite early is how do you just dynamically generate a UI that's useful for a user or in some cases these things kind of move into the background. It's just part of your chat. How do you think UI will evolve for these different use cases or how do you think about that as a problem as you sort of address both of these segments?
