OpenAI President Greg Brockman: Our Plan To Merge Chat And Agents

I think that's a pretty good perspective. And I think to really zoom out, the thing we're actually trying to build is an AGI. If you think about what people have been using since ChatGPT, it's a language model. There's a big gap between these. It's amazing. You can talk to ChatGPT, it talks back to you. Great. Wonderful. But when we launched in 2022, there was no memory. It's not hooked up to any tools. Has no context. And so it really is that this conversational intelligence is only one part of what people really need to get work done, to be able to achieve their goals. And where we're going is to have an AI that's really looking out for you. That you can provide the goals, the directions, that it's constantly thinking about what can I do for Alex today? That it's able to go and solve super hard problems, very mundane problems. You wake up, your inbox is organized, but also if there's a health plan that you are thinking about, that it can help you achieve that, figure out medical treatments, or provide you with that kind of information at least. And I think that the question of what's the interface you want, what is the product that you want, is what we spend a lot of time thinking about. And the answer is you want almost no interface, you want no product. You want this to be like what's the interface between you and me? Just being able to talk to a persistent entity of some form that's able to go and accomplish goals for you. And so building that is hard. It will take time, but we have a lot of the pieces. We're increasingly bringing together the product layer, trying to make the models better, trying to make the whole system just so there's less clicking buttons and toggles and changing modes and all these things. Not to say that there won't be some of those along the way, but the long-term trajectory is towards simplification, unification.

That's exactly right. And I think that if you've used Codex, and by the way how many people in the room have used Codex? People, yeah, decent number of people. And our goal is to really bring the power of Codex to everyone, to bring agents to everyone. That technology exists right now. You can hook up my Codex to Slack, to my Gmail, to my calendar. And there are many people within OpenAI, non-technical users. It's got code in the name but it's not really about code. It's really about having this general purpose tool using harness an agent. And the kinds of things, for example, someone on our comms team does is she was organizing an event and it would just ask all of the event attendees for their dietary preferences, set up a whole seating chart, kind of did all of that work so that she could focus on the parts that she wanted to and really thinking about the vision of what she wanted to achieve. And I think that we're going to see this across the board. So it's not sci-fi anymore to think about an AI that's hooked up to these tools. And I remember with our very first attempt at tool use in ChatGPT was 2023. I think in like March or April or something we released plugins. Do people remember plugins back in early chat days?

It didn't work at all because the models weren't ready. The form factor is correct. Obviously you're going to have an AI that's able to talk to your Gmail, no question. But we could only have like three different connectors exposed to the model at a time or start forgetting. We had like 2K maybe 4K token context. There's just no memory. It's kind of like when you had early computers in the 60s or 70s, you had tiny little memory banks and today you have your phone that's better than any supercomputer from that era. And I think that's where we're going with these models. The rate of improvement has been so steep. So now you can have hundreds of different tools accessible, we have the ability to hook them up to whole file system. So you can almost have the full power of the internet and almost any application you want at the model's fingertips. And it's smart. It's got 52 million token context, depends how you squint on it. And the capability level is also getting so powerful. These models are now solving unsolved math problems and physics problems, and really helping people be able to achieve things they couldn't otherwise. We are on the edge of this era of agents really transforming how we all operate, whether it's in software engineering, finance, legal, sales, and in our personal lives too.

Exactly. So it uses its Gmail connector, searches through your inbox to find all the people who are attending, and then if you're on the dietary restriction, it sees oh these people I already have their dietary restrictions, these people I do not, drafts an email depending on exactly how you have things set up. It might say hey I drafted these emails, can I send them? If you have a connector that doesn't even let it send emails, it says I drafted it, you need to send them. And in a different world, you could also imagine that you've built enough trust with the system where it says I drafted the emails and I actually sent them. And I think that this actually points to a really important aspect of the agentic era, which is trust. That we need to really learn how to build trust with these systems, where they're good, where they're not. Figure out what you want to delegate to them and how you want to entrust them with responsibility. And that's something we view as earned. It's not something that we can grant, but by providing lots of tools and control and oversight and supervision to the operator, to the person that this AI is operating on behalf of, we think that that is going to be such an important thing and so that's a key product feature and differentiator.

Well, first of all, I'd say that this is not theoretical at this point. People have been using Codex. It's a separate product, separate app. You have to install it separately. Really starting to focus on software engineering. But the amount of non-software work that has been happening in Codex has been absolutely exploding. It's been this incredible exponential curve, exactly the thing that you would expect. And within OpenAI, we basically have the same level of penetration now in usage as Slack. It's like everyone at OpenAI is an entirely Slack-based company. We do not use email for the most part. It's like really if you're not on Slack, you're not going to do any work. And it's kind of feeling that way now with Codex app as well. And everyone's Codex is hooked up to all of these tools. How the ecosystem evolves, I think it's going to be a very nuanced thing because I think one thing that is very important is that we believe that there should be an ecosystem that gets to be vibrant and thriving and that people can really build and see the benefits. And so we've actually seen this from partner companies where we said hey we really want to train our AI to be really good at using your software and we didn't know what they would say and actually the response we got is this is the most partner-friendly outreach we've ever had. The idea that you will make your AI specifically good at using our tool and they just see the opportunity because their tool will be used just so much more as a result and that everyone is trying to think about how do they not just survive as a company into the AI era but thrive. How do you really get the advantages of the fact there's going to be so much more activity and if you don't have AI in there, if you shut it out, then you're actually going to be declining, not thriving.

I think that you could describe it that way, but I think of it a little differently. The way that I think about this is what is the ideal interface to an AGI or we call it kind of a personal AGI. And I think that it's again the same interface that you and I are using right now. You just want to talk to an assistant. You want to talk to something that can go and work and operate on your behalf. And so yes, that agent, that AGI, that AI will have its own computer. It'll have its own access to things. Maybe an ideal coworker would be they can come over and type things on your computer too. So some access, some delegated access to your own system. And maybe you delegate access to your inbox sometimes, maybe it has its own inbox with some sort of window into the things that it needs, you forward emails to it. These are not actually unprecedented. It's like the way that you work with an assistant who's a person, or any coworker really. We've spent a lot of time thinking about how do you build these trust boundaries and make sure that you're able to operate together. And so I think of it as just a different thing. You could think of it as an operating system, but an operating system is almost something from a different time. It's a different layer of the stack. This is really more about how do you interface with technology broadly. And I think that the beautiful thing about AI is it's really about bringing the machine closer to the human rather than us having to contort ourselves into files and folders and all these details that somehow are not natural, that are more about how the machine operates rather than how we operate.

Uh no, no, I missed it. I was banned, but I watched it on TV. Come on, Apple. Anyway, it does look like you and Siri, the new Siri, are going to come into competition, right? Because they're an app that's going to sit, or an intelligence that will sit on top of all of your apps and let you take action. And ChatGPT will be an app on the iPhone. So talk a little bit about whether that positioning is going to be difficult for OpenAI and how you're thinking about that strategically.

Well, I just think again, think of it a little differently. I think that we're in the beginning of this new agentic era and the way that this has always gone in AI is that when you have a new level of capability, it means you have an opportunity to rethink everything. Rethink how people interface, how like what the technology is capable of. And I think that this is no different. In my mind, the kinds of things that I see on the horizon, for example, AI for solving scientific problems. And I think we're starting to see the inklings of this. For example, today we announced we have in peer-reviewed literature, people, doctors who are using o3. Remember o3? That was like forever ago now. That was like one of our earliest reasoning models, using that to find diagnosis for people who had no answers from doctors for many many years. There's an example of someone who had spent 20 years with a mysterious ailment. Finally, it's been diagnosed through the use of this technology. And if you're like, okay, you've got models that can do that. And then it's really about the same distribution and can you get access to an app? To me it doesn't type check. It's like we have something fundamentally new. And so that's not to say that there won't be competition. I actually think that there will be and it's going to be great for everyone, but I just think that the ways in which you're going to use this technology, the things it will be capable of and what it'll make you capable of doing are just totally different from anything we've seen before.

It certainly has been publicly reported.

Well, look, I think again I would just step back and say that I think this is the beginning of something very new and that I think about the way that... I think the biggest shift that has happened in terms of interface, again it's not even about devices and things like that. It's really about the shift from conversational intelligence, like the chat paradigm where it's like you have an AI that's personalized enough to you that it's worth reading its output. You ask it a question, you get an answer, it's something that's useful to you, to agents where they're capable enough to actually do things for you. That is a big shift and that implies a difference in how you want to interact. And so you kind of are just going to want a single agent that has access to your context. And this will be true in personal life. This will be true in a business context. You imagine, for example, having a PhD in every field coworker, Nobel prizes, multiple of them, and you hire one of these, you hire a hundred of them, and you don't invite them to any meetings. They're not going to be very useful. And so there's something about how do you get context into the AI and not just statically but dynamically as context evolves, as your business processes evolve. How do you have a context layer that is accessible to an AI that lets the AI operate to the extent of that raw intelligence? And so finding ways to make that AI be accessible, so available in your meetings, to make it very ergonomic, very easy to get access to. I think all of that's going to require a rethink, but I think it again it's just the core for me starts from thinking about the agentic form factor and then working backwards to how do you just make this have the context it needs and again the trust is going to be such a core part of making this whole equation work.

No. But no, more seriously, I think that the general shape of the technology, the way that we've had voice models, a really cool voice experience for a year and a half, two years now. We first demoed it back in March, April of 2024. Brought it to market maybe late that year. And the way that it works and the way that everyone's models work is that you basically chain together, well the original way that these things worked was that you would chain together a speech-to-text model, then you do a text-to-text model, and then you would do a text-to-speech model. Horribleness. Like these three things chained together. It still has been the case that even if you have one unified model that's able to take in input and then output a response, you still have this problem of turn-taking. Imagine that we have this, you cannot overlap, you cannot interrupt. It's just like once you speak to me in a turn and then you got to wait for me to finish my whole response. That is not how human conversation works. And so we basically have a hack where we have these models that determine oh it seems like the turn has ended and oh it seems like the turn has started and we're like why are we talking about turns? Turns again are so unnatural. This is the humans contorting ourselves to the machine and its limitations. And so the obvious thing that you want to accomplish is a model in AI that works much more like you and I do. That's able to process input at the same time it's processing output. And all of that is of course something that many people in this field are trying to run towards. I think it's going to be very exciting as you move to these natural, very human, fluid conversational interfaces. No one's seen anything like it. One thing that I think about is the current interaction with ChatGPT voice. In many ways it's magical. So many people use it on their commute, able to ask all these questions, but it also is so frustrating whenever it breaks the magic because it's like you realize, oh, I want to add some follow-up and it keeps talking over you and it didn't, it's just like that doesn't make sense. And so I think that part of what we need, the whole point of this AI is to be something that you can interact with fluidly and naturally. And by the way, I think it's not just going to be about the sort of use case, we kind of think about the personal use case, but it's also really the work use case. And I think some of the most magical experiences that I've had with Codex have been when operating it through voice. Many people, we have a voice built in. Some people use third party apps for it. And you just get a very different experience when you start to realize that typing a quick message to give some feedback is easy, but writing out a whole paragraph and everything you want is horrible. No one wants to do that. You just want to be saying things and you want the real-time feedback loop and all of that is going to happen and it's going to be amazing.

Well, I think that this is a place where when you're kind of building these models, you get a sense and an intuition that I think is harder to get from the outside because we see all the data points and we see also the work that goes into these improvements. And so there's two parts to the answer. One is I think that the fundamental science is one of the most mysterious and important just scientific discoveries and empirical observations that that I can...

That I'm aware of that I can imagine, that we are able to actually build these models and that the scaling laws continue. It just is the case that you can just keep training these models, more data, more compute, better architectures, and there's a lot of improvements that go in. But every time we've kind of run into a, 'Oh, this isn't quite scaling the way we expect,' it's we have a problem, we have a bug, that our math wasn't quite right, that our implementation isn't quite matching the math, whatever the thing is. And that is, I think, a very important thing to internalize. Actually, if you go back to the beginning of the field, neural nets themselves were designed in the 1940s, before computers, as a model of maybe this is how the brain processes information. First hardware implementation was 1959 with the perceptron. And if you look at landmark results in the field, they follow this incredibly smooth deterministic path of more compute being poured into them. So 70 years, maybe 80 years now, of people saying this stuff is never going to work, never going to scale, going to hit the wall. Hasn't hit the wall yet. There's still no wall in sight. So I think the fundamentals allow it. Now the practicality is hard. Actually building these massive supercomputers is hard, expensive, not easy. We have teams that work so hard to solve these incredibly hard technical problems. We have our own network protocol that we've had to design. We have people who look at every single layer of the stack. There's weird wiggles in the graph, and the way to think about these neural nets is that there's no abstractions. Any little piece that's wrong can have a ripple effect that only shows up down there. So you need people to deeply understand all of it. And yet, if you get the right team together, put the right mission in front of people, and people do that grind, the outcome is worth it. It's achievable and possible. So I think for those reasons, the progress will continue.

Well, I think there are several dimensions to the answer. Number one, I do think there's a bit of an attractor state where from a business model perspective, every provider sells out all their compute. Okay, right. I think that is just the world we're heading towards where there is not going to be enough compute to serve all the demand. We're heading to this compute-powered economy where everyone's going to be using these models all the time to accomplish tasks of interest. And we just see it. Right now we're talking about compute constraints, and the number of people using these agents is on the order of 10 million, 20 million. We're not at planet scale. ChatGPT is like a billion users, but we haven't brought the agentic power there yet. So you're looking at these factors, and the depth of usage is also tiny compared to where we're going. So I think we're just going to be in a world where even if you have different vendors, different capability levels, open-source models, all these things, neoclouds, compute is just going to be the scarce resource. It's going to go to use. So to some extent, is this a good business to be in for new entrants? My answer is actually yes. I think there is a huge market that we are not going to be able to address, and we need much more energy and momentum there. But a second thing is that it also misses the fact that intelligence is not a unidimensional thing. If you really zoom in, being good at different domains is something where even if you have a lot of raw intelligence, getting good if you've never practiced, like you've never actually done a pitch, you're not going to be good at it your first time. There's lots of different. You've never operated a spreadsheet, you're not going to be able to succeed at doing some complex modeling. So I think there is something we have been internalizing: we look across different industries and domains and we have to prioritize. We can't possibly be great at every single area at once. There is definitely a lot of 'hey, you just get the general intelligence up and it'll experience a lot of these things,' but to really become a domain expert, to really be that PhD, to really be something that can help push forward the ambition of a field, that's hard. By the way, one thing I also want to say is that understanding what happens when you successfully do that is important. Rewind to AlphaGo, move 37, that move changed people's understanding of the game, and now more people play Go than ever. It actually inspired people to do even more. I think we're just going to see that. So the depth is never going to stop. How deep can you go on science? People have sometimes thought we found out all the physics, we're all done. I don't think that's the future we're signed up for. I think we're signed up for one where every time you unlock one mystery, it unlocks like 10 more. So there's just going to be so much more to do and tons of room for differentiation across different companies.

Well, the way that I look at it is on the fundamentals. You need to really look at the fact that compute is multiple years out before it actually arrives, depending on exactly what you're doing. For example, we've been investing in our own chip program now for multiple years. Super exciting progress, we'll have more to announce pretty soon. But the fact that we're able to do that is something very unique. Really think about the full vertical integration of the supply chain. I think the world we're heading towards is one where again there's just not going to be enough compute in the world to satisfy all the demand. We see this very concretely. Look at the exponential of ChatGPT, look at the exponentials we're on now. Think about the problems we are able to solve. It's actually kind of interesting that we just yesterday, actually two days ago, announced a new result in chemistry, being able to synthesize new improved reactions. All of this is without much attention. The thing I just said, if you go deep in a domain, you can really transform it, and we're not even scratching the surface yet. So the way to think about it is the economy is so massive. We see it very concretely in terms of our own growth, in terms of what people are willing to pay and the size and growth of this whole industry. So the thing I think about the most is how do we meet the demand and how do you actually have something that can help support all the work people want to do in the economy? I think that is such a vast thing. I don't think any of us have internalized it yet.

Well, again, I look at it from a different angle. If you look at the whole history of what we've done, we actually have been increasing the intelligence and cutting price. For a fixed amount of intelligence, people somehow just like the Japanese paradox keeps happening. So I think frontier intelligence will always be something that is going to be the priciest thing, but a year from now that level of intelligence is going to feel pretty mundane and much more available. I think the world we're in is one where people are starting to really think about value. It's been a very interesting shift. Over the past first quarter, maybe up until now, people have been like, 'This AI agent stuff, it's all new. We need to bring it into our enterprise. We don't want to be left behind.' And now people are like, 'Okay, let's make sure this is actually delivering ROI and value.' I think that's a great place to be because people are asking the right questions. I had some customer meetings today where people were saying exactly this: 'How can we have even just good spend controls? How can we have observability?' We literally just released spend controls today.

Exactly. We are really investing hard in enterprise readiness and the tools that our customers are telling us they need. I think that shift we've also been going through as a company is really not just thinking about releasing models, but really thinking about the end-to-end of the business. How do we bring this into solving real problems for real customers? That is happening so quickly across every single industry, and the number of different companies that still feel like they're wrapping their mind around how to best make use of these models, we're learning at the same time. It's just so early in this whole game. The absolute size of the market growing so quickly, our own revenue ramp growing so quickly. I think none of us are anticipating how steep that's all going to go.

So again, the answer is always yes, right? But it's about what's going to keep happening. We're going to have frontier models. I don't think there's going to be a massive shift in the short term. But the thing you should anticipate is that over a year-long time horizon, to get to today's level of intelligence that feels very premier, it's going to be much cheaper. But there's going to be a new thing that is so much better, and you're going to be like, 'Why would I ever use this other one?' It's just how it's always going to be.

Well, look, I don't think that there's any layer of the stack here that is going to just be removed from the value chain. I think these things multiply together. If you think about the base layer of compute, it's like no compute, no AI. To some extent you could say compute is commoditized, it's just flops, who cares? But in reality, look at today's chip stocks, look at people who are selling compute, what the market is valuing people at. They see that there's a fundamental asset here that is so critical. That is because it is a revenue center. Anyone building AI has to rely on it, and there are a bunch of very interesting dynamics in terms of efficiencies, margins, all these things. But fundamentally, even though you can kind of squint and say it's commoditized, the value doesn't go away, the margins don't go away. It's something the market will reward because it has fundamental value, and its importance will go up over time. You can see that with some of the prices people are paying for H100s. Hoppers are a previous gen chip, and in any normal situation where you're not totally supply constrained, no one would be buying them. But instead, market prices are up relative to where they were before. So there's this inversion happening, and again, I think it's going to keep happening because everyone has this avalanche of demand. You're going to see prices, margins, all these things continuing to increase at various levels of the stack. I think the same kind of applies for models. The models themselves are also, again, there's a lot of competition there, and I think that's very good for the enterprise and customers and consumers. But I think there's a lot of areas where, for example, our models have always been the smartest ones, the ones able to solve these incredibly hard problems. We're just starting to reach a phase where you're going to see the transformative impact from that. If we're really able to speed up science through models, the smarter the model, the faster it goes. That's very different from a model with a conversational interface that can book your travel or organize your calendar. That's also a dimension we'll do a very good job in, but it's a different area. Then the question of how do you actually connect the intelligence to your own customers, to real value, you have all these enterprises that have built incredible businesses in different domains. It's not something where if you don't have domain expertise, you're just going to be able to do it right. Part of it is you think about regulated industries, education where you have a parent, a teacher, a student, these different parties that need to interact in very thoughtful ways. All these domains have a lot of value to be built by being in that area and thinking about how the workflow should work, how these models should be orchestrated. So I really think there's more than enough to go around, and we have to work together as a whole ecosystem to deliver the kind of value that is possible from these systems.

Look, I think the most important thing that is happening right now is the usage of AI in the economy to really transform the economy and uplift everyone. So I'm really focused on that. The more people are trying to make that happen, I think that's better for everyone.

I mean, look, you should always expect better, faster, smarter, the whole thing.

Definitely believe everything you read on Twitter. Yeah, maybe not.

Absolutely going to become standard. Absolutely. I personally have a number of friends who have done very similar things: get the data right, your health diagnostics, and use codecs, use these models to get insights from them. There are many people. I think there's about 230 million people each week who use ChatGPT for health queries. That's a staggering scale. These are people sometimes you upload a scan, sometimes you have doctors who are telling you conflicting information. I think we've been in a world where patients are not empowered. Patients have to be the doctor. You're the decider, you are accountable. A doctor makes a mistake and you're going to be paying the price for the rest of your life. It's a very different kind of incentive. This is very personal for me. My wife has a number of health conditions, and I think we've been, I don't even know how we'd be able to manage many of her conditions right now without the use of ChatGPT. We're just at the beginning of this journey. Even if you have the best medical team, the best access, the best experts, there's only so much that can be done. Think about the things that are just outside the reach of humanity, or sometimes it's like someone didn't even read the chart and missed a detail. All of that we should be able to improve massively through these tools. So personalized medicine, sometimes it's going to be about drugs and drug discovery for mass market, but sometimes it'll be for the kind of NF1 things, the disease diagnoses I mentioned earlier today. Sometimes it will be for trying to understand conditions and come up with new potential therapeutics. All of that is happening right now in front of our eyes. It's not theoretical. It's really happening. So one of the most astounding possibilities of AI is how much it can improve our health. Think about the ripple effects. So much spending on the healthcare system happens right now, that's a massive part of the economy. If you're actually able to help people prevent issues, get ahead of potential health problems, that alleviates a lot of burden and strain. We're in a world where doctors are burned out, nurses are burned out, there's a real crisis. AI will be able to help with all of that. We have that potential if we deploy it wisely and well. So applying AI to medicine is a personal motivation for me in thinking about this whole journey of what we're building, what we're trying to do with OpenAI. I'm hopeful that we as a world and community can make the most of that.

Let's hope. I think we will. I'm very confident.

Thank you.

You going to come? Yeah.