We are here with Greg Brockman. He is the president and one of the co-founders of OpenAI and we get the opportunity to chat with him, ask him some follow-up questions following Sam Altman's keynote, kicking off the day. So, Greg, thank you so much for being here and for helping millions of people figure out what the heck is coming next.
I'm excited to be here. Happy to chat.
So, you were the founding engineer and first CTO at Stripe. You are on a mission to become the best engineer in the world. If you were meeting with the best engineer in the world and maybe it's you, what is the best engineer in the world doing right now?
Working with Codex. But I think that being in AI is just such an incredible opportunity right now. I think that we've very much been seeing that people who are engineers in startups or in other domains, HFT, anything like that, coming into AI can have this massive impact. So I think that the best engineering is possible and is really rewarding within AI.
Do you have specific features that you're thinking about when you think of the best engineer that is super ahead of the curve? Day-to-day, what are they doing?
Well, I guess the way that I think about it, I can't comment necessarily on every engineer. I can talk about myself. The way that I work is that I'm constantly context switching. I'll be writing some function or something and I'll hit a little mental block. I'm like, oh, exactly how should this be structured? What should this abstraction be? Something like that. And I'll flip to Slack for like 10 seconds and then I'll flip back or I'll go to email. I find this constant reset means that I can background process a problem, and by the time I get back to it I know the answer. So I think it's really about iteration speed, about problem flux, and just trying to get as much context and speed as possible.
This reminds me of when people would go to sleep with a math problem in their head in high school and they would wake up and it wouldn't solve, but you're doing that while awake.
Yes. I actually literally did the fall asleep thing. I had this research problem I'd been working on in high school in math for months. I remember I was working on something like Fermat's Last Theorem, not quite that advanced. It was a problem in de Bruijn sequences, which is pretty cool stuff. But I remember falling asleep and I was half asleep and I saw the answer. It was this phenomenal experience. I remember feeling like I need to figure out how to replicate this. There is something about the subconscious just chewing on a problem.
So almost like early exposure is one of the most helpful things. Getting in practice cycles is definitely one of those starting points.
Right. And I think it's also just doing. I think the most important thing if you're starting out is not to be afraid. You write some code, it doesn't work, fine. There's no consequence to it. I think it's really about trying to find projects where you can learn, where you're a little bit uncomfortable, and where you can just go and try. Especially with tools like Codex, it is just so much more accessible to get into programming.
When I was showing people ChatGPT and they had never seen it, more often than not I would ask it to write a poem about their hobby and that was just a really fast way of seeing the magic. Is there a 'do X for Y' that is a great starter spot for Codex users?
Well, Codex is incredible at front-end engineering. I have a particular hobby project that I started working on. The first time I built it myself was back in 2008 or something, which is a competitive chatbot game. I call it a reverse Turing test. Traditional Turing test, you have two human participants, one of them is a judge, one of them is a player. The goal is for the judge to be able to tell if they are talking to a human or an AI. In my competitive reverse Turing test, you are hooked up to two terminals, each player. One is your opponent, a human, one is an AI. Your goal is to tell which one is which before your opponent does the same. The interesting thing about this is that the optimal strategy for the bot is to act kind of botlike, or for the human to act kind of botlike, because you don't want to distinguish whether or not you are the human. But also to ask somewhat discerning questions to figure out if your opponent is that or not. Back in 2008, I wrote a heuristic-based bot that would replay conversations from its database that were similar to the ones it was having currently. I got like a 30 or 40% win rate. It shows you that it was just a cool problem, a cool setup. The reason I bring this up is that I've actually started using this as a test of vibe coding. I just describe that to an AI. I've made versions of that with zero lines of code written by myself that are 10x better than the thing I wrote back in the day.
Really? And when you see that happen, are you excited because you're adding on new things, or bummed because it took your best idea and just did it?
For me, it feels so empowering because I remember the struggle of implementing this back in the day. There were a bunch of CS fundamentals I did not know. For example, serialization was something I'd not heard of, like JSON formatting or whatever your favorite serialization library is. I remember I just invented my own, like I'd have this secret delimiter that was a sequence of characters that would be unlikely to appear. I can't tell you what it is because then you'd be able to break my serialization scheme. I would just use that to separate records. It's a concept where when you're starting out, you just wouldn't even realize what the question is you should ask. Whereas today, an AI looking at that code would point out to you, hey, this is a much better way of doing what you're trying to accomplish. Or you can even just do so much more than is possible otherwise. For myself, I've always been an infrastructure backend kind of person. My front-end skills are a little bit atrocious. I used to basically just do the hackiest possible HTML, JavaScript, CSS, and then have a designer come and clean up my mess. Now I don't need to. So for me, it just supplements me in a way that I've always wanted.
I love that. As someone who's only capable of making horrific HTML, I'm with you. Have you interacted with an engineer recently where they said something about the way they're working and you said, 'Wow'? What was that person doing that might have surprised you?
Well, I actually remember around December or so, one of the engineers on my team who's absolutely phenomenal, one of the best engineers I would say in the world, telling me that he was getting a lot of value out of the ChatGPT desktop integration with his terminal. It was a very simple thing. He was just like, if there's an error in my terminal, I just ask ChatGPT what went wrong here, and it debugs it for me. You just realize the little pieces of friction. Obviously this engineer is capable of reading the error message, but it's annoying, it's overhead, it's slowness. I think that what I see is people trying to find where AI maps to their workflow, where it can remove little bits of friction. Sometimes that was the thing that made the difference. But all these things compound. People who are very discerning and focused on their own productivity and tools, it propagates into how people structure code bases. You think about, well, if I want to be using AI tools as much as possible, then how do I have code set up so that it's most amenable to AI? Fast tests, modular code, good READMEs for the API definitions between things, all of that stuff.
It's almost like, I know everyone talks about AI as an operating system. I lived in a 20-person AI house in San Francisco and I think that term was thrown around all the time. What I'm seeing now more and more with these real-time overlays is that it's not just the foundation of how things are built, it's also watching over, for lack of a better word, and co-working or pairing with everyone in every single line of business. It's amazing.
Yeah, it is really phenomenal. And I think we're still at the beginning.
Even at the beginning, that's this slope.
That was an exponential. I think we're still just at the point where it feels a little bit super for early founders right now. That could be Silicon Valley early founders or it could be an SMB solopreneur coach in the middle of their career. What are they building? What should they be building today?
Well, I think two things. One is honestly there is so much to explore that I think people should just, if you have an idea, build it. It's as simple as that. The barriers to entry are the lowest they've ever been. They're going to get lower. Don't worry. But we're at a point now where if you have a creative idea, you should see it, implement it, try it out. I think that is maybe the highest order bit. If you're looking to build a durable business, I think thinking about what are things that will benefit as the models get better, not be obsolete. Things that require going into specific domains, building a lot of expertise there, that's the kind of thing that can be really durable in this time.
Okay, so vertical AI, applying your expertise using AI to support that mission.
