Ultimately, it was not any one specific incident. It was sort of this slow accumulation of things that made Dario and Daniela and the rest of that BATNA crew feel like they just couldn't trust Sam, they couldn't trust Greg. This had been a long process where he tried to make it work because he thought, you know, this is the lab that's going to make it to AGI and I don't want to, you know, waste time going and setting up my own thing and does the world really need one more AGI lab? And so ultimately, you know, he just reached his breaking point and I think that was avoidable but I don't think it would have been easy.
We're live with Kevin Roose who is a tech journalist. He was at the New York Times for around a decade. He was a columnist. He was the host of the podcast Hard Fork and now is the co-host of the new podcast Machine Gods. And he's the author of this new book, The AGI Chronicles, which just came out today. So, Kevin, welcome.
Absolutely. So, I think the most interesting thing in this entire book was something that a lot of people were not really talking about, which was this Dario memo called The Path to AGI where he laid out like how we're going to reach AGI. First, OpenAI would train an LM using all the text on the internet. Then they would expand the context window, train it on multimodal data, then use RL to improve its capabilities in math and coding, and then use its math and coding capabilities to recursively self-improve. And it's like, wow, that's literally exactly what the AI industry has been doing, like bar for bar, and Dario wrote this in 2019. So, like, what was he able to see that almost no one else at the time, even people in AI, even people who had read LessWrong, what was he able to see that they weren't?
I mean, I think he had been working on this intuition of his that scaling was the best way by far to improve AI systems. He had this other memo that, you know, is published for the first time in the book about the big blob of compute back in 2017. So he'd been kind of iterating on this hypothesis that basically once you have the right architecture, in this case the transformer-based large language model, you can kind of develop these scaling laws. And, you know, he knew the scarce information of the time that was not public yet in 2019 was that scaling was working and was working in a predictable way. And as he put it to me, you know, he looked at this thing that they were building, GPT-2, and he said, you know, we could add eight more zeros to this. Like we could train this instead of on thousands of dollars of compute, we could train this on millions or billions of dollars of compute and we could, you know, pour all of the data on the internet into it and then more and maybe it would just keep improving steadily. So I think that was the thing that he kind of knew before anyone else by virtue of just being part of the team that was, you know, running the team that was working on scaling.
Well, similarly, you know, AI has been extremely popular this year. It's been popular among those in the know on Twitter or whatever for maybe four years now, four or five years. But it seems like all of the ingredients were already there. And about 10 years ago we had OpenAI already founded, it was backed by the most famous tech billionaire on the planet. We had AlphaGo about 10 years ago. We had clear evidence of some LLM scaling stuff. So, like, what took everyone so long to take it seriously? Like, yes, I understand that the formalized versions of the scaling laws weren't public until around 2020. But like, it just seems like the entire world missed out on perhaps the most important thing for such a long time.
I think that's fair. I think there were people even inside the AI industry. I mean, one of the chapters in the book is about Google and how even though they had invented the transformer and come up with so many of the building blocks of modern AI, even though Ilya and Dario and all these superstars had once worked there together at Google Brain, like they just could not stop building these sort of smaller, narrower, single-purpose models. That was just part of their DNA was like you have one model for translation, you have another model for recognizing the images in photos, you have another model for doing like autocomplete inside Gmail. And the idea that all of this could be done by like a single multimodal model was just kind of anathema to them. It was offensive. You know, the bitter lesson is called the bitter lesson for a reason, right? It is painful to watch your narrow progress in one domain just get totally obliterated by some big general all-purpose model.
I want to shout out our sponsor, Lovable. You know, the app you've been meaning to build, the internal tool, side project, or product you'd otherwise lose a weekend to. With Lovable you get software you can ship today that includes an editable codebase you can inspect and change, plus two-way GitHub sync. Lovable also handles the backend and infrastructure, including managed Postgres, object storage, hosting, payments, and more. So you don't have to wire it all up yourself. And through Lovable's MCP server, you can create and deploy your project directly from the agents you already use. Turn ideas into software people love with Lovable. Lovable.dev. Now, on to the episode.
If you had to pick a top three or top five moments in the history of AGI that will be seen as the most important, kind of like how today in American history class you can boil it down to like the Revolution, the Civil War, World War II, the Civil Rights Movement, whatever. Like, what are the equivalents of this for AGI?
Well, one of them has to be the moment where Ilya Sutskever shows up at the desk of this junior OpenAI researcher, Alec Radford, clutching a copy of Attention Is All You Need, the transformer paper, and says, we have to experiment with this. Like that is sort of the Eureka moment where all of this gets born. I think another moment is the evening in 2023 when a couple of researchers at OpenAI are sitting in a conference room looking at the reasoning traces from the first reasoning model, this model Q* that sort of displayed for the first time that you could tell an AI model to slow down and think step by step and that it would do better on math problems. So that was another moment. Obviously the moment of Sam Altman's firing and rehiring, the blip, is a big one. I also think, but you know, was it that historically consequential? Was it that like counterfactually important? Like a little bit.
It was amazing. I was there. I was watching it.
I think it was historically important. I think had it not happened the way it did, like things would look very different. Maybe I would still be at OpenAI. Maybe if they had carried out the coup and not brought Sam back, maybe they would all have ended up working for Microsoft and being sort of subject to their corporate bureaucratic pressures. So like I think there are many different ways that could have gone, but I think that was a moment at least for the governance if not for the technology. But I was really interested in telling the story of the technology. So I love these like little Eureka moments where things just start to work in a way that they hadn't before. And so there are a bunch of those in the book.
