We spoke with tech journalist Kevin Roose about the AI development race over the past decade, laid out in his new book, the AGI Chronicles. This ASI thing. You know, obviously, the president has said we're gonna redub it ASI. Is that something that was already being used in the industry? Because we've heard from Intel folks that this could be confusing because it refers to something else within the intelligence world, but this is a term... Seems like it's on your book. It was already out there.
Yeah. This is basically the sort of the thing that comes after... So there's, there's AI. Yeah. There's AGI, and there's ASI, which is artificial superintelligence. And AI is sort of like everything, you know, we have now. AGI is this kind of point where the AI gets roughly as smart as humans. It's roughly a human-level AI system that can do a bunch of different things at or around human level.
You do? That's what I was gonna ask.
Okay. I mean, people disagree about what the exact definition of AGI is. But for me, my sort of personal bar has been cleared. But that's what the whole book... That's what the people in this book are aiming for. They're not trying to make this thing human level at one or two or six things. They're trying to make it human level at many things. And then ASI is the step after that. That's when you have an AI that is better than every human at everything. And so there's sort of... Yeah. It's just that... And that is actually the ultimate goal of this. They are not just... The people who have been building these systems for the last decade will not be satisfied if these things top out around our intelligence level. They want to build something vastly smarter than the smartest humans who have ever lived.
The principals of this book have been wrestling with the kind of broader implications of this technology for a long time, but it strikes me that in recent weeks, recent months, the public has become aware of that. Wondered how well equipped is a Sam Altman to decide whether something is moral or not or what the implications for humanity might be. You've followed and spoken with the principals in this book. How well equipped are they to be wrestling with these kind of broader philosophical and existential issues?
Well, they might be well equipped relative to the average person, but there are so few of them. I mean, that... This is the thing. Like, the thing that I realized reporting this book... And I talked to more than 150 people at the AI labs that I covered: OpenAI, Anthropic, and Google DeepMind. And that was essentially how big the group of key decision makers was. It started off as, like, a couple dozen people who were interested in this about a decade ago, and they grasped something about the world that wasn't obvious, which is that powerful AI was possible and that some recent advances in things like GPUs had made it seem realistic. And then the circle kind of stayed small for a long time where, like, maybe 100 people, maybe 150 people actually were involved in creating the technologies that we see today. And no matter how good or moral or well equipped those people are, like, it's a very small group of people to be building this thing and making all the decisions that could affect billions of people around the world. And by the way, it's a very homogenous group. Yes. You know, these are, like, mostly men in San Francisco. They are blinkered. They don't sort of have the rest of humanity's experience to draw from, and they're all racing each other. That is the craziest thing. It's like they understand the stakes of what they're doing. They're not naive if they think this thing could blow up the world, but they're all racing toward it anyway.
But, like, to what end? When you say, okay, we're possibly on par with human intelligence. The goal is to be better than that. I mean, I say this jokingly, but also not. Did no one watch the Terminator? Like, every piece of science fiction we've made in the last 25, 30 years goes to the same inevitable end. You know these guys are aware of this. They're just, they're just doing it anyway?
Yes. And they have various justifications for that. Some of them think that the future will be amazing, and we'll have this sort of post-scarcity utopia where robots are doing all the jobs and humans are just free to pursue our leisure activities and, you know, make art and think about philosophy all day.
Exactly. So that's one rationale. The other rationale, the one that motivates several of the characters in this book, is that they think this technology is going to be built whether they build it or someone else builds it. They think it is inevitable. The recipe is not that hard. Get a bunch of data and a bunch of compute and some algorithms, and you throw it all into a neural network, and out comes intelligence. And so their fear is that if they don't build it first, if they don't get to this threshold of AGI or ASI first, then China or their rival down the street or someone they really don't trust will do it first. And that is the sort of fear motivation in addition to, like, the, you know, it's all gonna be utopia motivation.
Kevin, it's a small group as you've pointed out. Now we have policymakers and politicians interested in the prospect of regulating this. We've had a lot of conversations with lawmakers in which we've asked quite bluntly, is Congress equipped to do this? The answer is no. We do have a self-proclaimed high-IQ president. But what is the barrier to regulating it? Is it momentum? Is it Congress's lack of desire to do it, or is there really no way in which the government could, given the sophistication of all of this, have any hand in regulating this smartly?
