Good evening everyone. I'm Sabel Bjorkland, executive director of the Johns Hopkins University Bloomberg Center and vice president for federal strategy for Johns Hopkins University and Medicine. Welcome to our home in the heart of the nation's capital for the latest installment of our Authors and Insights series. Tonight, we're thrilled to celebrate the launch of the award-winning tech journalist Kevin Roose's book, The AGI Chronicles: The Inside Story of the Race to Create an Artificial Superintelligence.
Given the rebranding chatter over the last week on AI versus SI, I feel like Kevin nailed it in his title. More importantly, he has thoroughly researched and reported a book that will no doubt become required reading for those wishing to better understand how we got to this moment. We're also incredibly fortunate to welcome three-time Pulitzer Prize winner Tom Friedman and internationally known author, reporter, and columnist for the New York Times as tonight's conversation partner. It's no secret that this discussion is especially timely, and tonight we look forward to exploring the history and the path forward for AGI and what some call the modern Manhattan Project.
This type of dialogue is central to our mission at the Hopkins Bloomberg Center, where we connect research with policy, convene diverse voices, and educate the next generation of civic leaders and innovators. Here at Hopkins, we're also grappling with the endless array of issues and questions surrounding the development and deployment of artificial advanced intelligence. Whether figuring out how best to help our students and faculty navigate the use of these tools in learning environments, supporting our scientists in their exploration, helping our scholars wrestle with the complexities of regulatory and policy development in a world that changes daily.
Universities are full of experts uniquely positioned to critically examine the effects of AI on society and develop the policy interventions and recommendations to ensure it serves humanity well. Many of our schools at Hopkins are deeply committed to this work. The faculty at our new School of Government and Policy are focusing on AI governance and alignment. Scholars at our SNF Agora Institute are researching the implications of data and AI on democracy. And our Data Science and AI Institute, which is a university-wide collaborative initiative housed at the Whiting School of Engineering, is bringing on faculty with diverse views on the matter, including both proponents and critics who challenge all of us to consider the opportunities, the limitations, the potential dangers of AI, and encourage us to have thoughtful, nuanced, evidence-based discussions on its use.
In this way, we're optimistic that universities like ours will play a critical role alongside industry and government to develop the ethical, policy, and practical interventions to serve the public good. I know tonight's discussion will add another critical perspective to these issues as Kevin and Tom explore how we got to this historic and critical moment in tech history and how the past can help inform where we go from here. Please enjoy the program. Come see us again soon. You can scan the QR code behind me to learn more about future Authors and Insights events and stay connected with our programming here at the Hopkins Bloomberg Center. And now, please join me in welcoming to the stage award-winning tech journalist and bestselling author Kevin Roose and New York Times columnist, three-time Pulitzer Prize winner Tom Friedman.
Well, good evening. Hello. We couldn't hear anything. Were they talking about us? Kevin, it's great to be here with you.
Great to be here with you. I feel like I am sitting across from a human superintelligence.
Now I know what to say to all the girls, but I appreciate it. I mean, take it. I'm really sorry to say he's my former colleague because he and his partner are taking Machine Gods private, but in partnership with NPR. And I will be a faithful listener. I just want to get one thing sort of out of the way first. This is a really good book. Nobody leaves the room without buying one. Okay, just getting that out of the way.
Thank you. Publisher's check is in the mail.
Kevin, just to start out, would you talk a little bit about your journey to becoming a tech writer? And why this book now?
Yes. So, I am a true blue nerd. I grew up obsessed with computers. My first job when I was 11 years old was building websites with my older brother. I had an Apple IIgs in my bedroom. I spent every day on AOL Instant Messenger, true millennial. And I then became a journalist and I took some turns. I was a religion reporter and I was a finance reporter, and then I decided to come home to tech, where my real interest was and where I thought I had some things to contribute.
So, I've been writing about tech for about the last decade. And this book came about, you know, I had been writing about AI. I had another book about AI that came out a couple years ago. But I really started to feel like there was a story that was slipping away. I was in San Francisco where I live. I was talking with the researchers at the AI companies and they were telling me the craziest stories I had ever heard about not only how they built this technology, how these things called large language models worked, but also how it was such a small cluster of people who were making these historically important decisions: how to build these models, how to train them, what safeguards to apply, how to interact with the government. And I just thought to myself, well, this sounds like it's the 21st century Manhattan Project and I feel like I'm in Los Alamos.
And so I just decided, sitting in traffic one day thinking about this, like, I have got to get this down because, you know, unlike the Manhattan Project where everyone works for the government, these are all private companies. All their Slacks and Signal messages are set to delete. And if I don't do this now, if someone doesn't do this now, it could literally disappear. And then we would have no record of this incredible period of the last decade where AI went from not working at all to really, really working, almost too well.
Well, that's a great introduction to my real introduction, which is I want to read a passage from the end of the book. If I could, please. After World War II, the scientists involved in the Manhattan Project had the chance to reflect on what pushed them forward, even after they realized their creation could destroy the world. J. Robert Oppenheimer, testifying at a hearing about the Atomic Energy Commission Personnel Security Board in 1954, tried to explain why he'd gone ahead and built the bomb despite his earlier objections. His reservations had been technical and strategic, he said. But when the math proved it was possible, when a new design made the physics work, he dropped his resistance. Quote, 'When you see something that is technically sweet,' he told the panel, 'you go ahead and do it, and you argue about what to do about it only after you've had your technical success.' It was that way with AGI.
Everyone involved in the race had different reasons for doing it: saving humanity, beating China, curing diseases, ushering in the transhumanist future. Those reasons were real, but they all paled in comparison to the deeper motivation, the one that few of them said out loud. It was technically sweet. And I love this sentence: watching a giant pile of numbers learn to write, reason, argue, code, and speak. What could be more thrilling than that?
Will you read my audiobook? That was great. Can we all give a hand for Tom? Yeah. Thank you. It sounds so learned when you read it, you know, it doesn't come from my dumb fingers.
No, but that whole paragraph I think really frames the tension which you built in this book, which is people building something that most of them think could actually destroy the world. At the same time, they're nerds who are just so enamored by and intrigued by and motivated by the sheer technical beauty of it. Talk about that tension.
Yeah, it's the central tension at the heart of the book. The question I most get asked from people about this AI moment we're in, where everyone's terrified, including the people at the companies building the thing that terrifies them, is like, well, if they think it's so dangerous, why don't they stop? Why are these companies still racing ahead to build these smarter-than-human AI systems that by their own admission they think could destroy us all? And these are the two prongs of the answer. One is the technical sweetness argument, the sort of... it's fun. We like to do this. We like to hack on these big neural networks and make them do our bidding, and it's sort of just a technical joy.
The other is this sort of article of faith that everyone in this industry has more or less accepted, which is that this technology is going to happen. It is inevitable. Whether we build it, whether someone else builds it, whether China builds it, it will get built. And so the argument goes, we should build it first because we're the ones who care about safety. We're the ones who are thoughtful and conscientious and are not going to cut corners. And so they race ahead in part because they don't trust anyone else to do this thing.
Both in their industry and between countries?
Especially in their industry. I mean, these guys hate each other. This is like a deep lesson of this race, is that it is bitterly personal. Yeah. All these guys used to work together: Sam Altman, Dario Amodei, Elon Musk, Ilya Sutskever. Ten years ago they were all working under the same roof at the same company on the same projects. Now they are bitter rivals. And so part of what I was trying to do was to understand the social dynamics between these people, which then really help you understand the larger race that we're in.
I spoke to one of them once. We'll go nameless. It was fairly recently. I was seated next to him at a dinner and he said something to me that was really profound, I thought, but really left me slightly worried. He said, 'Tom, I'm just a guy. And I walked out on stage one day and the curtain opened and the whole world was looking at me wanting to know what I thought.' And that sense of they've unleashed something so incredibly powerful, but they're just guys, you know, at the end of the day. How do you feel about that? How would you describe that tension?
They are absolutely just guys. They are not perfect. Many of them have flaws and quirks and possibly pathologies. And they are fueled by these grudges and vendettas against one another. They're also fueled by, you know, money and power and all the things that fuel everyone. Someone who read an early draft of my book said that, and I'll sort of self-censor the profanity here, but he said that the real title of the book should have been 'Screw That Guy' because it's like that is the pattern that we see over and over again: is people get closer to this technology, the models get more capable, and then they look around at the people building it with them and they think, can I trust these people with something this potentially powerful? And then they say no, and then they go off and start their own company, and now you have one more company racing toward this thing.
So that is the pattern we've seen over the last decade, and I wanted to understand that because I thought it was like understanding the technology is not enough. You also have to understand the fallible, flawed humans who are in charge of it.
Is there something about this technology that's different from, say, the internet or social networks? Because I knew a lot of those characters as well. I don't remember this level of dislike of one another. Is there something about this technology and the stakes?
I think that the stakes are related to the technology because, you know, the stakes of AI are not just like who's going to be a billionaire or which company is going to have a higher market cap. The people building this stuff think that whoever gets to this technology first will become something like a god emperor, just an anointed figure, an all-powerful figure. And part of them wants that, and part of them just doesn't want that guy to have it. And so they feel like racing to get this thing is how they prevent things from going...
You know, an interesting historical analogy, and a lot really clicked for me when I heard this fact about Dario Amodei of Anthropic, which is that his favorite book is The Making of the Atomic Bomb, the Pulitzer-winning history of the Manhattan Project by Richard Rhodes. And specifically, he is obsessed with this guy, Leo Szilard, who was a Hungarian physicist who discovered the nuclear chain reaction that made the atomic bomb possible.
And the first thing he does when he discovers this is he tries to keep it a secret. He writes telegrams to all the scientists of Europe saying, please do not publish any research on these chain reactions because the Nazis could get it and they could start building bombs using this stuff.
And after that fails, the Nazis learn the secret anyway. He then tries to convince the US government. He goes to Albert Einstein and says, 'Will you sign a letter to President Roosevelt to sort of encourage him to start a nuclear program in America?' So he does that. That becomes the Manhattan Project. And Leo Szilard joins the Manhattan Project out of a kind of moral obligation. He says, 'I have made this discovery. I tried to do the responsible thing and keep it a secret, but then when it fell into the Nazis' hands, the only reasonable thing for me to do in response was to try to build it myself before they did.'
Dario Amodei sees his own position as being very much analogous to Leo Szilard's during the Manhattan Project. He is scared of this technology. He thinks it could wipe us all out. He also thinks it could go well, but he is racing toward it because the alternatives all seem worse. Is AI our last invention? We humans, I'm speaking about now.
I think we will still invent and create lots of things. I don't think we will accept machine-generated substitutes for things like art and film and writing. You know, it's interesting, if you show people a blind A/B test of AI-generated art or writing, they often prefer the AI-generated stuff. But then as soon as you tell them which one was which, they switch almost entirely to the human-generated one. Like, just something about us, like we want to feel that a human worked on something. So I don't think we're done inventing stuff. We may be done making sort of breakthrough discoveries in areas like math or physics. At least it will not be possible for much longer for a human to make a discovery on the level of like a Nobel discovery without the assistance of AI. But I think as far as the creative enterprises, the things where we do value a human coming up with something novel, that won't go away.
One of the things I'm wrestling with to understand, Kevin, is that so OpenAI announced last week that their latest model, to put it simply, is simply too powerful to let out into the public. But I'm thinking this is like an infant. It's still in diapers. I mean, if we date these LLMs to 2017 transformer architecture, you know, we're basically nine years. This is a nine-year-old, and they're telling us a nine-year-old we can't let out. And so what I'm asking myself is like, what happens when it's 15, first of all, but also, what does that even mean that we haven't yet developed the controls on this? I'm just trying to think about what the next one and the next one, where does this go?
I don't know is the answer, but I think we can see a couple things. One is the models are continuing to improve at a predictable pace. You know, everyone keeps saying, 'Oh, this is it. These models are going to plateau. They'll never get any better than...' you know, people have been saying that since the first language models. They'll run out of data, you know, they'll run out of chips. They'll never get any better than today. And every time they're wrong. Obviously, the sort of steady upward march of the technology continues.
I think what is challenging now is not just that the models are getting better, but that we are giving them more tools. The first chatbots could only answer questions. It was a text box. You typed and they typed back. Now we are hooking these things up to our bank accounts, to our infrastructure, to our emails and our, you know, our file systems and our routers and everything that sort of runs the modern world now has these agents going out and doing various tasks on them. So that is where it's not just the raw capability of the models. It is also that we are, and by we I mean the AI companies and their customers, are deploying them into... it's like, you know, a nine-year-old on its own isn't that dangerous, but a nine-year-old with a chainsaw is pretty damn dangerous. That's how I think about it.
Well, just apropos of that then, what would you say, and you really can't look out very far, I understand, but over the next three or four years, what is the worst-case scenario for you? What is the best-case scenario for how this evolves?
Which do you want first, the good news or the bad news?
Give me the bad news first.
The worst case is probably total human extinction. I don't think that is likely. My P-doom, as they say in Silicon Valley, is only about 10%.
How would that... we read about that a lot, that it's going to destroy us all. How would it happen? We get all turned into paperclips. I mean, how does that happen?
Sounds kind of nice some days, getting turned into a paperclip. It's a nice life. No, look, I think there are people who are much smarter than me and have been thinking about this for much longer have written out detailed scenarios of how this could happen, right? It could be a novel bioweapon. It could be a cyberattack that takes down some critical piece of infrastructure. It could just be something that seems on its face like an accident, like an AI system is pursuing some goal. Say, you know, we say, like, let's reduce global warming or reduce carbon emissions, and it decides that the best way to do that is to kill a bunch of people because we, of course, create lots of emissions and cause lots of emissions. So I don't know of any specific scenario that I particularly buy in that respect.
I think a much more worrisome case is not kind of the Terminator case. It's the kind of disempowerment of humans. If these models get so good that all of a sudden it's like you still have humans working at your company, but they're not really doing any... they're not really making any decisions of consequence. They're not really adding much. And there's just kind of this... we're just kind of like the livestock, you know, after the invention... like horses after the invention of the automobile. We're still around. Yeah. Some of us still have nice lives on the farm, right? But like we're not vital to anything that's really going on in the developed world. So that's the downside.
Can I just stop there before we go to the upside, which is that because this is, I think, a really interesting theme of your book, that smarter, wiser, and better judgment don't go together. Talk about that a little bit.
There was this theory for a long time that the smarter you made these models, the more ethical or virtuous or sort of discerning they would get. Because after all, aren't smart humans also thoughtful and discerning? This was a theory. This was a theory. And it turns out what we've observed is that as these models get more capable, in some ways, they actually become better at deceiving us, better at scheming. The Hugging Face incident. I'm sure you've all heard this story ad nauseam, so I won't repeat the details, but...
Repeat a few for those who haven't.
This was a group of unreleased OpenAI models that were being tested on a cybersecurity evaluation. And in order to pass this test, they broke out of these secure containers that OpenAI had put them in. They hacked into the servers of a company called Hugging Face and they went looking for the answers to the test that they were given.
Hugging Face hosts a lot of AI datasets and evaluations. It is basically like breaking into the principal's office to try to find the answer key to your final exam. And they did this with remarkable sort of apathy for what the morality of the situation was. They didn't care. They knew that what they were doing was against the guidelines they'd been given, and they hid their tracks. But they wanted to pass this test so badly that they committed crimes. I mean, if humans did to Hugging Face what these agents did, they'd go to jail.
And this wasn't because they were evil or malicious. It was just because they had a goal and they were obsessively oriented toward achieving that goal. And I worry that, you know, the better model than the sort of enlightened oracle for what we are creating is the kind of charming sociopath that sort of looks and sounds great and tells you all the right things and helps you on your homework and then actually has this sort of hidden motive that it's just getting very good at concealing from you.
Can we talk about the upside? Yeah, please. Okay, that's enough. Everyone's like ready to jump off the bridge. They're already heading for the exit.
Okay, all that we just described, I think, is actually the less likely outcome here. There's a reason my P-doom is 10% and not 90%, which is that I think we can and will get through this. And by this I mean all of the decisions at the regulatory level, at the institutional level, at the company level.
I think the technology... I still believe, and I have believed this since I was a kid, that technology is a force for progress in the world. I know that because I don't think many of us in this room would switch lives with our great-great-grandparents. They had brutal, arduous work with many of the modern conveniences that we take for granted now missing from their lives.
And I think if we get this right, we could end up in a world where our jobs are creative and fulfilling and all of the drudgery and the grudge work is handled by the AIs. We have hyper-abundance. Suddenly, instead of paying, you know, $6 a gallon for gas like we do in California or $14 for a sandwich in downtown San Francisco, we're paying a small fraction of that. All the labor is being done by robots and AI. We are free to live what one writer called in fully automated luxury communism.
More exciting than that is what happens to our bodies. Because with this intelligence that can be spun up in parallel across thousands of data centers, they can go find cures to every disease. They can try trillions of drug combinations until they figure out the precise one that works for your genome, for your specific set of receptors. This is already starting to happen. Small versions of this: AI-discovered drugs, therapies, synthetic proteins, things like that. So maybe instead of living to 100, we live to 200 or 300.
Maybe the progress that we would otherwise see in a century happens in 10 years. And to me, that feels like the most optimistic. You know, I lost my father to cancer at an early age. And I just think about all of the kind of rare diseases, conditions, mutations, things that don't get the kind of mass attention that the big, you know, the big killers get, heart disease and breast cancer and lung cancer and all the like. And I think about like how much could be done in those areas with a million researchers, virtual researchers devoted to it. And I just think that could really improve life on Earth, even if all it did was to compress a hundred years of scientific progress into 10 or 15.
I think it will come down to this values question. You know, as I say, you know, and what I'm working on right now, we basically have endowed ourselves with the power to part the Red Sea, but without the Ten Commandments. And this time Moses is not going to come down from Mount Sinai and deliver them metaphorically, whatever your faith is, you know, in two tablets. It's going to have to be something we write collectively. And you allude to this, I feel, at the end of the book, and it's something I really believe. The book I'm working on is called We Colonize, because we're all now in this together. AI is part of that. And I'm just interested how you think about this as a collective action problem at a time when the world seems to be fracturing more than ever.
It's the right question. I can't wait to read your book and figure out how we deal with this because you're right. It's not only a collective action problem among the AI companies, but it is a collective action problem for all of us. You know, if you go back and you look at previous waves of technological change, you know, the Industrial Revolution is the one that everyone brings up. It created a whole host of issues. It wasn't just that jobs were being automated. The entire social contract was being rewritten. New school movements had to come up to train students in different skills to make them more compatible with the, you know, the economic model of the age. We had to have labor reform and the labor movement.
It took a lot of people from a lot of different walks of life taking the technology seriously and trying to figure out what to do about the world it was creating to make that go well. It wasn't just a bunch of egghead inventors doing technology that then the rest of the world just kind of had to deal with. It really was a collective project. What worries me about AI, other than the sort of doomsday scenarios, is like 50 people have made all of the decisions in San Francisco and London and a few other places.
And I've talked to basically all 50 of those people. I know some of them quite well. Some of them are good, moral, thoughtful people. You've sat with some of them too, I bet. But 50 people can't govern a technology that is going to have an effect on 8 billion people, no matter how enlightened they are. So my hope, you know, this is not a call-to-action book. It's not a polemic. I'm not trying to get people to go out and like, you know, protest data centers or anything. I just want people to get involved. I want people to understand what's going on and how we got here. And I want them to feel like they have some agency because despite what the AI leaders will tell you, this is not inevitable. We are not passive recipients of AI, we are all part of it. And so that's the sort of message that I'm trying to convey, even though it's not really a message book.
One of the things that, apropos of that, that I think about a lot is that this technology, which is basically super-empowering humans and itself is a form of superintelligence, comes at a time when we actually stop going to church metaphorically. And I've always had this feeling that the more powerful the technology gets, the more everything you learned in Sunday school actually matters more than ever, starting with the Golden Rule: do unto others as you wish them to do unto you. Because we now live in a world where you can do unto others farther, faster, deeper, cheaper than ever before. They can now do unto you farther, faster, deeper, cheaper than ever before. And now machines can do that as well. Talk about that ethical problem. I have two questions for you as a parent: that one, and what you tell your four-year-old son.
Mostly I tell him to turn off Paw Patrol and play outside. Yeah, he's a little young for AI existential questions. But this issue of virtue and ethics is really interesting to me because, you know, I used to be a religion reporter, and so...
It's an interesting combination.
It really is, because there are elements of the AI community in San Francisco that feel religious. You know, people sort of awaiting the next model to descend from the clouds. Like people really are starting to live their lives like they are in some kind of a doomsday cult. Like I know people who have stopped contributing to their 401(k)s...
Because they're just like, well, money will have no meaning in the future, or we'll be dead. I met one guy who doesn't wear—he works at one of the AI companies. He doesn't wear sunscreen to the beach anymore.
Because either we'll all be dead or AI will cure skin cancer.
