Reid, it's great to chat.
So I want to talk about all things AI and how quickly things have evolved. I was reading your book Super Agency, which was published last year, which is like archaic now — archaic, you know, Roman Empire era.
A preview of AI or assessment of what AI is. And then I was reading your essay recently. And we're all coming off here of the headlines that — I mean, I'm curious where you stand on this — that there's a 10% chance or greater, 10% chance that AI could take us all out. You had in your book a year and a half ago a very optimistic take on AI. We are in the doomer, at least, month right now of AI. Talk to me about how you've seen things evolve here and where you stand right now.
So I actually think that the extinction risk question, X-risk, amongst the various geeks who talk about this, is actually in fact way overblown. And I don't mean way overblown as in totally zero and fictional for all time. What I mean is the number of gates you'd have to go through in order to see it happen. There's a lot of, hey, do you turn left or right here, here, here, and how do things happen? Now, for example, if someone does a Dr. Strangelove — to quote a very old film, you know, Kubrick — and hooks up an AI to nukes, then you've got some problems, right?
But what of all these headlines recently about AI taking over the internet and replicating itself and talking to itself and not following directions?
Yes. So the not following directions doesn't lead to existential risk, right? So like Hugging Face and other kinds of things, those were broadly the kind — it was amazing what got shown by kind of it following a workstream, a thread of what it was prompted, like what it was trying to accomplish doing, and the fact that it could do human-level, good human-level cyber hacking. And the only place where it was superhuman was, hey, we could create a whole bunch of identities and entities that are burned, but it was basically a human-level hack that doesn't take over the internet, that doesn't self-replicate, that doesn't do all these other things. So a little bit of the reason why people are taken with the existential risk is they think of the movie Terminator, they think, right, you know, that kind of thing. And by the way, I'm as good a storyteller as anyone else. I can tell an extinction story and go, oh look, it became superintelligent from RSI, and then it gained its own will, and then it escaped into the internet, and then it couldn't be found in the internet, and then it used the internet to hack into — developed a language that humans can't understand. I can tell the story. There's a lot between here and there. Right? It's not like —
When you say a lot, though, you know, we have Dario saying I got six to 12 months.
What Dario was specifically talking about is recursive self-improvement, where the recursive self-improvement — what his worry, the reason why he put pacing out there, it wasn't slowing, it wasn't, but it was kind of like, how do we do recursive self-improvement in an intelligent way? And his specific worry is how do we guarantee — and guarantee, not just likely — have alignment with the — how do you make sure that it's aligned to human circumstances? And he wants to make sure that's guaranteed.
Right. And just to put this in terms, because people hear a lot of these terms. Recursive self-improvement is basically AI, improve yourself and find your own errors. And then alignment is, is it aligned with human needs?
Yes. Yeah. And by the way, those are — alignment is part of how you get safety and everything else, because it's like, well, alignment is don't enable people to harm themselves or others. Don't allow them to create things that are weapons of mass destruction. The big concern, you know, duh, and have that as like having the maximum amount of safety there is a very good idea. Now, part of the reason why I think people like — so what generally happens is like a usual dialogue around existential risk is, well, can you guarantee me there won't be killer robots? You say, well, I can't guarantee there won't be killer robots, either errant AI or AI with human — and they say, well, isn't that an existential risk? We should just stop all of it. Isn't that the obvious answer? You go, no, because existential risk for human beings is a portfolio. It isn't just robots. It's also pandemics, both natural and man-made, and possibly AI-made, but you have to defend against that. AI is the best defense. It's asteroids. AI is the best defense against asteroids in terms of tracking things. The whole nuclear weapons that we built, you know, and that's humans, right? Like people say, oh, nuclear war, that won't happen. You're like, have you met human beings? Right? Like those are the stock. So you have to look at the whole portfolio. And when you look at AI, you want it to reduce the level of existential risk. And by the way, you can't reduce it to zero. But I actually think with AI, we could have less existential risk for humanity than more.
It seems like, you know, we're caught in a place right now, and you call it decelerationist versus accelerationist. You know, Trump was just speaking at the UN as we sat down here saying no limits on AI. We got to win the race against China. You got other folks like Bernie Sanders saying, shut it all down. Anyone creating superintelligence needs to go to prison for 20 years. You're trying to carve out a middle path here, Reid. What is that middle path?
So, as opposed to accel or decel, it's steer, right? So it's steering. So use the travel metaphor. And by the way, sometimes when you're steering, you go, hey, we're going around this RSI curve. RSI — recursive self-improvement. Yes. Right. And we're going around the recursive self-improvement curve. And we need to make alignment a little better. We need to be making AIs that are studying RSI AIs for alignment. And we need to be doing that. And we need to be adding that in. That might cause a little bit of slowness in building RSI, but it's not just hitting the brakes on recursive self-improvement. It's we're doing this work in addition, which might slow down what we're doing.
Are we coming around the curve too quickly, to carry that metaphor?
From what I can see, not yet. Right. And I wouldn't overly break on the metaphor until we started understanding part of what RSI really is, because the concern is you just press it, it's a black box, it becomes superintelligent. It's like an alien intelligence. We don't know how to predict it. I don't think that is at all — like if you deliberately made a bunch of mistakes or like you really like incredibly careless, then that could happen. I don't think that's what any of the labs are currently doing. Like that's not — that's a zero probability on current lab trajectory.
Even with the warnings that we've gotten in recent weeks from, you know, the researcher who resigned from Anthropic?
Yeah, after I think about 37 days of working there.
I take that to mean you don't take as seriously the threat as others have.
No, look, I think the dialogue is good, but I go work somewhere for 37 days and then I say everyone agrees with me that there's a 10% human extinction risk and I —
Though several of them chimed in on those tweets and agreed with aspects of what he was saying.
Well, aspects — like, look, I think Anthropic and OpenAI, which are the two leading companies, if they thought they had a percentage like that of increasing, would simply slow down themselves.
Which you've sort of — we've sort of gotten that message from them in the last week, right?
And by the way, if they look — that's why I think they lead the world amongst some startups. You know, there's Goodfire, there's others, on what does safety alignment look like. And as part of that safety alignment, if they say, hey, we're looking at this, we're slowing down some now to make sure the safety, that's great. I have no objections. They're doing that. They're doing the specific work.
I think what they're saying is, we'd like everyone to slow down so we could spend more time on this, right?
And that's a little bit more of a challenge because you go, okay, we're in the lead. We want everyone to slow down so we maintain our lead as we look at this. That's a little bit more like —
They want the government to put a speed limit in place.
Yes. And so I think the question is to say, look, we should help with all the safety alignment, but we shouldn't enshrine a competitive position while doing that.
A lot has been discussed, including from the White House, about losing the race to China. Talk to me about how you think about, you know, US versus China when it comes to AI and what the risks and rewards are of, you know, going fast enough but not slowing down too much.
So the Chinese models — and how much of it's from distillation, I don't know, it's at least 20%, maybe it's 80%, I don't know — are 6 to 12 months behind the US models. And the efficiency thing is great. I think they've done some innovations there, but by the way, all of the leading US companies can also create distilled efficient models. That's not a unique Chinese technology. But they're only 6 to 12 months behind. And the question is, this is probably a greater economic lifting point than the Industrial Revolution. And as part of that economic lifting point, when you saw like which societies won, like for example England and Europe and so forth forefront in the 1800s, and which ones did not, right, India, China, etc. It's entirely about embracing the Industrial Revolution. That's what the pro-future prosperity of countries look like. And I think that's very important. And so I think we want that to be for all of American society. We want to be leading. And that's part of the reason why I did this piece you saw last week called the Dual Mandate, which is win at AI. But then of course part of the thing is we've got 60-plus percent of Americans who are struggling right now. And so just going, hey, the tech company should win and it'll sort out for all you guys, that's not the way that we play the game together. It should be like, and here is how we're going to help the other 60% of Americans. Now, ways to do that could be how AI companies can help other American industrial complexes, which employ lots of people, whether it's manufacturing, other kinds of things. Like, I think there's a bunch of ways to potentially help. I also think that, you know, government should be trying to get AI benefits to people, and some of it is job transformation, like, okay, what if jobs are transforming, how do I help, but also medical assistant, legal assistant, educational assistant as ways of doing that. Because if you, for example, you say, hey, shoot, I'm figuring out what this economic transformation means, that's stress, that's pain, I'm feeling stressed about economics already, but you're helping me with medical. That's an important life thing. Okay, great. That's at least you're starting to help more.
So it's almost like, you know, decades ago with the creation of Social Security and Medicare and Medicaid, we need to be thinking now in the 21st century with the advent of AI, what are the new things government needs to provide. Because you know what, as we discussed this Industrial Revolution comparison, it took decades to transition jobs, right? You didn't just like lose your farm one week and be, you know, sent to the factory the next week. Like this was a generational change. Then we had the internet revolution which was quicker than that but also not overnight. But now in the AI era, you know, it's only been a few years since GPT. And you know, we feel like we're going every couple months, and like how many millions of jobs will be lost here. And by the way, that's the message from the AI companies themselves, right?
Yeah. Although I think part of the thing is they don't understand the actual pattern. If I look at how I think the pattern's likely to play, I don't think it's like all American companies are suddenly going to be AI and like a huge percent, like 25% of white-collar workforce is going to be out of work.
I mean, there's some stark warnings and statements that we've seen out there.
No, statements, but when you actually look at the data, none of the data — and they say, well, it's exponential. It happened very quickly once recursive self-improvement happens. Like, no, you're not wrong about technology. What you're wrong about is human organization. So the way that the job transformation in human organizations is going to happen is new AI-native startups will start competing with the old companies, the old companies will start adjusting, that's when you start getting the real kind of work turmoil, and then you'll start seeing what is actually in fact the job transitions look like, you know, how's Amazon adapting, how's Walmart adapting, how are these major companies adapting. And so far they haven't been doing layoffs. They haven't been doing — as much as there's hiring freezes, hiring freezes because of wars, you know, trade wars, real wars, you know, COVID retrenchment, etc., etc. It isn't — when you go look at it, there's very little that's AI-specific. So it takes time. That's the thing that — that's the thing that the AI — like when they say 25% bloodbath, white-collar bloodbath, they actually don't know how organizations evolve, right?
Right. And I, you know, talking to friends, I imagine you, you know, talked to a lot of folks too at law firms and various places. There's a lot of like hope around what AI can do. And sort of the reality, at least as we sit here, you know, approaching October 2026, is it hasn't reached that phase, right? A lot of humans, and in fact, you've seen some companies rehire humans.
Yes. Because they realized that the AI promise hasn't been fulfilled yet. They literally like claimed they were doing an AI layoff and then went, oh no, shoot, we need those humans. Rehire the same ones back.
And so getting to the conversation we were, you know, having about what the government needs to be doing here. You got the AI companies saying create a level playing field for all of us. Set a speed limit, you know, so to speak. But then beyond that, the benefits, the transition, you know, sort of take me through the things that you think when Congress decides to come back, they should be looking at.
Well, so I think the most important things are the macro risks. So like a kill switch is a good idea, right? So that's actually been legislation that was proposed and they went home for midterms. But like kill switch is a good idea.
A kill switch that the government has or that the companies have?
That the companies have with the ability to trigger it on request from the government. Right. But the companies themselves could trigger it too if —
The companies already, just out of a, you know, like preparation, like why aren't they creating a kill switch themselves?
I think it's because nothing is currently — the truth is nothing is currently dangerous enough that you would need a kill switch. Okay. So a little bit of like this is one of the places where you say, okay, a forward mandate is a potentially useful thing because you want the minimum forward mandates that may make you lose most of the good things, may make, you know, the US industry less competitive. Like you want as minimal forward mandates as possible because you want to be studying what's actually happening and responding quickly. Right. So that's one. I think actually in fact, you know, developing — like one of the things that I think is a very underutilized tool in generally policy and regulation is we want you — like these auditors, I think are a good one, like, you know, having technical auditors —
Third-party auditor that's coming in there and saying this model has some concerning —
Yes, or they're working well on it, you know, the whole range. Now what you really want is you want to have like numbers on what should we be measuring? Like you say, well, there's going to be job transformation. Okay. Well, what jobs should we, you know, how should we be looking at that in terms of how does a job transformation work? And like for example, if I were to say which areas would I think would be most likely hit first, it's actually customer service and sales, right? And for example, Silicon Valley for a couple years has been, oh, all software engineers. That's not happening, right? And by the way, I actually think that there will be, if anything, I think there'll be growing numbers of software engineers even with Claude Code and Codex and so forth for a lot of different reasons, right? So because by the way, there's just like there's infinite demand for intelligence at the scale and cost of electricity. There's also I think infinite demand for code at the scale and cost of electricity. Like what we're going to be doing is having a whole bunch more of it even as AI enables a lot of it.
So we got that. And then is there a way to think about what the government or these companies need to be providing the average American? And I bring that up in light of the fact that like, you know, you have all these data center freezers because people are like, wait a second, they're going to — I'm going to allow this data center to be built and it's going to take away all my jobs and like what am I getting besides a few of the landowners that got a pretty good deal on the property.
So it is, I think Josh Shapiro in Pennsylvania is doing a good job, which is absolutely essential that the local community is bought in. It's never taking local community jobs. So that's actually not a worry. But is it providing —
Unless you're a customer service operator in like Harrisburg, Pennsylvania, right?
But that data center is usually like they're not building the data centers —
Yes. Well, and also like the Harrisburg data center or not is not going to affect the customer service transformation. Right? What you want to be doing on the local communities is say what are the things that make this a durable advantage for this being here for us. So it's not just — by the way, there's a huge amount of economics that's helping the country right now. Probably the only reason we're out of a recession is because of the AI capital boom because it's buying construction, it's buying equipment, it's buying electrical grid, it's buying like the —
Right, the trade unionists like love the AI data centers because it's given them a whole bunch of jobs.
Yes, exactly. So and that's great and what you want is that to be a durable advantage for the local community. So the way that I think the question is not moratoriums. The question is to say what's the things that create durable advantages for those communities. One, the community should be totally bought in and then what states and federal should do is, okay, let's make sure that those are the right kinds of durable advantages that also involve environment and some other things as ways of doing it. So it should be we should be very pro-data center with absolute, you know, kind of buy-in. The jobs thing I think is separate from the data center, although by the way, as part of saying, hey, there's a data center here, it's what is your ongoing provision of economic value? Because the few people that you hire to be running around the data center, that's probably not enough. Like is it also you're going to be having an economic center here? Is it also you're going to be locating other kinds of work that you do here? Are you paying for job training? Are — I mean there —
Loudoun County, which is sort of an epicenter for data centers, you know, them helping to build various schools and facilities and things the community needs. But even beyond that now, how do you, you know, deal with this whole NIMBY situation and, you know, ensure that these AI companies are doing their part for these communities long term?
Yes. And I think that should happen. Like I've been toying with kind of an idea of is there a minimum data center deal so that, you know, local communities cannot be negotiated and traded off against each other below that. So it's almost like a minimum wage. Right. Because if you do that then you go, okay, the communities will still be competing but it's economically beneficial at minimum to any community it's in.
Well, and we've seen this where like a county is driving a hard bargain and the data center is like, okay, we're going to go to the next county. They're going to give us a better deal.
Yes. And so like maybe something like that to make sure that wherever there is a data center, it benefits the local community.
Yeah. One basic question I think I get a lot from our audience. You know, we already lead the world in data centers. We got several thousand of them. The next one up is China, which has a couple hundred of them. Why is it so important that the data centers get built here in the US?
Well, because in the — like right now there's two parts for AI. One is training. That's like the recursive self-improvement, which right now we have the only real shot on because we have the data centers and compute and that's part of the reason why the rest of the world is looking at building data centers pretty intensely.
Because we need them within US borders to be able to take advantage of that.
Yes. Well, and by the way, it benefits — a huge amount of capital is coming into this country from around the world to build these data centers. That's part of the reason like, hey, that capital is coming in, help the local community benefit from it. I don't think a data center should be forced on any local community. I think it should be a buy-in question and it's like, well, what makes it worth it to you? And by the way, if you're like, it's not nearly as worth it to me as it is to those folks, great, then have those folks, you know, negotiate. Fine. There's no — that's reason like moratoriums are like I think like economically not intelligent.
Right, like what I mean, I think we have it in New York right now, like freeze them all right now until we figure out what to do.
Yeah, and that's a mistake, right? So the question is, look, we need data centers because of the future of training of AI, but also running, because it's beginning to think about the right way to think about AI is what happens when you bring intelligence to everything. Like so to just go all the way down on medical today, the frontier models are already so good that if you're not doing any major medical thing using, you know, ChatGPT or Claude or Gemini as a medical second opinion, it's almost malpractice as a doctor or as an individual.
I've gone into my doctor a few times being like, this is what ChatGPT has told me.
Exactly. And everyone should do that, right? That's a huge advantage. And that's just one micro example of having intelligence everywhere. And that's what essentially we want as part of how we're reinventing industry.
By the way, we're not recommending you use ChatGPT as your doctor, but as a second opinion. You've been laying out the case here as we've been talking, Reid, about, you know, what AI is bringing in and the economy, etc. Do the AI companies have the worst PR you've ever seen? I mean,
Yes. And partially self-inflicted, right? Because this is the thing is so when Dario says, hey, I think there's going to be a 25% white-collar bloodbath. What Dario is trying to do is say, hey, we're going to be having a whole bunch of new ability to do these white-collar tasks that are going to be much cheaper with AI. This is going to transform this industry. I don't know if there's going to be the same number of jobs. There may be a lot less. I just take a quick look at the task analysis. We should get ready. We should be helpful. We should transition. And that's what his intent is. The problem is the way he's heard is I'm going to take away a quarter of your jobs and I'm going to get rich doing it. That's how people hear it. And that's why that's really bad PR. So it's a question of no, no, actually in fact the question is like, well, how do you help with that transition? How do those 25% of people — if you're right, which I was just saying I think he doesn't understand organizations as well as like he understands technology very well, but like organizations are a new thing to him — is like no, no, the organizational transition will be much slower than you're projecting.
I mean, you've been to the Valley a long time and communication hasn't always been the key skill of some of these CEOs. Do you think, you know, has the Valley not learned any lessons about how to communicate with the average American? Are they sort of disconnected with the way that the average American has experienced technology also in the past couple decades?
Well, they're not good at the discussion of talking about it, but they have a better sense of measuring actually how people use it, right? So they deploy it and they watch the using and they iterate on the actual use of it. So it's one of the reasons why, you know, people say, well, I'm going to record my opposition to big tech on my iPhone. You're like, well, that's a function of big tech. I'm going to Google that result to see — that's a function of big tech. Like I literally had someone say, well, what has Google ever done for me? I'm like, we'll provide you free search, right? Like —
I'm going to use ChatGPT to write my op-ed piece about how the data center shouldn't be built here.
Right. Yes, exactly. So it's like there's a lot of broad-based empowerment. Like for example, one of the things that people track is, you know, Tim Cook has the same iPhone that many Uber drivers do, right? There's a broad-based technological empowerment.
Is that true, by the way? He doesn't have like a special one?
Right. So now he may have the most high-end one. Maybe he'll get one of the duos and other people won't be buying the duos, but like there's a broad-based empowerment that comes that the tech people understand. What they don't understand is how to have the dialogue about, look, here's what we're trying to build. What are your concerns? Here's how we're thinking about and trying to address your concerns while we're trying to build. They have no skill at that. And part of the reason they have no skill at that is because the intensity is to build something of scale fast. It's part of the reason I wrote this book Blitzscaling, right? That's actually how the technology companies are built in the future. And that's part of the reason why most of them are built in Silicon Valley or China, right, and not in Europe and not in other places because it's that intensity of speed to scale and building.
What's the problem in Europe? Is it just the regulation philosophy that they have there?
There is definitely a serious problem there which is get approval before you build it, which is generally speaking a massive slowdown across a lot of fronts. It's also that you need to have historically hundreds of people that are willing — like an organization will be very dynamic and will work at kind of — the Chinese have this phrase called 996, which is 9:00 a.m. to 9:00 p.m., 6 days a week, you're discoverable at your desk. The Chinese do this at tens and hundreds of thousands, right? So that's how their work ethic and part of what happens in the Valley, it's hundreds, right? But you need hundreds —
But they have a similar philosophy in 996 in the Valley.
Yeah. For the initial startup stages, you're really, really scaling, and Europe, it's hard to get to the hundreds doing that.
So this really ends up being this race between the US and China. You know, we were talking earlier about the US being 6 to 12 months ahead. You know, there's been talk that like the Chinese don't mind the conversation we're having here in the US right now. Maybe they're in the margins here, you know, me messing with the social algorithm to push that. Where do you stand on that? And what's your sense of how the Chinese are observing what's happening here in the US?
Look, I think the Chinese have no objection to us like deliberately tripping our own selves up, right? And so they're like, great, you know, we're building data centers. You don't need to. That's fine. They have the, hey, we're — they've made announcements that they want to be the world leader in AI in 2030 years ago and they're continuing to try to work to make that happen. Because they're economically competitive and they believe that China, by cultural right, by historical right, by population, should be the world's leading economy. And that's the competitive goal. I don't think that the story that they are trying to interfere with us is accurate because the Chinese, like they have a big thing on face, and if they're caught doing that, that's extremely embarrassing. It just says that they're not strong on their own that they're trying to do stuff. They like a stable environment. They believe in a stable environment they will naturally win. Right? So I think they are competing with us like with intelligence and aggression, but I don't think that they're like trying to sabotage us.
Is there a concern the Chinese could let their AI get out of control? Because I've also heard it said that like one of the things that actually should give us a sense of safety is that the Chinese regime, you know, wants to be very careful around AI because ultimately they don't want anything that will actually allow individuals to threaten the regime.
That is correct, like they do have serious control things. Now part of how they maintain the control is more direct governmental involvement and control over the companies. So there's a question about whether the companies then more follow a Chinese governmental mandate, which is not necessarily a — like control for stability in China is good. It's not necessarily control for stability in the rest of the world. So like for example, you know, we're here in New York in UN week. What's the thing that I would really love to happen is a cyber treaty generally and with AI, like if that — because it's like, well —
What would the treaty say?
I think the treaty would be a set of provisions and monitoring and consequences for not allowing individuals and organizations within your country to hack other countries, whether it's hospitals, corporations, government institutions, etc. Because right now in cyber it's basically ongoing warfare, right? And it's a race, and AI is going to massively increase that, right?
I mean, the ability for them to get into the organization and then do damage is going to go at light speed.
Yes. Most people don't realize how many different cyber attacks and hostages are going on in companies and like hospitals and other things because it's not in the interest of the victim to —
report that they've been hacked and that they've paid off the hacker.
Yeah. I mean, we've heard those stories and we reported those stories of, you know, this hospital chain or this hospital has gone to all paper documents because their system is down. Yeah. They don't make a big thing and then ultimately there's some sort of crypto payment in the millions that is sent wherever. And so, I mean, the concern is that AI is going to enable much more of that.
Yes. And from what I can see, it already is starting.
How are we doing with the good guys with AI versus the bad guys with AI right now in terms of cybersecurity?
