Welcome to our DealBook panel today: Artificial Intelligence and the A.G.I. Horizon. I want to briefly introduce everyone at this table. To my right, Max Hodak is co-founder and CEO of Science Corporation. You're also co-founder of Neuralink. You are literally doing the work of bringing sight to those who don't have it. Am I getting that more or less correct? That is godlike and extraordinary. Fun fact about you: you have described your ultimate life goal as disappearing into the matrix.
Dan Hendrycks, executive director at the Center for A.I. Safety, and I believe until literally hours ago, a safety adviser to several A.I. companies. We'll return to why that relationship changed during our conversation. Dan, you've created an alphabet soup of A.I. benchmarks and functions, including the Gaussian error linear unit, which I hope to return to never in this conversation. [laughter]
On my left, Lila Ibrahim. You are the chief A.I. readiness officer at Google DeepMind. That job didn't exist, I believe, until you took it. Pretty remarkable obligation you have to literally make government, society, schools, industry ready for what A.I. will bring. You have said, I think quite memorably, that while A.I. is revolutionizing the world and your world, quote, "it cannot automate my Tuesdays."
And we have Mati Staniszewski, co-founder and CEO of ElevenLabs, which specializes in generative audio, something I care a lot about, could be in my future, and the future of the show I host, The Daily. Something I love about you is that you have been working with the estate of Albert Einstein to clone his voice for an A.I. agent that we would all, I'm sure, appreciate hearing from. And if we are lucky and the traffic cooperates, we are going to have Clem Delangue, Mati, to your left, co-founder and CEO of Hugging Face. The fun fact about Clem is that as a teenager, he was so successful at selling bikes and ATVs on eBay through his dad's business that the company offered him an internship.
So, thank you all for being here. I want to set the stage for this conversation. A couple of things have happened in the last few weeks that I think feel like touchstones for what we're going to be talking about. The first was a midterm election in which we learned that American voters don't really like where A.I. is headed and have registered their disapproval with the current hands-off approach of the Trump administration. Voters passed dozens of moratoriums on new data centers. I think we're going to end up talking about that. The second major event was what happened over at Nvidia. Its CEO, Jensen Huang, was forced to resign over multiple instances of misconduct.
And that has left the company pretty much leaderless at the moment, a company at the center of the A.I. boom, and has opened up a fascinating and complicated debate about whether he should come back and run the company. Finally, we have Anthropic going public in a few days. I think by the time this is released to the world, Anthropic will have gone public, and that is a moment that is going to literally put a price on the core technology of the A.I. boom.
So with all that in the background, please know that these subjects are just touchstones and will give us lots of room to roam. And let me start with the election. And Dan, how much has changed pre and post the midterms for A.I.?
Well, now because the midterm showed that the public has strong feelings about what to do about A.I., its salience has increased a lot, and so politicians have more breathing room to actually try and regulate the technology. The things that were thought outside of the Overton window are things that we can speak about freely in D.C., such as, for instance, pausing A.I. development. So I think, fortunately, this is making representatives represent their constituents more, and hopefully we can use this opportunity to actually rein in the industry and make them internalize their externalities and be on top of these issues compared to the past few years where basically nothing got done.
What was outside the realm of possibility when it comes to government's role in A.I. two weeks ago, that's now quite possible? Just be a little bit specific, and why would it matter to the voters who registered so much alarm?
Yeah, so I mean, I think, for instance, some basic things like liability. This was something that we tried having years ago: basic liability for mass casualty events. This is now more of a live possibility. Likewise, even just helping companies slow down if they're thinking, "It's too reckless, our model's not very aligned, so it's going to be difficult to release it, but we're pressured to because the other companies are releasing it." There's more of a possibility for the government to be helping here, too. And I think Congress in particular, though, is unbottlenecked as they were before in actually passing these laws. So I'm much more optimistic about the state of things. I think earlier for the past few years, nothing really got done. There's a huge backlog of things that we needed to do, some basic measures. So now I think 2027 will be a very interesting year for regulation and also for A.I. capabilities. So hopefully they keep up with each other more.
Max, how comfortable were you before voters went to the polls and said, with a certain amount of, you know, consensus, that they were worried about all this, literally registered that in public polling and the election? How unsettled were you by the A.I. universe and hands-off approach that existed before this moment, or did it work?
I mean, A.I. is kind of the ultimate dual-use technology, like all technology. Technology is usually not inherently good or bad, although it can create adaptive pressure gradients that kind of force people into situations that maybe we would have designed from scratch differently. But I mean, I think that many people don't feel like technology has improved their life over the last 20 years. I think that back in like 2000 there was this mass optimism about it, and we initially had social media that brought people closer together, and then we saw that it was like eroding our attention spans and causing people to be isolated, and then doom scrolling and attentional, like, algorithmic feeds were clearly not an unalloyed good for many people. And so I think there's just been this history now of, yes, our lives are better in many ways, but I don't think that many people feel this when they interact with technology.
And so I think that it's really a myth that data, almost any of the major hyperscalers' data centers use a lot of water, or I think my impression is that the main A.I. companies have ratepayer guarantees. So a lot of the specific things that people are latching on to, I think you can have issues with, but the meta observation that they're worried about how this advancing technology is going to affect their life, there's clearly something to that. And we have to heed that, like we have to paint a picture of a future that people can see themselves in. And part of the problem right now is that I don't think that anybody working in A.I. can really see the future. Like, growing up, I prided myself in being able to see kind of where we were going. I could always imagine this future that we were pushing towards, and now I have this event horizon within it. It was initially like the mid-2030s, then it was down to like 2030, and now it could be the next few years that I just can't see past anymore. And that doesn't mean it'll be bad, but I think it means that there's a lot of uncertainty and a lot of apprehension about where this is going to take us. And I think it would be a mistake to just stop, which I think is the feeling that many people are beginning to push for. I think that there really is a huge possibility of getting real advancements in healthcare. Like, that's what we do. We're a medical technology company. I think that it really is translating to real advancements, and there is a real possibility that within the next decade, like, yes, you worry about bioterrorism, you worry about kind of the school shooter with the DNA printer, but it's also possible we could just have eliminated all infectious disease, and that is like the best protection against these things. We could have turned cancer into a chronic disease for many, many people. I think these things are really possible. But you then juxtapose this against the possible rise of true ASI, and that again is a thing we just don't know how to reason about. So, um...
Including someone at this table.
I think we have met my definition of A.G.I. at this point. Like, Mythos, Fable, Astra, Sora — I mean, these are, they're able to do basically any economic task to varying degrees of, like, you need to supervise. They're basically as good as the user is, in my experience. If the user is a really capable programmer, they're going to be able to build incredible stuff very quickly. If the user is a researcher, they're going to be able to understand a lot of the field much more deeply than they could otherwise. But then going from that to something that is just superhuman in judgment and understanding of every field, and has perfect attention, can run thousands of times faster than humans can think — it's just, it's unclear what that would mean for the world. It could just be... I worry a lot about misuse by people. And in fact, I think probably the most impactful reform that I'm personally pushing for is, I think that a lot of the concern that we have around A.I. is actually addressable through antitrust law that we already have. Like, the policy that I would advocate for is that intelligence providers should be able to develop frontier models, but they should be limited to being intelligence providers. Like, I think the concern is that the frontier labs will realize that selling tokens is probably the least valuable thing they can sell, and they're going to end up... and so, like, what do you move to? You move to selling drugs and cars and products, and then the American...
And you'd like to see that limited.
Well, I think the American economy, the existential risk aside, is going to have this choice between do you get 20 trillion over three companies or do you get 200 trillion over 500 companies? And I think right now we see a lot of forces for centralization. I think that is also what a lot of the public is responding to, what they're afraid about. And so I think this is, in a way, really the exact inverse of nuclear weapons, where in nuclear weapons the safety was from lack of proliferation, when we enforced a non-proliferation regime quite seriously. And I think it is the opposite in A.I. I think that it's a discomforting disequilibrium that we're going through, but at the end of that, the only, the really bright futures, all of them that I can see, involve mass proliferation of the capability in a way that just pushes out the competitive frontier but establishes a new one. I think all the futures that I'm most concerned about are ones where there's a singleton or there's this mass concentration of intelligence capabilities in a small number of concentrated actors. That, I think, is the more worrying near-term possibility to me.
I'm going to tease you, but speaking of antitrust, what do you think about... I'm making a Google joke.
Thank you for clarifying that. [laughter]
Respond to that. You spend a lot of time thinking about, as we described in your title, the readiness of specifically the American public for this moment. I think the election reflects the fact that Americans didn't feel ready. And just to reset, because I think it's worth reminding us what this election revealed, let me just read you from a story my colleagues wrote. The headline was, "Voters rejected A.I. in the midterms. Now comes an era of scrutiny." And what this story said was that on election day, voters slammed the brakes on data centers and swept in governors and legislators who have promised to protect individuals from threats posed by artificial intelligence.
And the public polling on this that preceded this election is really fascinating. 52% of U.S. adults say they're more concerned than excited about the growing use of A.I., compared to only 9%. This is Pew, by the way, so not a partisan group, not an agenda-driven group. Only 9% are more excited than concerned about A.I. 67% of Americans believe A.I. advances are happening too fast, and 67% lack confidence in the U.S. government to properly regulate A.I. So how are you thinking about this?
There's a lot to unpack in this. Well, first of all, I do believe, much like my colleague said, that A.I. is probably one of the most transformational technologies of our time. So we have to find a way to regulate it, but do it in a way where we're regulating it well. And a blanket coverage for A.I. is not going to be the right approach, because then we need to figure out how do you manage the risk while also allowing for the opportunities. One of the things that I think is really important to keep at the center of this conversation, too, is it's not any one company or any one country. Like, the way I think about this relative to my time in the early computing industry was, you know, we had time to build things out, whereas A.I., a model is released and it's global, so we need to be taking a much more global perspective on this. So that is one thing that I think is really important, and it's part of the reason why, you know, Demis, our founder and chairman, started writing much more publicly about this even earlier this summer, and it's an area that we've been investing in since DeepMind's start 16 years ago.
Now, with the technology in general, you think about where global trust is at. I think in my 35 years in tech, I've never seen it the way it is right now. And at such a low globally.
Yeah, in some countries more than others. You know, I've just come back from Brazil and India where the sentiment is quite different. But part of that, too, is like seeing where A.I. does have this potential to bring opportunity of addressing diseases or helping farmers. We're working with 600,000 farmers in India on how to get more real-time information and insights into their crops. And we meet with them every other week on how to advance this, and it's an open platform. So I think...
That's an opportunity for a very different kind of trust.
And this is why I moved into this role earlier this year, of having spent the time building the organization and the technology.
You were chief operating officer.
I was chief operating officer at DeepMind, of saying, where I'm in this leadership role in this moment in time. I have diligenced everything I could the past in building, helping to build the organization and the technology. Now I think it's so critical that we think about how do we build this with and for communities. And A.I. isn't something that just happens to people. So, you know, even if you think about the scientific field, right, you can release models and tools, A.I.-supported tools for scientists, but just because they're a scientist doesn't mean they know how to use something like the AlphaFold database, right, or the AlphaFold tools.
Just explain what that is.
So, AlphaFold was our Nobel Prize-winning technology. It's an advanced A.I. system that predicts the 3D structure of a protein. What a protein is, because I had to go back to my old science books, is basically if you can predict how a protein folds, it tells you the function.
Right, and you can vaccinate against whatever disease it's on top of.
If you understand diseases, then you can come up with better therapeutics, right?
But it's not just for human diseases. It's also things like why are some crops more disease-resistant than others, or how do we deal with industrial waste. So there's a lot of applications of doing this. So the 250 million known proteins to humankind, we made it openly available to scientists worldwide who are now building upon that to advance their understanding of diseases and...
Right, that sounds almost of the scale of the Human Genome Project in its kind of revelatory power to the world of healthcare.
And I think it's the point that was raised, which is how do you manage this in a way where you are openly talking about the risk, which I think is so important, and addressing that, while also thinking about how do we make sure we're not limiting what the positive opportunities are. And I think it's probably a tension many of us feel, which is how do we hold both of those at the same time, right? And how it all goes back to, like, we can't do this on our own. All of this has to be done in collaboration within our organizations, across organizations. How do we think about the public sector, and how do we think about it globally?
I wonder if, to some degree, the healthcare A.I. advancements, because healthcare takes longer, haven't really had the opportunity to tell their story in the same way that the A.I. as threat to civilization, A.I. as threat to your job, has so clearly been able to tell a story. And is that just a public relations reality of where those two worlds are, or what?
Maybe I can quickly jump in. I think, you know, the advanced chatbots are how most people are interacting with A.I., and in many ways I think it surprised a lot of us how quickly this took off. Whereas, to your point, I think the scientific endeavor is a much longer process and much more systematic in how things are released. What we are seeing, though, is we are starting to be able to get an insight into shrinking that scientific discovery down, but it still takes time, right? The scientific method in general — I'll defer to our scientists at the table — but I do think that that is part of it. And even things like how do you make certain crops more drought-resistant.
Right, so 10 years it takes for crop breeders, and we're trying to shrink that down. We just announced a partnership in trying to do that, requiring gene mapping. But a lot of this is just, it will take time. So I think part of it is like the time horizons are quite different. And I also think the way that it's captured the public imagination through the advanced chatbots is, you know, in the news cycles, is something different. But I do think it's so important that we're having, like, these open conversations. I don't know what if... sorry.
Yeah, well, either one of you. Well, go ahead. I would like to hear how you're thinking about, Mati, the kind of pre- and post-election landscape. And I put that same question to you just so viewers, listeners can understand where everybody rests at this table, where you all sit, what your level of discomfort, alarm was about the world as it existed before the public, especially here in the United States, offered what felt like a real kind of hit-the-brakes message in this election.
Yeah, from my perspective, from our perspective, it showed a clear gap in terms of there's a lot of development, a lot of, as my colleague said, a lot of intelligence progress, but it's not showing up in daily lives. And I think the elections highlighted how big that gap was. And I think that's the big piece of what we hope to solve, and what I think we can solve as the industry, of how you bring all of the technology to the daily life of every person. It's slightly harder in this specific technological transition is that the development moves at such a high pace that diffusing that in the economy is required much quicker than in the past for people to be able to see that benefit. And I think that's the big part of how we can get it to every part of the industry, whether it's the healthcare example we spoke about, the education, whether it's making the daily lives more delightful, will be a big part of the equation.
Now, of course, we started the company in London. I'm Polish, so maybe I will add a European angle and perspective there too, which is, of course, I think the underlying worry as we think about the whole development is how do we make sure that this technology proliferates to the rest of the world too, and what's the alliance of the future? Because it's not only some of the examples you spoke about, it's of course cybersecurity, it's defense, it's one of the critical infrastructures of the future. And I think from that perspective too, and outside of the elections, that's the permeating piece of how we can solve that part as well.
But look, on the healthcare example, I think there's bigger questions. I'm an optimist, and I'm really hoping that we'll see some of the disease-curing come in the next few years. But we already see it in simpler ways show up in daily lives here in the U.S. Just last year, we worked with a congresswoman, Jennifer Wexton, who lost her voice and still wanted to inspire others, still wanted to work, and we brought that voice back, working with her to finally be able to have and communicate with her identity. And that's not an individual story. We had...
Although a very strategically wise one, because you're literally bringing improvement to a member of Congress to see how A.I. can improve their life.
That one was, you know, this one, of course, was more public than many others, but we now have worked with 15,000 individuals in that exact same situation. And each of those specific ones, when they were possible and people could communicate with their loved ones, with their families, changed the way they could operate. I think my favorite example was when the person was getting married. She lost her voice before she could get married. She got it back thanks to technology, and then they could recreate the marriage. You could see all the loved ones hear her vows for the very first time, and that changed their life and perspective for what that moment meant.
I feel like someone watching this is going to kill me if I don't ask how you bring someone's voice back at that level, for a wedding, for work in Congress. Are you retrieving old archival audio and somehow then generating it into a kind of current voice?
That's one approach. If you have recorded past recordings, you can use that. It's, you know, depending how much you have, the quality will be very different, but exactly right. But of course, as you lose voice due to ALS or MND, then usually you go through the process of voice capture to make sure that just before you get that voice, too, you got it, and then you can recreate it. Of course, with the modern technology, you can, you know, a minute of voice gets you a proxy of what it could sound like. And if you have an hour, it will sound great. But that's the big part. Of course, all of our voices are different. It will carry different nuance, accent, dialect, style, and depending on where we sign. So all of that is what you can now do.
But across the healthcare example, I think we're seeing increasing amounts of helping people communicate in a new way. A lot of the work Max and team were doing in the past, where we were helping in that ALS domain too. So I think there are pockets of the promise that we can point to as well.
I think what we're... Go ahead.
Yeah, I'd like to at least note that the frontier companies, except for Google — Google has a large investment in A.I. for science — but a reason we don't see the benefits is in part because they don't particularly prioritize things like healthcare. The number of people at these companies working on just making them faster at coding and better agents for making PowerPoints and things like that is orders of magnitude more than the people working on healthcare. Maybe at some of these places you have a small team, and at some of the frontier ones you have zero or one person.
Because it doesn't benefit them as much economically. The tokens or the revenue is coming from coding tokens. So that's what they're prioritizing, and this is then incentivizing them to make better and better agents. I think part of this is there isn't actually that much of a focus on the A.I.-for-good part. It's part of the portfolio. I'd say it's highest at Google, where they're focused on agriculture and healthcare and weather. But it's not as much a priority elsewhere.
And those benefits we really do want, and I think you could get, for instance, an A.I. moratorium on frontier training runs while still having various narrow A.I. carve-outs. So like AlphaFold is not a general LLM. So that could get some type of carve-out. Likewise, voice models, etc., are not these...
Kind of retinal improvement.
Exactly. So a lot of the benefits that we want, we don't need superintelligence necessarily. We can have A.I. technology, and we can train them to be especially good at these various tasks. But then we don't have to eat the risks of extremely capable agents getting loose or swarms being released by malicious actors next year. So I think there's a way that we can get many of the benefits while giving us enough time to figure out how to deal with the extremely powerful agents that are arriving now and next year.
Is that the world you imagine, Max, we may end up with, where the LLMs, the companies that you're talking about, end up being the most heavily regulated, and we create space, carve-outs, exceptions for the companies that seem to be singularly focused on the public good, on public health, on the food supply, on the water supply, whatever it is?
Yeah, I mean, obviously that would favor... I have a hard time imagining that there's just like a broad moratorium on training. My guess about how this will unfold is that there... like, it's clearly untenable for private companies to be the only entities that have like a true superintelligence capability. But I don't think that like the NSA or like the other governments are ever going to turn off that ambition.
Right, of course, like, of course not. To the degree that it's dual-use, it's one of the most powerful national security technologies ever conceived. And so my guess is that what will happen is there will be a superintelligence frontier that is owned by the government, and then there'll be an industrial intelligence frontier which is made widely available, and the government will always have the most capable, most dangerous models. Whether, how they govern that and how they use those and how they contain them and protect them is — I mean, I hope that it's competent, and I hope that they're able to figure out controls for that, but I don't think that's likely to stop.
And then for the rest, it's just like, I think that we are rapidly gaining capabilities. Like, we've just seen in the last six months the ability — like, we use Mythos internally for a range of engineering projects, and its ability to do things like develop better proteins is advancing pretty deeply.
Yeah, I mean, I just — there's already a very large technological overhang in medicine. Like, the gap between what's feasible in the lab and what is delivered to patients is quite large, and I think that there's regulatory reform that we need on clinical research and in clinical translation in order for that to reach more patients. But I think that just as these models advance, that gap is going to get larger, and that is going to become untenable. I think that people will expect access to the state-of-the-art therapies, not just wealthy people somewhere in the world, as it becomes clear that these things are feasible.
Intelligence is a conversion ratio. There's like, you can make it as you have it, as the value of more intelligence asymptotes towards the kind of the impact of the decisions that you can make. For example, if you're playing chess, you can't use that to like take over the internet. You're bounded to the game of chess. But when you're faced with each choice, you can make better or worse choices, better or worse decisions. And so as these models get smarter, what we'll see is that there are more actors around the world that are able to just make consistently closer and closer to optimal decisions always.
And that will then run into other like rate limits that affect how quickly things diffuse out or what you can do with that. And I mean, we're seeing really interesting emergent behavior from these agents. Like, I'm not saying that I think that like, oh, everything is just like totally safe, we should just plow straight ahead as fast as we can. But I don't think that the genie is going to be put back in the bottle. There's some really interesting, really fundamental unifications between A.I. research and neuroscience that I think really hint to me that this is a real effect. It's not a gimmick. It's not hitting a wall. And the benefits of it are just going to be so, so great that I think we're going to have to figure out how to diffuse them into society. I don't think we're going to stop.
Dan, does it pain you to hear Max say that the genie can't be put back in the bottle? Because you do talk about the need for moratoriums, and you do take a more sweeping approach to how to rein this in.
Well, yeah. And historically I was more into really smart, technocratic, extremely narrow interventions, but you need a lot of them to make things go well, and that needs a lot of time to be implemented, too. The past several years, the companies blocked all that. So now we're going to need a more blunt intervention. That's how it is.
As for can we end up stopping frontier training runs, you noted that there's some geopolitical or national competitiveness reasons for this. It still is the case that it could be a risk to national security of having extremely powerful AIs that you can't reliably control. So there may be incentives in much the same way that we don't develop bioweapons, even though that could provide some strategic capability if you did a lot of research on them to get them more targeted. We have a ban on that internationally, and likewise for chemical weapons, but they're difficult to control. So that's one part that could be a threat to national security.
Then the other is that states can deter each other, too. So they could say, if you're trying to use this ASI to destroy me by doing a sweeping cyber attack, that would be a splendid first strike. I will deter you from doing that by threatening you through some means, or I will poison the pre-training data and add backdoors to these models so that if you're going to try and sic it against me, it will actually turn against you. It's very easy to add backdoors to these models, and so there are some strategic... because the AIs are not — we can't make them reliably reliable and fully secure. We have basically no traction on the problem of backdoors, as an example.
It's just a technology that's going to require a lot more safety and security research and interventions before it can be sufficiently de-risked and be a source of a reliable strategic advantage. And at the same time, if it's also creating economic upheaval or causing other disasters along the way, I think that's what could be a trigger that would make a moratorium that seems, you know, hard to see right now actually fairly feasible if there is the national...
But I mean, I don't... I mean, I want to make sure I understand what your blunt, what your blunt instrument solution is to...
The blunt is there's still carve-outs for narrow interventions, but it's just a frontier training run.
But it's like, it's a war on drugs. It's a ban on... you're doing an indefinite pause on their big runs.
I mean, just to be really clear about how — I suspect you understand how that may be kind of quixotic, fanciful, pick your SAT word — because like literally Anthropic is going public based on the unlimited ability to do the very thing you're suggesting there would be a moratorium on. So that doesn't...
They can sell inference. So they can still later on, let's say we stop middle of next year as an example, the A.I. agents will be so economically valuable. You mentioned there's all these diffusion overheads, and there's still so many implications that we can work through with these technologies. So I think that they'll be able to still sell their tokens then, and there's a lot of compute demand for that compute.
So just to push us one beat further, because I think it's clearly the most ambitious idea we're going to probably hear at this table when it comes to the regulatory world. What do you suspect happens during the period there's a moratorium, and how is it constructive?
Sure, I mean, so we could work on making the AIs actually benefit people. We can work on these diffusion, and we can work on figuring out how to make the systems more reliable, more controllable, more secure, make them not have their jailbreaks, make them not have backdoors, make them more aligned with various different communities, and also securing these A.I. systems, too, because it's very easy for other nation states to just steal the weights. This will take multiple years to even be on top of that. We can also defend the world, too. We could be patching our critical infrastructure, making it more secure for future generations of cyber-capable AIs. So there's a variety of things we could do.
I suspect everyone at this table has a thought about that. But if Jensen Huang were at this table, what he would say is China's not going to observe that moratorium. But what were you thinking?
No, exactly that. It's, um, agree. If you get a global alignment — and can you get a global alignment is the big question mark. Otherwise, 100%.
Otherwise... otherwise it's just a waste of time and... as in, if you cannot get global alignment, if you cannot get the key frontier labs to agree, that then becomes a problem.
A big problem. Also, we're doing those things, right? Like, people for the last eight months have been frantically patching critical infrastructure. Like, every time these models come out, many companies now have harnesses that just like funnel all their code through it and look for new vulnerabilities and patch them, and that cycle is getting much shorter. I think that people do find value from A.I., as much as, as controversial as it is. Like, a billion, like a billion monthly active users are not using it because they just hate it. And so clearly this is still available. People could still use the...
But I think the regime that you'd have to put into place to enforce that type of ban on training, I think is like as — that is as draconian as whatever people are worried about Anthropic is doing. Like, I think that this is a surveillance — you're setting up like an exhaustive surveillance state on the internet to enforce this.
And that begins taking you down, and that will be, by the way, that will be powered by A.I., who will then use this to enforce this prohibition. And I worry that that's a hole you never dig yourself back out of.
The moratorium would require the kind of invasive structures that you don't unbuild those, right? When we set up the Patriot Act and like the national security state, and countries do this, they don't change these laws back.
So this isn't saying you can't train a local image classifier or anything like that. This is frontier training run with a certain number of flops that require hundreds of billions of dollars or tens of billions of dollars worth of GPUs.
But that's how — that's data... well, you can use that compute for narrow, you know, actual applications, but not train — but not just making generally smarter agents.
Yeah, but there is a G factor in the models, like the better models. So smarter models not trained on some specific domain get better at that domain. There are generally shared intelligent capabilities here, and so I don't know that you can just say, okay, like for example, AlphaFold is a narrow model that was trained very specifically just to predict the distances between base pairs. That is not a general-purpose LLM. I think we'll have some of those, and like we use models like that to develop new proteins, but the generally intelligent models are essential. There's a lot that they will do that you are not doing with these narrow, task-specific models.
Access to the existing ones or the ones at the time of that, and you can customize them specifically for the types of tasks that you're wanting. So you can use the pretty high G factor that's already in these...
But who — the weights weren't released because it might have been dangerous, and so how do we know at this time that's some other people... I'm guessing that this idea does not appeal to you, but I don't want to guess.
I don't think it's an either-or. I think it's an and. I think we, you know, all the things that were mentioned are things that we have to be building in from the start. I don't think being safe and responsible is something you bolt on to the end. I think it's something that absolutely has to just be baked into the way that a company operates, because I wouldn't be in this role or in this industry if I didn't believe in what the positive potential could be, which is how do we — like, we are not solving the world's challenges — like, we may be creating some, but we're also not solving the world's challenges as fast as they're coming, and we don't have time. And so I really believe that A.I., you know, led by a human — so keep a human in the lead — has the potential to really help us build a better world for my kids and their generation.
But it requires us to be actively working on all of these things all along. And it's part of the reason why I actually embrace a lot of these conversations, because the way I think about it is that Hollywood has given us a gift of science fiction movies, because it's created this like common language for us to have these conversations of what could go wrong, so that we can have them openly and put some more accountability into the system. And I think that's really what we need.
And to your point earlier, like the election showed that the public has all of these concerns, and I think it's up to all of us to have these kinds of conversations and start thinking about how do we rebuild the trust. And that is like, okay, if we're going — how do we have more transparency in the approach that we're taking as we're building? How do we — you know, when you're doing something pioneering, even if you have all the right structures internally and you're doing the testing externally with people before you release it, something could still go wrong. It's the nature of doing something brand new that's never been done. It's that ability to respond and react.
I think that's also going to be really important, is how quickly can we respond and react when something does go wrong, and how do we do that collaboratively? Things like the...
frontier model forum or publishing some of our frontier safety frameworks, like what are the critical points in which something is starting to look like it's getting risky, and then what are the remediations that we're going to take. So some of those we've been, many of the labs have been very public and open about, and I think that's the type of thing that we shouldn't overlook, but we should also have some accountability that we're actually living up to doing what we say we're going to do.
Do any of you worry that if you, if especially the LLMs, the big companies get this wrong, that this will inevitably follow a familiar trajectory in American politics, which is that a problem that everyone can see but no one is dealing with? It just keeps growing and growing until there is not a midterm election, but usually a presidential election. And here I think about Donald Trump in 2016 and then again in 2020, where there's such a kind of mass discontentment that someone comes into office with a seemingly somewhat revolutionary agenda. And here, let me just spell it out: a Democrat running for president in two years or so picks up on the numbers that I described earlier, the public discontent with AI, with data centers, this sense that overlords are not watching out for them but instead are exploiting them, and runs on a message that this whole thing needs to be massively reined in, and that reining in is going to be maybe a shock to the system because it hasn't been firmly established enough along the way. That seems like a very familiar cycle to me in American politics.
Yeah, I mean, we did this in the '60s and '70s with nuclear energy. I think we largely regret this now. I mean, certainly these cycles have happened. I think that realistic... we freaked out and we over... Yeah. I mean, the NRC has, up until very recently, issued zero... Yeah, zero permits since its inception until just very recently for new... really fascinating metaphor. And it's like, how do we make nuclear energy perfectly safe? Well, if we never do it, then we will never have any accidents. And so clearly America has a history of regulating like this. I think people will find it too useful.
I think AI is just too useful. And I think this is like part of the central paradox: it really does have the potential to make people's lives so, so much better and improve, like, not just individuals and their community, but also like the thriving of America. There are these tensions here. So like right now people obviously are trying to stop data centers. They're asking like, well, why do you have to build them here? Why can't you build them overseas? And it's like, well, we did this before and everyone got really mad about it. [laughter] And I think ideally we would build these things in America, but...