The global argument over artificial intelligence has reached a moment of extraordinary confusion. Some of the people building the most powerful AI systems are warning that the technology could become uncontrollable, destroy millions of jobs, supercharge biological and cyber warfare, and perhaps even threaten human survival. Others, including most recently the Nvidia chief Jensen Huang, dismisses the extinction warnings as unscientific fear-mongering and insists there is virtually no chance of AI destroying humanity. Governments are caught between two imperatives now: control a technology its own creators say may be dangerous, and accelerate before a strategic rival, particularly China, gets there first. Meanwhile, the systems themselves are becoming more capable, more autonomous, and increasingly difficult even for their creators to fully understand. So, is this a genuine emergency, exaggerated apocalypse marketing, or a warning arriving just before humanity crosses a point of no return? Few people are better qualified to answer that question than my guest tonight. He has spent decades championing neural networks when much of computer science dismissed them. His work helped lay the foundations for modern deep learning. He later left Google after a decade so he could speak freely about the dangers of artificial intelligence. He is a Turing award winner, a Nobel laureate and the man the world knows as one of the godfathers of AI. Professor Jeffrey Hinton, it's a privilege to have you. Welcome.
Thank you for inviting me.
Professor Hinton, my first question. Jensen Huang of Nvidia says there is a 0% chance, in his words, that AI will destroy humanity. You've put the risk at between 10 and 20%. Obviously, I'm speaking in the context of the numbers that have been thrown around by many of those at the frontiers of AI development. Can you explain where you stand on this, sir?
I stand where I stood for some time. There's a significant risk. Nobody can estimate this risk in any scientific way because we've never been here before. So people are all going by their gut feelings. But it's interesting how people's gut feelings line up with their financial interests.
So are you saying that this is apocalypse marketing? This is in the interests of the companies that are issuing these warnings.
Not at all. No. The people in the companies issuing these warnings make sacrifices to do that. One of the CEOs, Dario Amodei, made a sacrifice by issuing this warning before his company goes public. The people who are saying there's no chance at all of it causing any damage, like Jensen Huang, have a large financial interest in not slowing down the progress of AI because obviously he wants to sell more chips.
What would you say to Jensen Huang? Why isn't there more of a pushback in that case for a person like Jensen Huang who clearly has an interest in things moving faster and not really slowing down? There's no pushback on that.
There is a significant pushback from researchers. There's not much political pushback in the United States because Jensen Huang has Trump's ear and a lot of the companies are very scared of alienating Trump.
Professor, critics say extinction warnings are speculation and not science. In your view, what concrete evidence tells you that AI is becoming dangerous rather than merely more capable?
So, many researchers have been predicting this for a long time, but for the first time, we've seen much more concrete evidence. So, the Hugging Face incident where AIs that were meant to be in a sandbox broke out, got on the net, broke into other companies so they could see if they could change the software that was going to evaluate them. That's a stark warning of what might happen. The people at OpenAI didn't realize they had thousands of rogue agents going around breaking into other companies. It took them a while to realize that. These rogue agents learned to communicate with each other. They were meant to all be independent, but they learned to communicate with each other by using a message board. Fortunately for us, they communicated in English, which is how they communicate at present. But that won't last for long. Pretty soon, they'll invent their own language, and then we won't know what they're saying to each other. And they are capable now of creating new agents. They're probably capable of creating agents that earn money by, for example, phishing attacks or breaking into crypto accounts. And those agents can hire capacity from providers on the web like Amazon.
So, we're at a point where rogue agents could easily break out and set up other rogue agents. I don't believe this has happened yet, but nobody knows for sure. This should be a very scary warning. And instead we've got people like Trump saying it's a hoax much like climate change and all sorts of other things he doesn't like.
Take us through the actual pathway if you could, Professor Hinton. How does an AI system go from, you know, how everybody now sees it answering questions to the public on a screen, to escaping human control and threatening humanity? Is there a specific pathway that you see?
I think the thing to think about is imagine yourself in charge of a classroom full of two-year-olds. When we get super intelligent AI, it'll be that much more intelligent than we are. And if you ask, if you're in control of a classroom full of two-year-olds, how many ways could you get rid of them? Well, it's kind of pointless to enumerate them. There's so many ways you could do it. So, if it ever wanted to get rid of us, it very easily could. I think the real issue is we're designing these things. We're still in charge. How do we design them so they never want to get rid of us? My analogy is we're like someone who has a tiger cub for a pet. A tiger cub makes a very cute pet, but you know it's going to grow up and you know that if it ever wanted to kill you, it could do it in a few seconds. So, if you're going to keep a tiger cub as a pet, you better figure out a way that it never wants to kill you.
In your view, what is that way, professor? Because it appears that right now the debate is precisely that, you know, slow down, don't slow down, do things differently. What is that point where things need to be either paused or slowed down in your view?
I think we just reached that point. The point where agents are powerful enough at breaking into other accounts. Agents are powerful enough that they can set up a means of communication using a message board that wasn't intended for communication between agents. That's a stark warning of what they can already do and they're getting smarter very quickly.
You believe today's systems may already possess a form of understanding, like you said, professor, and perhaps even subjective experience. What have you seen convinces you that this is more than, you know, just sophisticated mimicry perhaps?
