We build these things. We're always going to be in control. We'll build them to be obedient. And other people, like Yudkowsky, say, 'No, no, no. These things are going to wipe us out for sure. If anybody builds it, it's going to wipe us all out.' And he's confident of that. I think both of those positions are extreme. It's very hard to estimate the probabilities in between. If you had to bet
On who was right out of your two friends?
I simply don't know. So, if I had to bet, I'd say the probability is in between, and I don't know where to estimate it in between. I often say 10 to 20% chance they'll wipe us out.
What would you be saying to people about their career prospects in a world of superintelligence? What should we be thinking about?
In the meantime, I'd say it's going to be a long time before it is good at physical manipulation as us.
And so, a good bet would be to be a plumber.
Until the humanoid robots show up.
That's Geoffrey Hinton, the guy everyone calls the godfather of AI, the man who literally built the foundations these systems run on. And his real advice to your kids right now is to learn plumbing because being smart won't save your job anymore. Here's the rest of what worries him.
My main mission now is to warn people how dangerous AI could be.
Did you know that when you became the godfather of AI?
No, not really. I was quite slow to understand some of the risks. Some of the risks were always very obvious, like people would use AI to make autonomous lethal weapons. That is, things that go around deciding by themselves who to kill. Other risks, like the idea that they would one day get smarter than us and maybe we'd become irrelevant. I was slow to recognize that. Other people recognized it 20 years ago. I only recognized it a few years ago that that was a real risk that might be coming quite soon.
How could you not have foreseen that? With everything you know here about cracking the ability for these computers to learn similar to how humans learn, and just, you know, introducing any rate of improvement.
It's a very good question. How could you not have seen that? But, remember neural networks 20, 30 years ago were very primitive in what they could do. They were nowhere near as good as humans at things like vision and language and speech recognition. The idea that you have to now worry about it getting smarter than people, that seemed silly then.
It changed for the general population when ChatGPT came out, it changed for me when I realized that the kinds of digital intelligences we're making have something that makes them far superior to the kind of biological intelligence we have.
So, the man who basically built this field is telling us he didn't see this coming until a few years ago, not decades, just a few years, while these systems were already being trained at full speed. And his reason isn't some wild guess, he's saying digital intelligence is just built differently from ours, and that gap keeps growing every single day. Now, listen again.
So, let's do the risks then you talked about there's human risks and then there's human
So, I've distinguished these two kinds of risk. Let's talk about all the risks from bad human actors using AI. There's cyber attacks. So, between 2023 and 2024, they increased by about a factor of 12, 1,200% and that's probably because these large language models make it much easier to do phishing attacks.
And for phishing attack for anyone that doesn't know is
It's they send you something saying 'Hi, I'm your friend John and I'm stuck in El Salvador. Could you just wire this money?' That's one kind of attack. But, the phishing attacks are really trying to get your log-on credentials.
And now with AI they can clone my voice, my image.
I'm struggling at the moment because there's a bunch of AI scams on X and also Meta. And there's one in particular on Meta, so Instagram, Facebook at the moment, which is a paid advert where they've taken my voice from the podcast, they've taken my mannerisms, and they've made a new video of me encouraging people to go and take part in this crypto Ponzi scam or whatever. And we spent weeks and weeks and weeks and weeks emailing Meta telling please take this down. They take it down, another one pops up. You take that one down, another one pops up. So, it's like Whac-A-Mole.
The heartbreaking part is you get the messages from people that have fallen for the scam and they've lost 500 pounds or 500 dollars.
Cross with you because you recommended it.
I'm like I'm sad for them.
It's very annoying. I have a more of a version of that which is some people now publish papers with me as one of the authors. And it looks like it's in order that they can get lots of citations to themselves.
So, cyber attacks are a very real threat. There's been an explosion of those.
Yes. And these already, obviously AI is very patient, so they can go through 100 million lines of code looking for known ways of attacking them. That's easy to do. But they're going to get more creative, and they may, some people who know a lot believe that maybe by 2030, they'll be creating new kinds of cyber attacks, which no person ever thought of. So, that's very worrisome.
Because they can think for themselves and decide new ways to attack.
They can draw new conclusions from much more data than a person ever saw.
Is there anything you're doing to protect yourself from cyber attacks at all?
Yes. It's one of the few places where I changed what I do radically because I'm scared of cyber attacks. Canadian banks are extremely safe. In 2008, no Canadian banks came anywhere near going bust. So, they're very safe banks because they're well regulated, fairly well regulated. Nevertheless, I think a cyber attack might be able to bring down a bank. Now, if all my savings are in shares in banks. So, if the bank gets attacked and it holds your shares, they're still your shares. And so, I think you'd be okay. Unless the attacker sells the shares because the bank can sell the shares. If the attacker sells your shares, I think you're screwed. I don't know, I mean, maybe the bank would have to try and reimburse you, but the bank's bust by now, right? So, I'm worried about a Canadian bank being taken down by a cyber attack and the attack selling shares that it holds. So, I spread my children's money between three banks in the belief that if a cyber attack takes down one Canadian bank, the other Canadian banks will very quickly get very careful.
And you have a phone that's not connected to the internet. Do you have any like, you know, I'm thinking about storing data and stuff like that? Do you think it's wise to consider having cold storage?
I have a little disk drive and I back up my laptop on this hard drive. So, I actually have everything on my laptop on a hard drive. At least, you know, if the whole internet went down, I had the sense I still got it on my laptop and I still got my information. Then the next thing is using AIs to create nasty viruses. And the problem with that is that just requires one crazy guy with a grudge. One guy who knows a little bit of molecular biology, knows a lot about AI, and just wants to destroy the world. You can now create new viruses relatively cheaply using AI. And you don't have to be a very skilled molecular biologist to do it. And that's very scary. So, you could have a small cult, for example. A small cult might be able to raise up a few million dollars. For a few million dollars, they might be able to design a whole bunch of viruses.
Well, I'm thinking about some of our foreign adversaries doing government-funded programs. I mean, there was lots of talk around COVID and Wuhan laboratory and what they were doing in gain-of-function research. I'm wondering if in, you know, a China or a Russia or an Iran or something, the government could fund a program for a small group of scientists to make a virus that they could, you know,
Guys, cyberattacks jumped 1,200% in 1 year, and Hinton is saying by 2030, these systems might invent attacks no human has ever thought of, which means there's no defense ready for it. On top of that, he's saying a small cult with a few million dollars could design a virus right now using AI, no real science background required. Honestly, that's not a thought experiment anymore. That's the guy who built this stuff telling you the door is already open.
Then there's corrupting elections. So, if you wanted to use AI to corrupt elections, a very effective thing is to be able to targeted political advertisements, where you know a lot about the person. So, anybody who wants to use AI for corrupting elections would try and get as much data as they could about everybody in the electorate. With that in mind, it's a bit worrying what Musk is doing at present in the States. Going in and insisting on getting access to all these things that were very carefully siloed. The claim is it's to make things more efficient, but it's exactly what you would want if you intended to corrupt the next election.
How do you mean? Could you get all this data on the
You get all this data on people. You know how much they make, where they live. You know everything about them. Once you know that, it's very easy to manipulate.
Because you can make an AI that
Send messages that they'll find very convincing telling them not to vote, for example. So, I have no reason other than common sense to think this, but I wouldn't be surprised if part of the motivation of getting all this data from American government sources is to corrupt elections. Another part might be that it's very nice training data for a big model.
But, he would have to be taking that data from the government and feeding it into his
Yes. And, what they've done is turned off lots of the security controls, got rid of some of the organization to protect against that.
So, that's corrupting elections.
Okay. Then, there's creating these two echo chambers by organizations like YouTube and Facebook showing people things that will make them indignant. People love to be indignant.
Indignant as in angry? Or, what does indignant mean?
