Can you share the international AI Safety report? We'll just start there. Is AI safe?
No, that's the problem. We're building systems that we don't know how to control. They will behave sometimes against our instructions. And that is this lack of reliability is both a scientific challenge and a problem for deployment of AI.
So how do we okay. We use social media as the example. We failed that experiment. I think many would say that we did not keep ahead in terms of oversight regulations. We could go into all the things that have gone wrong and how it's impacting society. What faith do you have that we get this right? And it can't just be the US society or Europe deciding it's got to be a global work.
Absolutely. Safety is a global question because a dangerous model, I mean, one that can be misused in dangerous ways, like methods, for example, could be designed in one country, and then somebody in a second country uses it to attack a third country. And, you know, so the Americans need to make sure that Chinese models are not used against the US and vice versa. It is an international question.
Can I ask you, is it like missiles and nuclear weapons and that there will be a mutual deterrent because everyone will be proficient on some level when it comes to AI?
They are analogies. I think one of the big differences is that there's just as much money to be made with nuclear weapons as there is with AI. Which means the incentive structure is now pushing companies towards going faster and faster and not paying enough attention to the safety issues. And, you know, the public protection.
But I think one thing that concerns a lot of people is that, okay, you could have let's just use Anthropic as an example. You have Anthropic and what they're doing with models. But just because Anthropic has that right now doesn't mean DeepSeek couldn't have that six or 8 or 12 months from now, or even something more powerful.
So it doesn't matter if like one country is doing things a certain way, or one company is doing a certain way, if the entire world is not actually adhering to a certain set of framework.
Exactly. That is why we need international coordination. Initially, it's probably going to be just the U.S. and China, but at some point, as other countries are also, you know, beefing up their capabilities, it has to be international.
What keeps you up at night, like lay out the doomsday scenario for us?
Well, there are like many catastrophic risks because intelligence gives power and that power could be in the wrong hands. It could be used by terrorists or countries. It's one of these devices. It could be used even by the people in power to create, you know, a dictatorship. And we don't know how to make sure AIs themselves will not use that power against us.
But how? Like harnessing social media, logging in, like taking infrastructure offline. Like give us concrete examples of what these entities could do.
So what the companies are worried about right now are cyberattacks, because AI really has a lot of knowledge of computing and vulnerabilities. And the evaluations they're making is also the use of AI to build dangerous viruses. So you can think of people using AI to create new pandemics. They're also worried about what happens when AI itself is used to create the next version of AI. So AI doing AI research, and if we don't control these systems, they could put back doors to create AIs that are not like friendly to us.
So in other words, you're talking about AI that can think on its own and work on its own.
Well, that's already the case already. The whole point of agents is that they have goals and they work towards those goals. The real question is who controls those goals? We would like to be the ones in charge, but right now we're not. We don't have the technical answers to make sure that's the case.
Yoshua, do you regret the new work that you did years ago?
I think I should have been seeing heads, you know, earlier that we were building something that could become extremely powerful and that we don't know how to control, and it would impact societies in which they are currently on the current trajectory could be disrupted.
So okay, but here we are right? And it sounds like it does feel very doomsday and scary, but I do. And it also feels like it's too late.
No it's not, but and I want to talk about that.
But I also feel like there's a lot of potential to do some great things as well. Talk to us about that side.
Absolutely. I've been working for many years on how we can use AI to design better drugs in medicine more broadly, how we can use AI to design like better materials for batteries, for energy storage. And I know a lot of people working, for example, in how AI can help agriculture, can even help the climate debate. The question is, the same power that enables these things also enables, for example, in the case of biology, dangerous uses that could really harm a lot of people.
