Scientific Director, Mila
AI Scientist Bengio: Building Systems We Don't Know How to Control
Yoshua Bengio, Founder & Scientific Advisor at Mila Quebec AI Institute and Co-President & Scientific Director at LawZero ...We're speaking with Yoshua Bengio, founder and scientific advisor at Quebec AI Institute, co-president and scientific director at LA ...
About Yoshua Bengio
Yoshua Bengio, scientific director of Mila and co-president of the nonprofit LawZero, has continued to speak publicly about the risks of advanced artificial intelligence. In multiple interviews and appearances between June and July 2026, Bengio argued that AI systems are being built faster than governments can regulate them and that current guardrails are insufficient. He described the convergence of AI and biotechnology as a major threat to biosecurity, stating that the technology could soon allow someone without a scientific background to design a dangerous pathogen. Bengio also said that AI models are already capable of persuasion that could undermine democratic institutions, and he warned that if current trends continue, a small number of countries could dominate geopolitically through AI. Bengio has advocated for a regulatory model similar to those used for drugs, airplanes, and cars, where developers must demonstrate safety before deployment. He called for international coordination, including between the United States and China, to prevent AI from being weaponized. Bengio also discussed his work at LawZero on mathematical methods that could provide guarantees about AI behavior, a development he described as previously impossible. He said that if he had an unlimited research budget, he would fund a "Manhattan project" for safe, powerful AI that benefits the world.
Transcript (44 segments)
I
Interviewer0:00
Can you share the international AI Safety report? We'll just start there. Is AI safe?
Y
Yoshua Bengio0:05
No, that's the problem. We're building systems that we don't know how to control. They will behave sometimes against our instructions. And that is this lack of reliability is both a scientific challenge and a problem for deployment of AI.
I
Interviewer0:25
So how do we okay. We use social media as the example. We failed that experiment. I think many would say that we did not keep ahead in terms of oversight regulations. We could go into all the things that have gone wrong and how it's impacting society. What faith do you have that we get this right? And it can't just be the US society or Europe deciding it's got to be a global work.
Y
Yoshua Bengio0:48
Absolutely. Safety is a global question because a dangerous model, I mean, one that can be misused in dangerous ways, like methods, for example, could be designed in one country, and then somebody in a second country uses it to attack a third country. And, you know, so the Americans need to make sure that Chinese models are not used against the US and vice versa. It is an international question.
I
Interviewer1:11
Can I ask you, is it like missiles and nuclear weapons and that there will be a mutual deterrent because everyone will be proficient on some level when it comes to AI?
Y
Yoshua Bengio1:25
They are analogies. I think one of the big differences is that there's just as much money to be made with nuclear weapons as there is with AI. Which means the incentive structure is now pushing companies towards going faster and faster and not paying enough attention to the safety issues. And, you know, the public protection.
I
Interviewer1:42
But I think one thing that concerns a lot of people is that, okay, you could have let's just use Anthropic as an example. You have Anthropic and what they're doing with models. But just because Anthropic has that right now doesn't mean DeepSeek couldn't have that six or 8 or 12 months from now, or even something more powerful.
Y
Yoshua Bengio2:01
That's right.
I
Interviewer2:03
So it doesn't matter if like one country is doing things a certain way, or one company is doing a certain way, if the entire world is not actually adhering to a certain set of framework.
Y
Yoshua Bengio2:11
Exactly. That is why we need international coordination. Initially, it's probably going to be just the U.S. and China, but at some point, as other countries are also, you know, beefing up their capabilities, it has to be international.
I
Interviewer2:24
What keeps you up at night, like lay out the doomsday scenario for us?
Y
Yoshua Bengio2:27
Well, there are like many catastrophic risks because intelligence gives power and that power could be in the wrong hands. It could be used by terrorists or countries. It's one of these devices. It could be used even by the people in power to create, you know, a dictatorship. And we don't know how to make sure AIs themselves will not use that power against us.
I
Interviewer2:48
But how? Like harnessing social media, logging in, like taking infrastructure offline. Like give us concrete examples of what these entities could do.
Y
Yoshua Bengio2:56
So what the companies are worried about right now are cyberattacks, because AI really has a lot of knowledge of computing and vulnerabilities. And the evaluations they're making is also the use of AI to build dangerous viruses. So you can think of people using AI to create new pandemics. They're also worried about what happens when AI itself is used to create the next version of AI. So AI doing AI research, and if we don't control these systems, they could put back doors to create AIs that are not like friendly to us.
I
Interviewer3:33
So in other words, you're talking about AI that can think on its own and work on its own.
Y
Yoshua Bengio3:38
Well, that's already the case already. The whole point of agents is that they have goals and they work towards those goals. The real question is who controls those goals? We would like to be the ones in charge, but right now we're not. We don't have the technical answers to make sure that's the case.
I
Interviewer3:55
Yoshua, do you regret the new work that you did years ago?
Y
Yoshua Bengio3:59
I think I should have been seeing heads, you know, earlier that we were building something that could become extremely powerful and that we don't know how to control, and it would impact societies in which they are currently on the current trajectory could be disrupted.
I
Interviewer4:19
So okay, but here we are right? And it sounds like it does feel very doomsday and scary, but I do. And it also feels like it's too late.
Y
Yoshua Bengio4:29
No it's not, but and I want to talk about that.
I
Interviewer4:31
But I also feel like there's a lot of potential to do some great things as well. Talk to us about that side.
Y
Yoshua Bengio4:37
Absolutely. I've been working for many years on how we can use AI to design better drugs in medicine more broadly, how we can use AI to design like better materials for batteries, for energy storage. And I know a lot of people working, for example, in how AI can help agriculture, can even help the climate debate. The question is, the same power that enables these things also enables, for example, in the case of biology, dangerous uses that could really harm a lot of people.
I
Interviewer5:10
Do the potential positives outweigh the potential negatives?
Y
Yoshua Bengio5:16
Well, there's so much we don't know here. You have to weigh in the uncertainty and like it's because it sounds great to have better medicine. It sounds great to have more efficient batteries. It sounds great to have an ecosystem where people are able to learn more and do it quicker and be more efficient. But none of that matters if the AI agents are going to kill us. Or if we lose our democratic institutions and we end up in a dictatorship.
I
Interviewer5:42
So these are already having like if I think about democracy, let's go there. We're already having these kinds of problems.
Y
Yoshua Bengio5:50
We do. And this is a pretty... it could get worse. There's studies showing that AI is now getting better than people at persuasion. In other words, making people change their minds. Imagine this applied to change political opinions. Change public opinion. This is like shaking the foundations of what democracy rests in.
I
Interviewer6:08
You know, I don't know. You know, people talk about the new industrial revolution. They make comparisons. Do we have any comparison in terms of the impact?
Y
Yoshua Bengio6:19
I think it's no. I mean, it is something else that's changed our world dramatically. The light bulb. As usual, I think, you know, the past can tell us things, but you have to realize we've never been in a situation where we build technology machines that would be smarter than us in many ways. And this is already the case in, you know, various domains. And it's just growing. There's nothing in history like we built machines that help us physically have much more powerful like muscle wise, but brain wise, this is completely new.
I
Interviewer6:53
We're speaking with Yoshua Bengio, founder and scientific advisor at Quebec AI Institute, co-president and scientific director at Mila, focused on building safer AI agents. The productivity question is one that we want to get to. The investment that's been made from a CapEx perspective has just been so massive. The promises are that this will actually make us so much more productive and will lead to higher returns on investment. Is that true?
Y
Yoshua Bengio7:21
I don't have a crystal ball, but I see there are different scenarios about how AI intelligence capabilities will grow in the future. That's a great question if you can. I think the default is going to continue on the same trend, but it could also saturate. And maybe some of these promises will not be realized. The other issue is even if AI is more capable, it doesn't mean that it is behaving well. In fact, there is a serious scientific hypothesis that intelligence and the goals to which that intelligence is put to use are like two very different things.
I
Interviewer7:57
All right. So we talk about races in the world. And this is certainly a tech race. An AI race right. So it sounds like for things to go better safer you've got to have global collaborations. So what is happening on that regard or what needs to happen in your view based on your knowledge you understand how this works. So what needs to happen to make sure that we don't have AI taking over?
Y
Yoshua Bengio8:21
I would say the most important thing that needs to happen is better understanding and awareness of what we are talking about by the governments around the world, and of course, China and the US in the first place. But eventually, you know, many governments who have researchers and companies building AI as well, because once you understand the magnitude of the risks that we're taking, we'll sit down at the table and negotiate.
I
Interviewer8:51
One of the things I'm going to go back to what you said about productivity, what about the labor market like is it going to create? It's already tough for some of those starting jobs. I think a kid's coming out of college. Or like how what does it mean for the global economy? What does it mean for the labor force? Does it create jobs? I mean, building data centers. Yes. But what how do you see it?
Y
Yoshua Bengio9:11
I mean, the number of people needed for data centers is peanuts compared to the effect that they could have on the job market. But the economists don't agree. It really depends on the curve of advances in AI in the future if it continues at a current rate. And yes, there will be massive automation coming too quickly for society to adapt.
I
Interviewer9:29
Yoshua, given the work that you've done in understanding the potential pitfalls of the technology. Name names for us. Who's doing this well? Who understands the potential repercussions here?
Y
Yoshua Bengio9:43
I don't name names. I think there are a lot of people who have good intentions. The issue that makes the current dynamics dangerous in some ways is the incentive structure. Is the competition between companies and between countries.
I
Interviewer10:02
So maybe let's ask this question a different way. What if you could have Dario Amodei in a room with Sam Altman, along with Jensen Huang, along with the leaders of all the hyperscalers, and then, of course, the leaders of the Chinese companies as well? What would you tell them? What do they need to know?
Y
Yoshua Bengio10:19
So they know already, right? They know what I'm going to tell them. What they need to do is to agree with each other on a set of principles about safety and like, careful deployment of AI. And they're already talking about this. What they are asking actually, is governments to agree with each other, to make sure there's a level playing field for all the companies so that every company, before they build and deploy very powerful AI, needs to show to the public, to the governance that their system isn't going to be, like dangerous in the wrong hands and all these issues we're talking about.
I
Interviewer10:58
But are the financial incentives getting in the way the race is on? Yeah, it seems like a prisoner's dilemma. And it's who can go fastest and get it all done. So does that prevent the right thing from having just got about 30s?
Y
Yoshua Bengio11:12
No, we've dealt with situations like this before. I mean, most governments have regulated things like drugs, you know, trades, needs. Because if it was just competition, we would end up with, you know, dangerous food, dangerous planes, dangerous drugs. But now we enjoy all these things that are safe and useful because governments have intervened.
I
Interviewer11:36
So, in a word, yes or no, you're more hopeful than worried?
Y
Yoshua Bengio11:39
Yes.
I
Interviewer11:42
Hopeful or no?
Y
Yoshua Bengio11:44
I'm... I don't know. I mean, I feel like if it's not my vision that matters, it's our actions that we take collectively.