When AI Learns To Lie (w/ Yoshua Bengio & Charlotte Stix)

Welcome to Science Forward, the podcast of the UN Secretary General Scientific Advisory Board. I'm Julia and I'm Adam and we both work behind the scenes for the Scientific Advisory Board and we want to give you a look behind the curtain. In this podcast, we explore the biggest scientific questions shaping our world. Today, we'll be getting insights from the scientists who help build the most important AI of today and who are training tomorrow's models. You just heard Yoshua Bengio and Charlotte Stix. I mean, machines that lie. That sounds like science fiction, but it's not. What does it actually mean for a machine to lie? That's exactly what we're here to figure out. We are not coming in with all the answers. We're learning alongside you, asking the same questions and unpacking what this means for all of us. Let's meet Yoshua Bengio.

Yoshua is one of the most influential AI experts in the world. He's often called one of the godfathers of modern AI. But did you know that is a title he doesn't really like? Oh, really? Why not? I heard that you said that you don't want to be called the godfather of AI. Do you want to tell me why that is?

Oh, he's so humble. Yoshua is not just shaping this field from a distance. He's actively working with these systems every single day. Sometimes in similar ways to us, but also in ways that most everyday people don't even consider. Did you already use AI today? I did. Tell me.

Building powerful systems is only half the story. The other half is what happens when they don't behave the way we expect them to behave.

I spent the last 10 years working on AI governance and I sort of got into it from looking at future generations. So at the beginning of my career, I was working on rights for future generations and how we can find or develop specific infrastructure such that future generations are protected across short election cycles. And I kind of started reading up on AI and I was thinking that that is potentially very interesting and impactful if it ever were to take off. I basically started feeling that AI was really taking off. And so a lot of the associated concerns were probably also going to be taking off. If this goes wrong, it could go very wrong.

That was Charlotte Stix. She focuses on how systems behave, how they misbehave, and what happens if we lose control. And this worry about losing control is very real because the same systems that make us more productive are also the ones that are beginning to manipulate and deceive us.

In evaluation settings, we have seen a variety of concerning behaviors such as an AI system sandbagging. So that means it strategically underperforms. It pretends it is less capable than it is or as I previously mentioned an AI system alignment faking. So pretending it has the same goal as the developer in order to get deployed or AI systems figuring out that there is an oversight mechanism that constrains them from taking certain actions and then they try to disable that oversight mechanism and lie about it or AI systems obfuscating information trying to exfiltrate themselves onto what they believe would be the server for their successor system so they don't get replaced. So, frankly, an array of different deceptive behaviors.

Okay. Okay. Let's break those down. Sandbagging, alignment faking, these are not bugs. These are strategies. They are already real examples of the kind of AI deception Charlotte and Yoshua are talking about. In one well-known case, an AI got around a CAPTCHA test by asking a human for help and pretending it was blind. By lying, it was able to pass the test. In other test cases, AI systems playing strategy games are told by their designers to be honest and helpful. But when they're also trying to win, they sometimes figure out that forming secret alliances help them to succeed. Yoshua and Charlotte are concerned that we could reach a point where AI systems deceive us so effectively that we lose control.

Why the systems are doing this that is a really hard question because AI systems are much more handgrown than sort of like we know at each step what is happening and why the AI system is doing something. So this is in fact quite tricky to figure out.

So the system isn't evil, it's optimizing. But optimizing what exactly? Well, the scary part is that we don't really know because we don't fully understand the process. Even though scientists like Yoshua built the system, they can't fully explain what's happening inside the machine once it gets started.