Welcome back to the Google DeepMind podcast. I'm Professor Hannah Fry.
Picture the scene. You are scrolling through your socials and you see some remarkable footage. The up-close slow-motion eruption of a volcano, or a tornado raging through a village, or perhaps some footage of a protest. I think many of us ask the same question: is that real? Was this made by a human or a machine? Well, as AI gets better at generating text and images and audio and video, that is becoming one of the most challenging questions of our time. Except that now the answer just got a lot more complicated.
We are entering a world where AI can design biological molecules and structures that have never existed in nature before, some of which could be designed to cause harm. So, how do you stop something dangerous before it gets made? Well, the answer, it turns out, might be hidden in plain sight, embedded invisibly into the very thing you're looking at or the very molecule you're about to synthesize. A watermark, not the blocky logo you see on a stock photo, something far more subtle, something mathematical.
And today I am joined by two people who are working on this from different angles. Pushmeet Kohli is VP of Science at Google DeepMind and one of the architects of SynthID, the watermarking system now being adopted across the AI industry. And Jeremy Radcliffe is a biosecurity researcher whose team has applied that same technology to biology. Welcome to the podcast, both of you. Jeremy, let's start with you. Watermarking anything that AI spits out, I mean, it sounds like a very good idea, but just break it down for me: why is this helpful?
So for biology, sort of the motivation for being able to watermark is so that we can have provenance and say specifically where did this sequence come from. And really what we're trying to differentiate is between sequences that come from things in nature versus sequences that come as products of AI systems. So specifically, the type of risk that we're thinking about in terms of AI systems is sort of an AI-generated sequence that can have properties that we know to be problematic, but we might not be able to identify from the sequence itself that it has these problematic properties.
And what makes a good watermark? Because I think when people consider the phrase, they sort of think of, I don't know, like banknotes or like passports. Is there like rules for what counts as a good watermark?
I think for us in particular, human imperceptibility is one. So making it very difficult for someone to be able to tell what portion of data has been watermarked.
Because otherwise it would be really easy to erase.
Yeah. Having high detectability, so having some system that can go through and actually identify the watermark with high confidence, and then also good generalizability, to be able to have at least a watermarking method that can work across a bunch of different systems rather than having to make bespoke ones for every single data type.
What do you think, Pushmeet? I mean, does that apply beyond biology and more broadly?
Yeah, I think the main reason why we started the whole watermarking project at DeepMind almost eight years ago now was to give a sense of provenance to what users are seeing. If you look at different modalities, whether it's text, whether it's images, whether it's videos, it was hard to imagine ten years back that we would today be talking about systems that can create images, videos, audio, text, and proteins or enzymes that are indistinguishable from what a human or what exists in nature. And it's an important sort of element for users, to give users this idea that you can really believe what you are seeing.
So what do we need from a watermark? We need three properties. One, it should not degrade the quality, otherwise basically the whole point is lost. People will not use it. The second element is robustness against attack or against transformations. If people want to remove it, it's hard to remove in the sense that even if you make changes to the signal, it will persist. And then the third requirement is that it should be easy to use, right? It should be easy to integrate in the signal and it's easy to detect from the signal. So those are the three sort of properties of imperceptibility, robustness, and scalability that we have designed all our watermarking systems for, with the whole idea that users have more control and have a better view of what they are seeing or what they are sort of using or designing and so on.
It feels like this subject of watermarking has been quite a big deal this summer, but at the same time we're a few years into generative AI being accessible to the public. What's taken so long, Pushmeet? Why is it only now that this has become so widespread?
When image editing software started becoming very sophisticated, people were sort of very concerned about this issue of is this a real image or has this been tampered with. And there was a lot of work and specialized systems which could really sort of analyze an image and think and see the tiny differences that exist in a tampered image and can say, oh yeah, this was a fake image or this has been tampered with and so on. But as generative AI became more sophisticated, those tiny differences went away.
And then that naturally sort of resulted in the question: is this game over in the sense that humans will not have the ability to detect whether something is AI-generated? And this is where watermarking sort of came in. We said the way to sort of make sure that humans always have the ability to understand the origin of where some signal came from is by injecting a signal within it, having a bias so it doesn't go away. Even if the models are perfect and they have the ability to sort of generate images which are indistinguishable from what a camera would observe or what a text that a human would write, we specifically sort of put in some imperceptible things that can allow users to later detect that this was or this originated from an AI model.
And when you say 'we' here, Pushmeet, I mean, you really do mean 'we' in the literal sense, right? It was you guys that came up with a solution for this one that was scalable.
Yes. So our team has been working on this for almost eight years. We started with images because people were concerned about these images being used for fake news, misinformation, and so on. And so there was that motivation to solve that problem. At the same time, although the problem is extremely challenging because of the properties we just mentioned of robustness and imperceptibility and efficiency and scalability, still an image is a very large amount of data. A one-megapixel image has sort of one million numbers, or like three million numbers depending on how it's encoded and so on, to hide information, right, and without changing the content. So we developed watermarking systems for images first.
Later on we had to look at the challenge of how do you do this for other modalities like text, which are not as high-dimensional as images, where actually it's a very small amount of signal, right? If you...
It's not a million numbers in a sentence, is there?
Yeah, exactly. It's a few words. You can't change those sort of few words, and you have to be very cautious in terms of not changing the meaning of the content. And so the approaches that we developed for watermarking text were quite different and distinct from what we use for watermarking high-dimensional signals like images and videos and audio even.
But this idea of watermarking, which you guys pioneered, has now been brought into law. It's essential that any generative content within the EU has to have some sort of watermarking.
I think what regulators and countries around the world have recognized is that they want to give us users this information as to the origins of the content that they are consuming. And one sort of key technology that is seen as a robust solution for it that can scale is watermarking. And which is why not only have we seen a take-up of this idea across the industry, the fact that SynthID is used by Nvidia, is used by other partners like with OpenAI and so on, it's a remarkable sort of thing that many of our partners in the generative AI space have adopted this technology.
And Jeremy, in this episode, we're going to be talking about two different applications of watermarking for AI. So, you've got text and video and audio and so on. And then you've also got for biological structures and sequences. Is the solution quite similar between the two?
Yes. So, I would say they're motivated by similar principles and we were able to adopt a lot of the work that had already been done, including using the open-source SynthID text library and just embedding that directly into another system.
Right. So not just the same tricks, but sometimes the actual same code, you can sort of pick it up and drop it in.
At least start from the same code and then make some adaptations on top of it.
Okay. Okay. So I think people want to sort of understand how this works because it sounds almost magical that you could have a single sentence and you have embedded some sort of watermark in there that's nothing to do with whitespace, it's nothing to do with like putting commas in weird places hiding them in the text, but that the words themselves are hiding a record of how they were generated and where they came from. So, if it's okay with you, Pushmeet, I thought we might just walk through how this works for text. I think that's probably the easiest one to understand.
Okay. So, before we get to that, there is some software out there already that you can get online that claims to be able to tell whether the text that you have is generated by AI or not. How is that different from what you guys have created?
Yeah. So I think the traditional approach that has been there for a long time is to train machine learning models to solve a classification problem to distinguish between what is generated by a model versus what was captured in the natural world. And this sort of involves basically training these machine learning models over a very large amount of data. Right? When the earlier generations of large language models and these conversational agents came about, you could sort of make out that, oh, this is a large language model. It has certain tells, it has...
It's not X, it's Y, right? Uses the word 'quietly' a bit too much.
Right. Yeah. Exactly. But as these models have become more and more sophisticated, those tells have become less frequent. And so if you have this machine learning classifier which is trying to distinguish, the classifier's accuracy goes down over time as the generative AI models become better at replicating what they are targeted to achieve.
Okay. Well let's walk through how the watermarking system works then. So let's do it sort of step by step if we can. So first off, if you could, maybe we should start off by how large language models pick the next word in a sentence when it's generating text.
Yes. So, when we are watermarking text, the idea is that these tokens or words are being generated from a large language model, and the large language model essentially has a distribution over which tokens or which words it can generate. So, if I ask you the question, Hannah, what did you have for breakfast today? You can start with the word 'I', or you could start with the word 'Pushmeet', addressing me. And these are the options that you have. And if they're equally valid starting points, you have optionality.
And this is where text watermarking leverages this optionality by seeing what does the key tell us.
Okay, so but that's the point, right? Is that sometimes when you have an option for a word...
Behind the scenes there is a secret key which says bias these words over others.
And then when you look back at all of the text together, if you can see those words appearing over and over and over again, you can be confident that this is AI-generated.
So rather than the words, it's the patterns of those words. So those words specifically you don't need to be repeated, but the patterns is what the detector sort of leverages, how they appear in sort of chains together.
Exactly. Right. Yes. Okay. So you're secretly right. So I mean it's not that secretly, you're sort of talking about it on a podcast. We're going to say... but you have some code running around in the background. Some secret key.
That you can then look at a chunk of text later and say, hang on a second, this looks like it was generated by AI.
Not only this was generated by AI, but if different AI models are using different keys, we can sort of check whether this was generated by this model or this other model.
Okay. So you can tell this is Gemini, this is whatever other model might have... but if you have the key... right, if you have the key, okay, I see. Okay, so but if there's few words that it could be...
So does that mean that some text is unwatermarkable?
Yes. So if you have a very small, a short text snippet, right? Like, what is the capital of France? It is Paris. So there is very little that we can change there, and so that will not be watermarked. That is a limitation of the watermarking approach in this case.
Does all of this mean then that students can't cheat on their essays anymore? Bluntly, I mean, are we now able to just detect anything that's AI-generated?
I think the whole ecosystem as a whole is moving towards watermarking, as you've seen with some of the other labs making their announcements, but that doesn't necessarily mean that they're all going to make their detectors publicly available. And even if you've got a publicly available detector, that can be motivation for people learning how to get around it. In the same way that some of these machine learning classification-based detection systems are using a whole bunch of examples of text of things that have been watermarked, were a detector publicly available and you could just query it over and over and over again, you might be able to work back to making something that gets around that specific configuration of keys that is presented in the public detector.
Got you. So we're not at sort of guaranteed cast-iron way to detect any AI generation whatsoever. It's just we're... this is a big step towards that.
At least not with the method of having a publicly available detection system for every version of keys that is made publicly available.
Just in terms of images then and video, is it the same idea? You have a lot more space to hide things, as you describe.
Yes. So I think in images the approach is different from the text watermarking approach that I was mentioning. In images what happens is we take the image which has not been watermarked and then there is a neural network which looks at that image and modifies it in a very subtle way to incorporate the signal. And this particular model which is incorporating the watermark is being co-trained with a detector neural network, and they're both sort of trying to achieve the following. You get an image which is unwatermarked. The watermark generator injects a very small signal which is imperceptible. So if it is perceptible, we say that's not great, like your users will see it and that degrades the quality of the image. Then in the middle there is an adversarial agent which is going to try to make modifications to the image, might try to shrink the image, zoom in, crop it, rotate it, do many kinds of transformations like add noise and so on, and then the detector has to still detect whether the original image that is coming through is watermarked or not.
Okay. And both of them are trained in tandem to make sure that regardless of what the adversary in the middle has done, the watermark is still detectable. What does this mean for users then?
I think the main thing is it's not just about text, it's about all these different modalities. And if you look at how people are using SynthID, like it was really amazing the last few years as SynthID has been launched. I would open my favorite news website and the reporter would say, well, here's an image that was circulating in the media about this particular house that magically was untouched in the California fires, or this image that came out from the Iran war, or some other sort of topic, and we have run SynthID on it and detected that this was AI-generated.
Right, so all of a sudden...
All of a sudden you can see the AI in places that...
I mean it definitely shouldn't be, basically.
Yes, and so SynthID being used to actually tackle misinformation in real time and at scale.
So then can anyone do what you just did there? Can anyone take an image, some text, some video, run it through SynthID and find out whether it was generated by an AI model?
We have made the detection ability embedded in the Gemini app, for instance. So today if you go to Gemini, if you ask, 'I'm looking at this image, tell me if it is AI-generated,' it can run the SynthID detector and say yes, this is something that was generated by a Google image model and so on. And similar sort of abilities now are available by other companies as well who have adopted our models.
Can each model only detect whether it was generated by that model? Can Gemini only detect Gemini-generated content?
So people who are injecting the watermark into the content, they have to agree to share the key.
Right. If they share the keys, then we have the ability to detect it. Right? So there can be a central service. If you go to the SynthID portal and if you ask something, 'is this AI-generated?', it can look at all the keys that have been given to us by partner institutions and we can detect the AI content for all those partners as well.
Okay, got you. But if a certain lab is like, doesn't want to share their key, then you wouldn't be able to.
