Signal's Whittaker on Privacy in the Age of Data and AI

I know you may not have the data, and Signal's, like, encrypted and everything, but if you could guess how many world leaders and CEOs are texting each other on Signal right now? My guess, because again, our whole deal is not knowing, is most of them. Why wouldn't you? Given the stakes. Given who you're accountable to and what you're accountable to do. Why would you trust that with a breach? Why would you trust that with a company that could be breached? So, all right. I think most people with high-stakes jobs understand that communication privacy is essential to getting business done.

I read the Atlantic story. At my kitchen table, I got up, I got a glass of water. I sat back down and I read it again because WTF was, in fact, my reaction. And then I didn't — I didn't expect we would become the main character because it seemed a pretty, you know, it was an outrageous, operatic kind of dramatic story, but it was ultimately a story about a guy making a mistake with his thumbs and using Signal for, you know, in place of a SCIF. But we did become the main character. There was like one day where there was like three stories above the fold in the New York Times, all of them had the word Signal in it, and I was like, 'Well, my comms person is happy.' But we ended up spending a huge amount of time effectively explaining the basics of end-to-end encryption, the differences, you know, of Signal versus other platforms, how these technologies worked at the most basic levels, because everyone was covering it, and it wasn't just tech journalists. It was, you know, the culture journalists, the finance journalists. Everyone was covering it. Everyone wanted a take.

And the real danger there was that a misunderstanding of how the technology worked would sort of, you know, lead to a story around Signal being flawed that we know has real consequences because we face disinformation campaigns constantly, pretty obviously meant to steer people away from Signal, which is secure, toward less secure platforms that would then allow intercept, would allow surveillance. And so we take that very seriously. So there was a, you know, a number of days where I was awake about 20 hours just repeating sort of, you know, like asking my comms guy, 'Do you have like a metaphor for encryption that might work with this person? And how do we do this?' Ultimately, it turned out well for us, but it was a bit stressful.

We had people we were talking to, you know, cutouts of people who knew them. We were, you know, again, we're not like — the car used for an event drove on the road. It's not really a story about the road, right? But what we wanted to make sure is that sort of the integrity of Signal was clear, that if there were questions about Signal, we were there to answer them and they weren't answered, you know, in strange or inaccurate ways. And that we were very clear that, you know, if Signal goes away, you can't replace it. It has over a decade of legacy. It is open source. Our cryptographic protocol has been reviewed by every significant cryptographer, security engineer. Have you found a bug in our cryptography? You would get hired anywhere. So there is an ecosystem of people who've been examining Signal. And don't forget, Signal came out in the late 2000s. So this is just around the time of the iPhone. This is before WhatsApp. This was not a saturated market, and network effects control markets for communication technologies. So if you got rid of Signal, you can't just snap your fingers, however technologically virtuosic you are, and create an alternative. Because if your friends don't use Signal, if your counterparts don't use Signal, you can't use Signal. So we recognize how precious this infrastructure is. It is the core communication infrastructure for militaries, governments, intelligence agencies, your boardrooms, I assume. And if not, good time, good time to start. And, you know, that's because almost everything else we use collects data, creates data that then can be used to compromise us. And throughout human history, the norm has been private communication that has been eroded quickly, then even more quickly over the past 30 years, via a business model that ultimately relies on monetizing data collected from our interactions, our communications, our social networks, etc. And so, you know, here I am sitting here discussing that.

I'm going to — you know, look, we're a nonprofit. I don't have a board pushing me. So I have the extraordinary luxury of being just really honest. Very relaxing life to lead. And go for it. That's what we want. And the idea is that what we are talking about is math. End-to-end encryption is a mathematical protocol, and either it works for everyone — the person you hate the most in the world — or it doesn't work for anyone. That includes the people you love the most: the human rights workers, the dissidents, the politicians who are trying to do their job in a tricky situation, whatever it is. Right? You cannot undermine encryption just so the good guys can get in. Any backdoor, any vulnerability will be exploited by bad actors, hostile nation states, hackers, you name it. That is the long-standing and clearly established technical consensus. And it doesn't matter if you have an AI magic wand or you burn some sage to the gods of math, it is not changing. Those are the stakes.

Now, I also want to say we are in a golden age of surveillance of data. Law enforcement and others have never had access to more data than exists right now. And the issue really is how do you make use of that? Right? Are you hiring more personnel? Are you, you know, spinning up an AI model to summarize it? But when you actually talk to the practitioners in the field, in law enforcement, oftentimes their issues are like, 'Yeah, we have so much signal, but we can't figure out like, you know,' or 'We have so much data, we have so much noise, we can't figure out where the signal in the noise is because it's coming at us so fast.' We have data brokers, we have pipelines, the different sort of infrastructures. We're sucking up all the SMS data. They have never had it so good there. Then how do you actually find the signal that a bad actor is going to act or who a bad actor is, given that there is so much data?

Well, okay, let's — that's — I've done a lot of AI research. I had the privilege of, you know, I started at Google in 2006, so back when the dinosaurs roamed. We had desktops, like they were, like, chained to our desk. There was no iPhone. Like, I had a BlackBerry. It was new to me. You remember the keyboard? I wish it would come back. So, you know, a completely different world. And so I had like this inside view from back in the day where machine learning was one of a grab bag of techniques that the researchers working on ad optimization would sometimes use. But it was like not that fancy. So now when like AI is kind of a new godhead and we're all just sort of invoking it, right? And what I saw then was the sort of, you know, development of a set of technologies that came into being in relationship to the fact that these companies had developed business models that had huge amounts of data and infrastructure to process and use that data. Right? Because they were indexing search, they were indexing ads, they were supporting large platforms with a lot of content on it. And there was a recognition around the early 2010s throughout industry that, oh my God, deep learning-based approaches that actually date from the late 1980s, early 1990s can do new and really cool things. Not because they are new, but because we now newly have access to huge amounts of data and compute, and then we can parallelize that using gaming and, you know, all of this. And when we do that, we can do really cool things like optimize our engagement-driven social media algorithm much better because we're able to process all of that data. So there was a new utility for old techniques discovered then, and we are still in that paradigm. We are still in the paradigm of large-scale, deep learning-based AI. There are many, many other approaches that could be used.

And, you know, you saw Dario up here, and he's a, you know, he's a very good actor in this space in terms of being willing to call out these harms. But his and Anthropic's focus, and many of these companies, is on building models that increase the sort of efficacy of that paradigm — larger, more data, sort of more comprehensive. And then we see these models implemented as part of so-called agents. Agents are sort of a broad term, but it basically means it has access to data and can act independently based on that data. The issue that we have at Signal is this: the implementations of these agents we are seeing, particularly in the operating system, which is the water that any application swims in, are pretty perilous for our future because they need access to data, they are acting independently based on this access. And so you have this sort of — the marketing pitch for agents: it's going to plan your birthday party for you, and it's going to tell your friends what time your birthday is, coordinate with your friends for your birthday party. Great. You have the magic genie. It's going to live your life and you can go out and sort of walk the promenade in peace. Of course, in order to do that, it needs access to your calendar, and it needs access to your credit card to book a restaurant, and it needs to be able to simulate mouse clicks, access your browser search history, etc. And in this scenario, it would need access to your Signal in order to contact your friends, message them, coordinate time, bringing that all back into a shared context window where you have kind of a data slurry created on your desktop that it's acting on so autonomously. And that is what we've called breaking the blood-brain barrier between the application and the operating system, because our encryption no longer matters. If all you have to do is sort of hijack this context window, you know, that has effectively root permissions running in your operating system to access things like your Signal data. So we have seen, you know, these systems aren't magic, right? These systems are hardware and software with all the limitations there. And we know that these things are extraordinarily vulnerable. And we know that, you know, if you give a system like that sort of root access permissions, it can be, you know, it can be hijacked, it can be undermined. And so I think this is, you know, it's incumbent on the companies that are deploying these, that are wrapping the models in hardware and software, to really recognize how reckless it would be to deploy these things in ways that undermine privacy and security of things like Signal. And so this is why we're talking about it. There are ways to do it better. I think Anthropic is, again, a very good actor here. But the stakes when you're talking about the operating system couldn't be higher because this is — yeah.