Is AI a Threat to Privacy? | Prof G Conversations

You know, if you have an agent running on your operating system, in order to do something like, you know, plan a work dinner, the agent will need access to your calendar. It will need access to your browser, perhaps to look for a restaurant, maybe your credit card or your EA's credit card in order to book that work dinner. All of that becomes a, you know, pretty frightening set of data access points and ultimately a security vulnerability.

I'm in New York City.

Uh, I'm in Europe a lot. I go between Paris and New York. Yeah, we're a small or we spread a lot of jurisdictions.

On a practical level, Signal is the most widely used actually private communications platform. We go out of our way to collect as close to no data as possible. And that's really what sets us apart because we exist in an ecosystem where, for better or for worse, in one way or another, most of the time you make money in tech by collecting and monetizing data. So you collect data about the users of your platform and then you sell access to different types of users based on that data to advertisers, or you collect data and you train your AI model with it, etc., etc., etc. That's kind of the economic engine of tech since the '90s and, you know, maybe before. Signal is obsessed with maintaining the human right to communicate privately, and we have built, you know, an alternative communications platform that does just that. We end up rewriting core pieces of the proverbial stack to enable us to do what is normal, to provide a, you know, basic and easily usable messaging platform in a way that does not collect your data and thus does not put us in a position of being forced to turn it over if we get a subpoena, of having a breach expose your most intimate information, of you violating the contacts that we make with the people who rely on us. So that's in a nutshell. We're also open source, and open source matters here because that means you don't have to trust me. You don't have to like me. You can actually verify that, yeah, the thing that she or anyone says it does is what it does because we can scrutinize the code. We can prove it.

Yeah. Well, the bar, the bar is where it is. Take it.

I mean, I think it's a little bit like, you know, the way skincare ingredients or like, I don't know, gold or something gets invoked, right? Um, we can say it's, you know, both of these have encryption in them, but one has 10% encryption or encryption is only applied to 10% of the data, whereas another is fully encrypted. And so if you look at, say, WhatsApp and Signal, WhatsApp uses Signal's encryption protocol, and this is the gold standard for encryption messaging. It was released in 2013, has stood the test of time, really, uh, advanced the field of, you know, privacy-preserving technology when it was introduced, and that's licensed by WhatsApp. But WhatsApp only applies it to one layer of the WhatsApp layer cake, so to speak. They use it to encrypt the content of your messages. So if I'm texting you like, you know, hey Scott, where are we going to meet at South by Southwest? WhatsApp would not be able to see that. But WhatsApp does not encrypt intimate metadata. And metadata is a fussy little term, but it's, you know, it's actually pretty revealing data. It's who you text. It's who's in your contact list. It's your profile photo. It's when you started texting someone, your therapist, your oncologist, your FBI cutout, whoever it is. That's very revealing data. And then, of course, we're not owned by Meta, which means that, you know, there isn't a bunch of Facebook and Instagram data you could then join that intimate metadata with to make profiles, etc., etc., etc. So Signal is, you know, encrypted up and down the stack. We encrypt the contents of your messages, but we also encrypt your profile photo, your contact list, who is texting whom, who is messaging with whom, who's in your groups. So you can look at our website, signal.org/bigbro, and we work to unseal any subpoena that we are forced to comply with. And what you see there is a long list of requests for data. That's normal. That's what governments assume an average messenger is able to give up. And then you see what we're actually able to give up, which is very close to nothing. Uh, we can confirm that a phone number has an account. We can confirm a handful of other things. But we have gone out of our way to be, you know, unalloyed, you know, 100% encrypted, to use that slightly metaphorically, but to, you know, you get the gist. We really take that extremely seriously. We're not just sprinkling encryption dust on top of a, you know, kind of ultimately non-private infrastructure.

Hi.

There you go.

Yeah, I mean, I think the risks are the flip side of the promises, really. And we actually started talking about this about a year ago when we were seeing things like Microsoft Recall creep into, you know, the product updates, in this case for Windows, uh, and really recognizing that, you know, Signal exists at the application layer, right? Which means that we have to trust the operating system we build on top of iOS or Android or Windows, and we have to trust that the operating system will be a reliable set of tools that we as developers can leverage to ensure that Signal works for the people who rely on us and that, you know, the users of the device can rely on. And our primary concern is that as agents get integrated into the operating systems by these, you know, AI companies, the people who maintain the operating system, and as they get leveraged beyond that in ways that are giving them very pervasive access to your life, it undermines our ability as Signal to guarantee the type of privacy that we guarantee at the application layer. And I'll give you, you know, that may sound a little bit arcane to people who don't, you know, live in these waters with me, but just a quick example. You know, if you have an agent running on your operating system or sort of given, you know, kind of deep access to your file system and other sort of data on your device in order to do something like, you know, plan a work dinner. Well, the agent will need access to your calendar. It will need access to your browser, perhaps to look for a restaurant, maybe your credit card or your EA's credit card in order to book that work dinner. And in a scenario where you are, as you should be, all using Signal, it will also need access to your Signal and your Signal contacts to text them and coordinate dates and times. All of that becomes a, you know, pretty frightening set of data access points and ultimately a security vulnerability that really undermines Signal's ability to protect your privacy with encryption, because instead of, you know, having to break our gold standard encryption algorithm, which has been, you know, tested and sort of mathematically proven to be secure, you just have to leverage the type of access that these pervasive agents are being given into your applications, into your intimate data, in ways that, you know, are just, from a security architecture perspective, very, very insecure. And I'll note that right now almost every agent that we're seeing kind of in the mainstream is relying on large LLM models, models that are too big to run on your device, which means that, you know, ultimately most of this data would need to be sent off your device to a cloud server to be processed for inference, you know, creating another security issue and potentially, you know, placing data in the hands of whatever company is running that agent. So that's, you know, our concern is really coming from a privacy integrity standpoint and from a concern for the people who rely on Signal who, you know, are in danger of having that, you know, having Signal potentially compromised by the introduction of these tools, which can be useful for some things but, you know, also pose this pretty significant risk that isn't getting the kind of attention I believe it deserves.