A Conversation with Meredith Whittaker

Yeah. Well, first, I'm really delighted to be here. Thank you so much. And thank you to Stacey and Olivia. You made it really easy on my team. There's a lot of ways to approach that question. I think, you know, Signal came out of an aspirational open source project that has actually been around since—we can date it to kind of the mid-2000s when we were beginning to see the growth of the commercial internet and the commercial—you know, the commercial internet as a driver of communications, whether it's Gmail sort of launching at the end of the, you know, 2007, maybe 2006, and a recognition among the technical community that these were not being built for privacy and that these were not being built with, you know, the human right for privacy in mind by the companies that were positioned to offer these as consumer services. And there's a long history here around sort of how the business model of acquiring data to build models that help you classify different people who use your services and then sell access to those people to advertisers and others sort of developed out of the '90s. And I might sidebar for a second, if you don't mind, to, like, a historical segue here. But I think it's important to recognize, like, there's some what I've called the original sins of the growth of the commercial internet that I believe should be more widely understood when we try to diagnose, like, what's going on and how do we solve this myriad of problems that seems pretty overwhelming and, you know, kind of related to tech but hard to pin down as one thing or another.

You know, so you had in the '90s this, you know, imperative to commercialize network computation and, you know, so you roll into the '90s. You have stagflation, you have diminishing manufacturing, you have a bad economy, and a need, particularly from the Clinton campaign and then the Clinton administration, to find something that would heal this ill, right, and commercial—you know, commercializing network computation, the internet, was kind of the thing they landed on for good reason it turns out. It was, you know, fairly effective. But this was—you know, there was a quote that I found in some of the literature that, you know, it was kind of referred to as the New Deal without the socialism, right? How do we—you know, how do we revive the economy by turning over infrastructure that had been built by the government—you know, government and for academic networks to the private sector to build businesses on, to, you know, revive the economy and, you know, you had this playing out across the '90s and this is going to relate to some of the debates around encryption that are still ongoing around, you know, how do we have strong privacy while also giving access to law enforcement. That really came to a head in the '90s. But throughout there was sort of this question of, OK, well, what are the regulatory rules of the road for these—you know, commercial internet, right? How do we know how to play? How do we, you know, kind of build these, you know, businesses around that? There was a lot of lobbying. There was a lot of lobbying from the advertising industry in particular because they didn't want to lose another platform. They had magazines. They had TV. They didn't want some other media entity coming into the fore and then, you know, not having advertising. And there were a lot of proposals. There was a public model. There were other ways of potentially funding the commercial internet. Advertising was the one that in 1996 in the framework for, you know, I think the information superhighway or whatever the bill was called ultimately the advertising as the economic engine of the commercial internet was sort of, you know, set down as the—you know, the regulatory paradigm, in addition to a decision that was also sort of heavily lobbied for by industry to put no privacy restrictions on that. So, you know, commercial internet companies, commercial companies, could ultimately collect data, could surveille, to put it in a more cynical term, as much as they wanted, much more than government. There were restrictions on government. Commercial entities could collect as much data as possible. So those two decisions in—you know, conjoined helped—have a lot of explanatory power for how we got here because, of course, if advertising is powering the business you want to know more and more and more about your customer, about who you're selling to, and the floodgates are open. There is no restriction on how you do that. So your value add is we know more about the micro demographic of, like, you know, women who live in New York with curly hair who do yoga. We can get you to that market and we can, you know, help you sell things. We can target them for consumer ads or we can target them for—you know, shift their political opinion, right? And so, you know, you can—

Yeah. Well, because—I think this is a great question and I think it ultimately gets back to, like, we're talking about communication networks, which are a very particular thing, and ultimately when you're talking about communication networks the first thing you're talking about the real value is human relationships. The thing I want to do when I pick up Signal is message a person. It's, like, you know, tell my mom hi. Tell my staff, you know, this is happening. I don't want to use a platform, and if nobody uses Signal it doesn't matter how idealistically pure I am about my belief in privacy. I have a choice between having no friends, no relationships, being excluded from my social context, from my—you know, like, my ability to live a full, rich life or switching to WhatsApp where they all are, right? So the network effect is powerful and this is—you know, this is why communication networks have had this sort of special place even in hyper neoliberal contexts as regulated monopolies because we understand that having eighteen different standards for telephony means you have to have eighteen different telephones on your wall and if there's another, you know, competitor that springs up you have to have nineteen and then you have to commit all of your friends to go out and buy, like, telephone number nineteen and remove their family portrait and put that up on the wall. So it actually is, I think, very difficult to get people to switch because what you're asking them to do is convince an entire network of friends and relationships—

—complex relationships to, you know, switch over to something else. And so that's the simple answer. People are switching, by the way. Our numbers are going up. I think the blood-level awareness this is not a theoretical concern but, you know, there is real power that those who have intimate information about you can wield in ways that could harm you if that information is leaked, is, you know, seeping into our daily lives. But, you know, nonetheless, it's a real struggle.

I mean, look, it's a pain to be a nonprofit and run, like, an at-scale tech. We have to, you know, look, feel, do like any other tech company. Like, when iOS rolls out a new version we have to, you know, make sure our iOS team is prepared. We've tested that. We're running on that platform. If something breaks there's a guy with a pager who has to get up in, like, 2:00 a.m. and fix that thing because we're critical infrastructure across the globe and it has to work in real time always. So it's not—there's nothing virtuous about that form that we sort of chose to be a sweet little charity. The issue goes back to that original sin, right. The business model in tech is collecting and monetizing data in one way or another, whether you're selling ads or training your LLM or what have you or producing kind of hardware and software that supports that overall ecosystem. There isn't really a business model for providing the type of rigorous, like, rights-preserving privacy that Signal provides. And so you can imagine having, you know, a for-profit model and then, you know, having, you know, some guy on my board or my, you know, investors come back and be, like, look, you're not breaking even. This weird licensing thing you have to try to get people to use it, like, it's not making the kind of money we expected in tech and it's not actually sort of commensurate with the overall ecosystem. So you're going to have to undermine that a little bit. You're going to have to do—you know, like, WhatsApp has WhatsApp for business that's less encrypted. That's where their revenue comes from, right? There are a number of, you know, kind of ways you could sort of carve out little reductions that collects the metadata and model that. OK, use homomorphic encryption but not all—whatever it is. But you would also—the pressure on you from that objective function, which is revenue and growth and, you know, figure out your Series B, whatever it is, is to reduce privacy because that's where profit also comes from.

I mean, user growth is a really clear, easily numerated metric. We also—you know, we look at things like who's adopting our open source protocol, our libraries. You know, we do have a social mission which involves creating and innovating novel privacy-preserving tech. We often have to rewrite various core libraries or other components of, like, the stack in order to do what we do but do it in a privacy-preserving way because, like, the norms and the dependencies around us, again, assume that sort of data-centric business model. So we—you know, we think of that as almost being a keystone species in tech as kind of holding that bar for the rest of the ecosystem and showing that it—you know, it can be done. It's simply that the economic imperatives that have shaped the ecosystem we live in don't, you know, prioritize that.

I don't—I mean, like, Europe also has—you know, Austria is sort of the mixing pot of, like, you know, data collection and surveillance. I think, you know, you have—often what I see in Europe under the guise of privacy is maybe a bit of protectionism and often what I see in the U.S. under the guise of security is, you know, I don't know, just a desire for more data. But if you kind of dig into GDPR it's not very well-enforced. It's not actually sort of protecting privacy. A cookie banner is just like another thing you have to click on. So I think that dichotomy isn't as robust as it might seem on the surface. I do think, again, you know, if you read back in the archives and, you know, some of the link-letter, you know, sort of documentation around that period you see that, you know, there was a very clear motive on the part of the U.S. state to make sure that U.S. standards and the technology that was coming out of this initiative to commercialize the internet in the '90s was adopted globally because, again, you want to be the one who is sort of creating the software everyone adopts and one of the major reasons that encryption was liberalized—like, the reason Signal can create strong encryption as strong as we want without having to have an export license under—you know, that is approved by the U.S. government is because there was a recognition that we can't sell software overseas if we can't ship encryption because no one wants to buy weak privacy from a U.S. firm. And so we're not going to be able to actually dominate this market unless we enable strong encryption, you know, across the board unless we liberalize encryption, and we're not going to be able to dominate this market if we don't have people adopting U.S. standards, adopting U.S. technology. So when it comes to sort of the explanation for why more innovation in the U.S. I think there are post-war imperatives. There was a huge amount of investment in science after World War II. You know, you have, you know, DARPA, the ARPANET, all of that, you know, kind of Cold War legacy and then you have a very clear and calculated strategy that's sort of laid out in the '80s and '90s, you know, into the 2000s to ensure that U.S. tech was adopted around the globe as a standard with the sort of, you know, commercial hubs existing in this country and we see now, you know, there's three companies—Amazon, Microsoft, Google—that have 70 percent of the cloud market, right? You know, most of the standards bodies have a lot of U.S. folks on that, you know, kind of U.S. industry represented. You know, it was a very successful—

Yeah.

Yeah. Like, basically, what about the bad guys who—

—seek shelter?

Yeah. Yeah. Yeah. What about the bad guys? I'm going to give a kind of flip answer and then I'll get into it because, like, obviously, we don't like the bad guys. Obviously, these are serious issues dealt with by serious people. This is not a sort of blasé—you know, you can't be blasé about it. The reality of encryption, the math that ensures we can have privacy on communication networks, is that it has to work for everyone or it doesn't work for anyone. The person you hate the most—think of the villain in your extended universe—has to be able to use Signal or the person you love the most cannot benefit from it. Now, that sounds a little blasé, right? I'm, like, you know, but how do we think about that, right, because, you know, we need a system that provides that level of privacy in a world where almost everything else is enabling data collection, where every day you see a data breach or a Salt Typhoon decimated American privacy with Vance and Trump as targets. Like, imagine—you know, Chinese actors know when Vance started going to an oncology clinic if he did. They know when their, you know, asset walked into the FBI office. Why? Because Chinese-affiliated hackers were able to breach the interception points. Those are the back doors, the undermining of the integrity of the network of U.S. telecommunications networks. That is probably still ongoing because once you have an infiltration of a network like that it's very difficult to prove that it has been sort of excised. And there's a New York Times story on it today. It's now I think eighty companies around the world were targeted by this, et cetera, et cetera. So, like, you know, I can kind of go into the sturm und drang of how serious this is but, ultimately, how do you preserve meaningful, robust privacy that is not subject to those kind of, you know, catastrophic hacks, like, life or death infiltrations while thinking about the real harms that can occur. And I think what we do is we—like, in a sense, the way Signal deals with that is the absence of certain things offered on the platform. So we don't have channels for broadcast, right? You can't go viral on Signal. It's not like Telegram or WhatsApp where you have thousands of people where you can just sort of, you know, disseminate sort of misinformation or dangerous information. It's not exactly privacy but it is one of the major concerns with these networks. We don't have a find a friend directory, right? I can't go on WhatsApp and be, like, all the people on Signal and, like, find someone. I have to encounter them on another platform or otherwise exchange information. We cap the size of our groups at a thousand, which is very frustrating to a lot of people but it also limits the ability to sort of have, you know, mass—

Yeah. But anything we build we build with that bar of privacy in mind.

There's just many things we don't build because we—you know, and we leave growth on the table. We leave, you know, a lot of things that, again, shareholders would be pushing us to prioritize because we don't think we can do it safely.

I mean, look, there is a lot of utility in sort of these large models, right? There's a lot that can be done by, you know, processing huge amounts of data, finding patterns in it, and then sort of applying that distribution to novel data, right? Like, we see that. There's no question around that. I think, you know, where I am focused now, you know, I don't have a—like, ultimately the real answer there is, like, who gets to decide how it's used, who gets to decide how those benefits are distributed. There's sort of—you know, it's more of a kind of social and political question in my view than it is a technical question, right? I am concerned about the concentration of the AI industry. I think it is, you know—

Not in AI, and this is the issue. You know, again, I'm going to caveat that the thing we are calling AI right now, because that term has been applied to a lot of various technologies over, you know, its eighty-year history—the thing we are calling AI right now is sort of the, you know, deep learning approach. It is the, you know, kind of scale logic so the bigger is better approach. It relies on massive amounts of infrastructure. There's no secret about the trillions of dollars being poured into data centers that are—you know, where the chips are moving so fast they're, like, building half a data center and then demolishing it and then building another one to keep up with demand. The energy use is incredible and the requirement for data in order to train and then continue to calibrate these models is enormous. And so what you're seeing, you know, and what I've sort of argued in different other—you know, elsewhere is that, you know, the AI we have today is ultimately built on the back of that kind of platform, you know, surveillance advertising business model because it's the companies that already had, you know, three things, right? They had infrastructure already built out, the data centers to collect and process data for, you know, creating ad models, selling ads. They had mass distribution platforms which would incentivize kind of integrating AI and also collect more data, right? Every time I look at Facebook they're collecting a huge amount of data that's not just sort of free and out there and it's incentivizing people to, like, deposit their data—their Flickr photos, their comments, everything—creating this sort of, you know, concentrated data source. And then they had access to market either through cloud infrastructure so they can license APIs like Microsoft does with, you know, the GPT APIs or they have platforms they can integrate AI models into to help calibrate advertisements or, you know, otherwise sort of integrate as features into user-facing platforms. So it's—you know, the resources required to create from start to finish large-scale AI, which is the thing we're talking about, are concentrated in the hands of, you know, five to seven companies, largely based in the U.S. because of the history I've over—you know, outlined, and in China because China was protectionist enough not to, you know, and had a big enough market to kind of create a homegrown version. Now, there are things you can do on the edges of that. There are really cool startups that sort of, you know, license or adopt open source models but they're missing key ingredients. So they're—you know, they're running on AWS or Azure, you know, Amazon or Microsoft. They are, you know, kind of scraping data and then, you know, not having necessarily that renewable data source or they are, you know, using open source models. Like, it's not—you know, the only entities that have sort of the full set of tools and the ability to bring them to market are these large companies. So it's not a garage startup situation, which doesn't mean it's not interesting. I have argued that, you know, there's a big incentive for these companies then to continue the narrative that bigger equals better. That's not actually empirically proven. A lot of times that is based on measuring the performance of models against fairly narrow benchmark data sets that don't actually reflect the use of those models in context whether it be a classroom or, you know, a hospital room, and when you begin to measure models and systems built around models in real life context you often get very different results and so I think, you know, in which smaller models more, like, you know, kind of or even non-AI models just statistical techniques work better in practice. And so I do think there are arguments if we can sort of get back down to the basic scientific empirics of this field for smaller models that would then produce a more interesting, innovative ecosystem of startups and innovation. But, you know, there's a sort of full nexus of political economic factors that have led us to where we are today.

I mean, that's a—I don't have an answer to that. I would actually—like, I'm, like, looking at the room. I'm, like, do you have an answer to that? (Laughter.)

Yeah, Mike. I think, you know, all I can do is I can, like, focus on the reality of the technical systems as I understand them, right? I know that security and privacy are important. Those are, you know, for government, for our economies, for our infrastructures. I know that it is dangerous to have, you know, the type of, you know, single points of failure that exists when you depend on five companies and I don't—you know, how many of you all are familiar with the CrowdStrike outage at Microsoft, right? You know, I'm going to say, like, unnamed but I was talking to a world leader who was in a war room for twelve hours wondering which planes were going to crash because their aircraft traffic control system was taken offline by that outage. You know, I wasn't there. You know, there were traffic lights. There were, you know, hospitals that went down. You know, and my understanding is there was a kernel update that went wrong that was sort of pushed by a team, you know, that led to that outage, right, so, you know, because everyone is dependent on that software. And so I think there are—like, there's a need to really get real about fundamental research, about investing in core infrastructures that are not dependent on those, you know, same incentives. You know, hardened operating systems, open source technology that can be validated, like, formally verified secure libraries that are used in military and for infrastructural context, auditing, red teaming. Like, real security research and privacy research where we're creating technologies that are open source and can be used and sort of, you know, built on by the community. And that needs to be—you know, we can't count on volunteers and good-natured folks for that. We need people who are, you know, incentivized to do that and we need to make, you know, that—like, I want every grad student to be, you know, thrilled to be working on those projects. I want them valorized and excited. I want them, you know, like, that's where I want, you know, the Turing award to go.

I don't—I think there is just so much hype right now and I, frankly, think that kind of—you know, the AGI narrative has gotten pretty overblown. You know, AI itself is not a real technical term of art. It's a term that was created in the '50s that actually applied to a very different paradigm of AI at the time. It was like symbolic systems, which was oppositional to the sort of, you know, connectionist school of AI which is the neural networks we now have, right? But the brand has traveled over decades and is now applying to, you know, large-scale connectionist neural networks, which is the paradigm we're in, this deep learning approach, right? So I think already AI is, like, kind of a tagline, kind of a marketing term. It's not a technical term of art. And then you apply, like, artificial general intelligence, which is just like a vibe. It's, like, really, really AI, and then you have that being sort of inoculated into a public understanding of this technology that is also, I think—like, there's something—like, we haven't talked about the chatbots and, like, you know, there's a real pathos to these stories that are coming out now around people being sort of coaxed and cajoled and, you know, like, soothed and, like, the emotional—I don't know. You know, the ELIZA effect is what we called it during Weizenbaum's time which was, you know, he created—he was an early AI scientist in the 1970s who created this early chatbot. It was like a command line tool. It was meant to synthesize therapeutic speech and he would say, like, my mother, she's—you know, like, I'm sad and it would say, tell me more, right. And it was just sort of, like, you know, really rudimentary rule-based system. But Joseph Weizenbaum, the scientist who created it, you know, in a book that he wrote reflecting on this that he was really disturbed because people were immediately developing an emotional connection to this extremely rudimentary chatbot. So we have known for years that, you know, conversational speech and simulate—like, these chatbots play on a deep emotional need on something. You know, maybe it's hardwired, like, we, you know, grow in relationship to each other, talking to people. We want to be heard. We want to be loved. We want to be, you know, in connection with a sentient entity and we project that. You know, it's like a pareidolia, that, you know, when you see a face in a cloud, right, we, like, make up a friend, and then to have these very, very sophisticated instances that are kind of, you know, being tuned to play on that in some way whose objective function is, like, flatter or, you know, make distressed, you know, I think, like, to me, that is deeply, deeply concerning. And then you think about the way the AGI narrative, like, this is artificial general intelligence, this is kind of a godhead, even though that's not augured in any scientific reality even though, you know, there's no way to prove that because it's not a technical term—there's no benchmark or evaluation for that—kind of inoculates a population who is having the sort of, like, chatbot effect play on some very deep human needs around human connection and, you know, kind of understanding. I think, like, anyway, has the understanding gotten better? No, I think it has gotten worse and I'm disturbed by the, you know, kind of nexus of—

I mean, it's definitely—like, the toward where is always the question but, like, it's astounding what these things can do. There's no—like, we don't want to minimize that. It is, you know, technical progress. The progress in parallel computing has been astounding. The progress in sort of, you know, networking infrastructure and data centers that are sort of, you know, connecting many different servers but—

Yeah. I would—I think that's fair enough.

Oh, my God. I just got to say I read that story twice before it sunk into the bedrock of my understanding because I did not—it was like a soap opera and then it was just a blur because we're a small team. So it's, like, me and a couple other people calling every journalist we could just to make sure that the facts were as clear as possible because it is so hard to be on the back foot having to explain a misunderstanding about a complex technical system when there's that much froth around it. And it was, like, there were some Hail Marys but we're glad we made it out. (Laughs.)