Reframing “Open Source AI” with Meredith Whittaker: AI Impact Summit 2026

So I think openness should be the floor. Signal models a kind of integrity that we shouldn't think of as novel or exceptional or special. We should think of it as normal and be looking at everyone else like, why the fuck aren't you following us? Because we have a shared, we're all in different places on the same boat, right?

That is a great and very, very big question. And I'm circling it to think about like, where do we start? Because there is an answer to this that'll flash quickly at least. Ultimately interconnected networks, communication networks or the internet require openness at some level. They require shared standards, shared protocols, shared operational norms that allow the network to connect, to be interconnected, to interoperate. And so, there's a layer of this that just the internet at many levels, at the lower levels of the stack at the TCP IP layer and many of these standards, is constituted via open standards and sort of openly scrutable ways of doing things without which it would not be possible. Right. And, you know, that's part of the answer. And then there's a sort of history around open source software and the commercialization of, you know, software as a product, as a service. And then the commercialization of the internet where there was a lot of discussion around, who should be able to build and scrutinize these things. Who should be able to reuse these things? How do we ensure that this is not simply a corporate technology? Or if it is, that these corporations are kind of sharing back in the name of innovation? And I won't go into this sort of politicized history of Richard Stallman versus Eric Raymond in the nineties because no one needs that. Nor do we have time, nor are anyone's attention spans now calibrated to, you know, do much more than, you know, half of what I'm gonna do here. But I think, but...

Yeah, and I think what you have is a term "open source" that was applied specifically to software. There were different licenses that were created around the concept of open source. You know, what can you do with open source software? You can reuse it, you can, you know, tweak it, you can modify it, but if you do modify it, you have to, under some of these licenses, provide those modifications back under the same terms so other people can use it. These were very precise and specific, and endlessly pedantically debated protocols around, you know, what open source meant in very specific contexts. And, you know, that applied to your software, which you could basically, you know, take a binary, take the package and, you know, rerun it on your machine, your set of servers, sort of reuse it, pretty self-explanatory, at least in contrast to what we're talking about with AI. Then we have the reemergence of this deep learning paradigm of AI that is, you know, effectively, at the early 2010s, you had a recognition that, you know, in the context of huge amounts of data, which had sort of been secreted and accrued in the context of the platform, you know, business model. In the context of huge amounts of infrastructure that was already built and calibrated to process and store this data, that early deep learning approaches, approaches from the late eighties, early nineties were really useful and could do new things when those resources were available. So, you know, that kind of marks the advent of this moment in AI. No, it wasn't 2023 when ChatGPT came out. This has actually been a paradigm shift a-brewin' for over about 10 years here. And the key element there, the key novelty that evoked this new AI moment was the presence of concentrated amounts of data that had not been available before. And powerful distributed computational systems to process that data, to train these models and then to perform inference or use these models. So we need to keep that in mind. Then when, and I think it was, I don't know, 2023, 2024, open source AI becomes a thing, right? At least rhetorically, there've been people working on more or less open models, et cetera. And what you see there is effectively what we've called narrative arbitrage, which is evoking the concept of open source, which is, you know, understood in the context of software. People have a rough idea of what that means. It's reusable, it's more democratic. We can all scrutinize it and tweak it. And that does decentralize power, arguably, right? Because you can then sort of access that and do your thing with it. Applying that term to AI models and systems without really explaining that the difference here is that you still need huge amounts of data. You still need huge amounts of labor. You still need huge amounts of infrastructure, both to train and to deploy and use these models. And so, the halo of democratizing AI, reducing concentration of power, increasing scrutability is assumed by people who understand what open source means in the context of software to apply to AI, when in fact those capabilities, those affordances, those virtues, do not cleanly map onto AI and AI systems. And I'll pause here just to say a little something about scrutability. Yes, an open weights model can be scrutable to some extent. But again, these are non-deterministic statistical models. You cannot just cut them open and get a cross section that will tell you why this answer and not that, why this determination and not that. And in the context of things like data poisoning attacks and the extraordinarily significant editorial decisions that go into defining the training corpus and then reinforcement learning with human feedback, which is kind of the labor intensive calibration process that gets these models prepared for business use and prime time. Scrutability doesn't give you the kind of insights in the context of an AI model that it would in the context of a software package. So that's both a too short and too long answer to that question.

I would almost argue that half of the technical terms that I came up using in very precise ways are now just vibes-based evocations of abstractions that the user seems to have either forgotten or never learned. Like we are living in a world where vibes and ideology and even a theological concept of what technology is seems to have inoculated the population zombie virus-like, and, you know, half of tech is just running around praying to a sky god that lives on an Nvidia chip. So with that as the context of my answer, I think, look, we're in a situation given the history of the commercialization of the internet and the way that historical investments and how that happened and the neoliberal zeitgeist that turned it over to the private market without understanding that communication networks are naturally monopolistic and thus enabled a monopoly to form in the US that spread. In terms of setting the standards, the terms and the infrastructural dominance of these platform companies in the US that sort of emerged out of the commercialization of the internet. We're in a situation in which the poles of AI power exist, you know, in the US. And then because China resisted the market reach of these US firms and sort of contained their market, built their own, had a large enough population to support that and kind of, you know, top-down state control to mandate it. You have two poles in AI and this is, you know, China and the US. This is well known. This is not a secret. Right. And you have other countries that have, or jurisdictions that have some capabilities, right? You have talents or you might have data from various vertically integrated industries that collect data. You might have a handful of supercomputers. You have bits and bobs of the resources needed to create and deploy AI. But you don't have the full suite. You don't have hyperscaler infrastructure that spans the globe, which right now, about three companies have about 70% of the infrastructure market, the cloud market. Those are Amazon, Google, and Microsoft. You don't have the distribution networks, whether that be licensing cloud APIs through your infrastructure that give people access to your AI models and let you make money off of them, or integrating these models into your social media platforms, which is a vehicle both for distribution of AI, exposing that to people who they call users, or the continual collection and creation of more data purportedly about these people and the context of their use of your service that can be used to feed the AI model and continually refresh it. And you don't have the legacy businesses, the marketplaces, the platforms, the advertising networks that can serve to cross-subsidize the extraordinarily expensive infrastructure and compute and energy requirements needed to train these models. So there is an anxiety, and rightly so, among anyone who's not a major AI company or the US or Chinese state around where they stand in relationship to this powerful, new, but also old set of technologies and a desire to figure out how you can have a piece of that as well. And I think that is matched then with, you know, what I've called intellectual shame, a kind of hesitation, particularly among powerful men that I see constantly, where the same people who would rip apart a bad P&L document in a board meeting with incisive questions fail to ask a single question when presented with fantastical claims about AI and its capabilities. I think largely because there is a fear of looking non-technical, looking ignorant, looking behind the curve. There's sort of a FOMO-driven juggernaut that is dictating the need to adopt AI. We don't know where, we don't know how, we're not sure how we measure it, but if we don't have an AI strategy, we are behind. So that sets the table. So why are we seeing openness invoked? Well, again, I think in the context of this failure to ask the right questions, in the context of this deep anxiety, in the context of a handful of US-based companies, you know, Chinese as well, but US is really dominant here, at least in the west, who have control over infrastructure, who have control over platforms that are dominating our information ecosystem and who increasingly have a control over what I'll call kind of bureaucratic decision-making via owning the models that are then licensed, you know, out to governments and others via API contracts or what have you. This sort of AI as a service. There's anxiety about sovereignty, there's anxiety about like, what is a state if these things are sort of ceded to centralized corporate actors. How do we get a piece of this? How do we maintain our status, our parity with these states that do have these, the two states that do have these. Terms like openness sound pretty appealing, right? Well, if it's open, if it's democratic, if it kind of looks and smells like open source software, that means we can have a piece as well. That means we can take an open model and we can use it. And I think there's two things going on. One, there's probably some honest misunderstanding because God knows no one's asking questions. And like, there's a lack of grounded material basis for this, for a lot of the claims that are being made about AI in government and elsewhere. And I think there is just a, I would say kind of weak political opportunism where you don't want to look like a loser. You don't want to look like you don't have a solution to this. You'd like to spend the remainder of your time in office or your time in power being a person who won, not a person who lost. So quickly trading on a little bit of rhetorical exaggeration may feel or be in the short term more advantageous for your own political positioning, allow you to sidle up next to those in power and pretend you're with the party as opposed to going direct with them and saying, this form of openness is not actually providing the affordances that we need, this form of sovereignty is actually not sovereign, this form of independence is actually not independent, et cetera.

Yeah.

I wanna separate those points because I don't think the large-scale, bigger-is-better, deep learning approach to AI is realistically separable from those big players. That's just, those are network effects, economies of scale, distribution platforms, and the entanglement between the state and the AI industry, both of which recognize these technologies as extraordinarily important tools of geopolitical dominance. So all of that tells me that, you know, we're not gonna have a one weird technical trick that just sort of unravels that. We're talking about political economic forces that are, even if we had technologies that were smaller, more nimble and better, there would still be a deep vested interest in maintaining the bigger-is-better narrative, because it is extraordinarily advantageous as a moat, a narrative moat around these significant political and economic advantages. However, I am not arguing, and I wanna make this totally clear, that the models and the documentation that are surrounding particularly the most open forms of AI are bad in and of themselves. What I'm saying is they don't do the things many of their boosters or their adherents seem to think that they do, and that claiming that they do these things confuses us and distracts us from the type of solutions that we need. Being able to reuse a model can be very, very useful. Being able to, you know, use it on-prem in a given industrial setting that isn't sending data to a cloud provider may be necessary to maintain confidentiality, being able to look at an open weights model and understand a little bit better what might need to be tweaked for our use case. You know, being able to examine data sheets or the dataset that was used for training or other affordances is very useful and being able to kind of play with that and extend it is also very useful. All of those things are great. What they aren't is democratizing in any meaningful sense. And so, you know, yes, by all means, please. But what I think we need to do as people who are, you know, technologically responsible and sort of grounded in a material understanding, is disentangle the rhetorical halo, which is misplaced, from what these things actually do, what the positive benefits of those might be in different contexts, and what they fail to do and we thus must solve otherwise.

I wanna pick that apart a little bit.

Because what I'm saying about scale is that it will never be, you know, that paradigm is not democratizable.

You can't compete on those terms.

What I'm not saying is that a given large-scale model that is open-sourced and usable, open-sourced in the limitations that we talked about and usable by others, doesn't have utility.

But...

What the utility it doesn't have is a utility to level the playing field vis-a-vis AI dominance. The utility it could have is you can use it on-prem, you can extend it, you can do X, Y, Z. But it is not democratizing, it doesn't level the playing field, and it doesn't address the significant monopolistic advantages that have accrued to US firms on the one hand, and Chinese firms on the other, and cannot be undermined simply by sort of innovation or open-washing.

Yes. I think there's sort of two things. Is open source AI models useful? They can be useful. Are they a challenge to the concentrated power of these companies? No. Even if you created an open source AI model completely outside that, right, on a supercomputer with data, you, I don't know, we've always got somewhere, but like ultimately, you know, all of that, it still wouldn't be a challenge because it doesn't challenge the actual resources that are currently concentrated that in combination are necessary, that include distribution networks, that include economies of scale, that include entrenched reach, that include the ability to define the tooling and the standards and yada, yada, yada, yada, yada.

I mean, in the context of Signal, my answer is always pragmatic, right? It's not like, oh, we disagree with this existing, so we will ignore it, right? Yeah. We do use a small, open source AI model, not an LLM. It's small, but we use a model that is, you know, meant for face detection and we use this to enable our blur faces feature. So when you take a picture of crowds, you can just sort of like push blur faces, it'll block out people's faces. So if you post on social media, it's not scraped up by a Clearview database that is very clearly a privacy-preserving use: everything stays on device, it's not sending things out to, you know, inference via some LLM on a cloud server, et cetera, et cetera.

It's been there for over five years now, and you do it in the media editor, you can find the setting.

So that's a, like, yeah, of course we'd use that. That helps people. That's totally aligned with our mission. It's not a purist stance. I think with these large, particularly LLM models, my reflexive answer is one, there aren't models small enough to run on device, and that's a problem. You know, we're, and then we get into what WhatsApp's doing where you're kind of like splitting the baby around, like what we're sending data offsite to be processed here, but we're saying that it's end-to-encrypted because we're just counting our server as an end, right? Like, that gets really messy and frankly dishonest. I think my other answer to that is like, I don't think people want AI in their clean, crisp, beautiful, cool messaging app. Like, there is not a need...