This panel we're going to explore, as Jose mentioned, the intersection of open source, open source AI and sovereignty. These are big topics. So, we're going to hopefully drill into some important pieces of this. What I'd like to start by doing is ask each of us to talk about an aspect of sovereignty that they think is important. And then we'll tie it to open source and be specific about it, hopefully grounding it in some real-world examples and projects that each of us is working on.
I can go first.
Okay. There are a lot of dimensions of sovereignty that I think are very important. There are a lot of ways to look at it and I'm tempted to go to the level of cultural or national or even industry or company level sovereignty. But I actually want to go to the personal level because I care a lot about how AI is going to interact with us personally, about systems that learn us extremely well, systems that we rely on as friends, almost as colleagues, as very close collaborators as systems improve. Now who controls those systems? Where does the knowledge, where does the learning of those systems reside? Who controls it? And to me that aspect of sovereignty is really important and I think it's a step change from how you can argue that today with internet platforms there's a lot that's known and collected about individuals, and that's true, but I think the level that AI will take us to is much beyond that. So I think this aspect of personal sovereignty is really important to me and I think a lot of people honestly.
Maybe I'll do the 'I know it when I see it' version of sovereignty for a second. So the way I think about sovereignty is basically: can you leave? And so if you think about that on multiple different levels, like when it comes to personal sovereignty like Anthony was mentioning, can you have your user context in such a way that you can then leave one of the platforms and go use another one and not have to retrain it from zero again? From the industry side, I think of it as: can you leave? Right now if you were to go build a company and you're going to use a cloud service provider, you probably wouldn't write code directly to AWS. You'd think about using something like Terraform or something to provide an abstraction layer so that when the sales rep from AWS gives you a call to try to screw you on prices, you can negotiate against them and say, 'I'm just going to pick up and leave.' So that's maybe sovereignty on that side. And then on the national side, I think about it as: if a bunch of CEOs in San Francisco decide to change the rules, can you leave? And so if you can't leave in these situations, then you're a renter, not an owner. And then I think that you've failed the sovereignty test in some way. So that's my definition from the 'you know it when you see it.'
So I think the main issue related to sovereignty, at least in the long term, is the fact that our entire digital diet is going to be mediated by AI assistants. We're not going to use search engines or things like that or Wikipedia, we're just going to ask our AI assistant for the answer to whatever question we have. And in the not too distant future, at least if the plans of my former colleagues at Meta succeed, we'll all be wearing smart glasses and have those AI assistants see what we see, hear what we hear, and be with us at all times, be our best digital friend, know everything about us, which I don't think is a problem. Actually, this is not the main problem. The main problem is that if those systems are provided by three companies on the west coast of the US or a handful of companies in China, we're in big trouble when it comes to linguistic diversity, cultural diversity, political bias, value systems, and everything else. And so we're going to need access to a widely diverse set of AI assistants for the same reason we need diversity in the press. This is going to be as important as having a free press. And there's a lot of discourse about this at the moment about who is practicing censorship? Is it the US or Europe? The US is accusing European governments of practicing censorship, and the other way around. And right now the data is that the country that practices censorship the most is actually the US, not Europe. But this is going to become extremely important for the preservation of democracy and everything else, and linguistic diversity, which is a very important problem in India, Indonesia, and a few other countries. So that's the biggest danger I see with the way things are going at the moment where frontier proprietary models are making progress and open source models are following kind of closely and really continuously. So the gap is actually not that big but all the best open source systems are all Chinese and that's a huge issue.
So maybe I'm jumping ahead, but I have a recommendation that I've been trying to socialize for the last two or three years. I tried to promote it within Meta and got absolutely nowhere. And the idea is to form some sort of bottom-up international consortium where different players from around the world, countries, regions, private entities, whatever, contribute to training a global model whose result would be as if it had been trained on the totality of all the data available everywhere, not necessarily publicly. There's a huge amount of data, cultural data in particular, that is either not available publicly or not even available digitally in any form, so it needs to be digitized. So a country like India or France or Vietnam or Korea can digitize all of their content and then contribute to training a global model that would essentially become the repository of all human knowledge, but they don't need to exchange or give away the data. They can keep sovereignty on the data and the way they contribute to training this global model is by changing parameter vectors. So if everyone agrees on a particular architecture or micro-architecture and then they train on a common data set that is available to everyone but simultaneously they train also on their local data that only they have access to, and then every few seconds, minutes, hours they exchange their parameter vector with some sort of central server and then everyone has a term in their loss function that attracts to the mean, the average parameter. Then eventually this is kind of a form of federated learning. Eventually this converges towards a consensus model that is as good as if it had been trained on the totality of all the data but the data has not been exchanged. This is the way to beat proprietary systems. And if needs be we can convince governments to establish regulations to make that more easy so that, for example, Brazil was thinking of establishing a tax on data. If you train an LLM using Brazilian data and you want to deploy your LLM commercially closed source in Brazil, you have to pay a fee, a licensing fee of some kind, like a tax. And I told the people in Brazil who are involved in this, I told them, waive the fee if it's open source. You have immediately a big incentive to contribute to an open source system. So open source can beat proprietary approaches because of access to local data and an international consortium.
That's a very interesting proposal. I would love to come back to that. What I wanted to dovetail into, you already did which is great, which is the intersection of open source and what parts of the open source stack in particular are most important for AI sovereignty in the ways that you've defined them and the way I defined it. So clearly models are very important. The data that goes into them, the way they're trained, if we can have convergence, if we can have diversity of data. Rafie, what else do you think is important?
I mean, I think about a few different things, but models are obviously the crux of it, but it's also all the unsexy stuff that goes around it that's actually really important as well. I think a lot about what regular everyday developers do and how they use it. And so the constellation of open source software out there is actually quite vibrant of all the different pieces that you might need in order to assemble what a stack looks like. But it's just really hard to do. It's just kind of a pain if you're a normal everyday developer. And I'm guilty of this too, like when I do prototyping I just call the OpenAI API because it's so easy to go do in comparison. And so I think that open source needs to figure out how to deliver a product which is just as easy to use, if not more joyful, than just calling a closed source system, and then we can start having a conversation about values on top of that because what developers end up choosing as their entry point sort of has a trickle-down effect to the rest of us. So I think there's one thing there which is the unsexy developer experience around the model, the infrastructure needed, all the way to the software layers, the API, and then of course there's data on top of that. So one of the things that we were doing at Mozilla is that we've created a two-sided marketplace for ethical and provenance-based data so that every radio station in the world could actually try to license their transcripts and audio data in a way that they can make money and preserve what their licenses look like for all these things. So I think those two things are some of the most important things that we need to solve as a community in order to pull this off.
Yeah, that's very interesting. So you've highlighted kind of two really key aspects I think of kind of related but somewhat orthogonal maybe. So functionally, can we converge to create a model that is not only frontier capability but broadly represented, and Yann you've suggested a mechanism to do that. Rafie, you've highlighted the really important part, which is like, okay, that's great, but you still need to be able to use it. It needs to be on par or better in terms of developer experience. So on that piece, let me just go back to you for just a second. What do you think we can do about it? I mean, the AI Alliance and organizations like ours, we can do things about it. What do you think we should do?
I mean, like I said, I think there's a lot of vibrancy on the open source stack already. And so I think what's missing in some ways is that we need the right metaphor. I think we did this in the early days of the web, right? We created the LAMP stack. We created ways to actually organize with open interfaces, putting different layers of the stack together. Everyone can contribute in the ways that they wanted to contribute. We can pull it together to be the way we want to serve information onto the web. I think we can do the same thing if we as a community just decide that we want to go do that. And I think there are people doing it. I'm not saying that no one's actually doing this yet. But I think what we need to figure out is what are all the different layers and how we're actually going to pull that off starting from the API calls to the guardrails and on the inference side, and there's all equivalent stuff that we need to do on the training side.
Okay. But especially at the actual AI layers, so at the model level, at what the models interact with and so on. Yann, I want to go back to you for a minute to the consortium idea and the convergent model, or I don't know if you want to call it that. So do you think that that approach where you're converging many people with many different data sets into a single model or model family, do you think that's a preferred approach versus enabling composable small diverse sets of models at a regional or company level or otherwise?
I mean I think those two things are not incompatible. You could have a global model that ends up being a mixture of experts of some kind. But the point is that because it's open source, it's easily customizable and distillable. So I think that could be the trick. But to balance on what Rafie just said, historically the advantage of open source has been in infrastructure software rather than end user applications. And so open source has never been really good at kind of easy user interface and stuff like that. Although some open source companies of course have kind of made a point of doing this, but the pressure from the market for something to become open source is stronger if that thing ends up being a common platform. So that's what happened with Linux. To first approximation every single computer in the world runs Linux. Then there is some marginal uses in desktops and a few iPhones, but everything else in the world runs Linux if it runs an operating system at all. That includes this camera, that includes the Wi-Fi relay, that includes the surveillance cameras, and about half a dozen computers in your car. So everything that is common infrastructure is going to end up being open source. So the next question is: is it the case that LLMs are going to become infrastructure? And usually it takes a few years. So if you remember the early days of the internet, if you wanted to set up internet service you had to buy your servers from Sun Microsystems or from Dell running Windows NT, or from HP running HP-UX, from Sun Microsystems running Solaris, and a few others. All proprietary, all with their own web servers. And then Google happened and they said, 'Look, you can buy just commodity hardware, run Linux on it.' And if it stops working, just throw it away and buy a new one. And that's what everybody is doing nowadays. GPUs changed that unfortunately, but basically the infrastructure software of the internet became completely open source because it's a common platform that needs portability, security, and everything else. And then the same thing happened for all the software stack of mobile communication. It's all open source. All the stack that runs in cell phone towers is all open source. It's not proprietary. So, is the same phenomenon going to apply to LLMs in the long term? There's no question it will. The question is how long is it going to take? And is there going to be regulatory capture in the meantime? Are governments going to, as they started doing, believe the AI companies when they tell them, 'Oh AI is super dangerous because bad people can make biological weapons with them so you should regulate it so that only people like us who know what they're doing can use it.' I mean, how ridiculous is that? That would kill open source.
But that's actually the reason why I think we need to invest in the developer experience because I think if we can actually fix that layer then we can actually rapidly increase the number of developers that actually have access and can use these systems and then could put pressure on: I need this to be cheaper. I need this to be more efficient. I need this to run on my device. And we can actually try to do some forcing pressure. So one of the things that besides just mimicking the OpenAI call so that developers who write Python can call it, there are things like Chrome has released their Prompting API so that now JavaScript developers inside the browser can start using it. Of course the way that Google has done it is that you're calling Gemini. So we have a version that we're allowing for choice on the back end. So you can actually choose which LLM provider you might want to go talk to. So I think that those type of experiences of allowing for choice to happen and to give access to more developers by lowering that barrier is one of the ways to speed up the desire to make this all open source.
