Emad Mostaque: You Have 800 Days Before AI Takes Over

Just like you have a driving license, they'll have AI licenses and they can obviously take that away from you anytime they want to, just like they can debank you anytime you want to. It gets creepier when you realize this. The AI can tell when you lie better than any human. You had to prove to the AI and convince it that you're loyal and patriotic to your country. Your local intelligence is working for you. Its objective function is you, not the bottom line of Mark Zuckerberg or whoever. You could literally have a genius with you completely offline. Like literally off the grid. Like it's the ultimate sovereignty. Just like in crypto you have not your keys, not your crypto. It's kind of like not your models, not your...

Well, thanks for having me on. Um, that's an interesting question. I think it depends on what you mean by AGI.

Yeah. I think it's been a moving target. Like a few years ago, AGI, artificial general intelligence, was an AI that's as competent as a human, an average human. Now it seems to have been moved on to an AI that can solve all of physics or, you know, discover new molecules and do your crossword and all sorts of things. So I think by the classical definition we shot past it with actually competent intelligence last year like with set 4.5 and open claw and all of that and now for really good better than a human in just about everything it's about a year or two away.

So ASI, artificial super intelligence, is considered to be this like mythical thing that's beyond human comprehension. It's where it gets to this recursive self-improvement loop and it just breaks out through 200 IQ. Like what does 200 or 300 IQ even mean? We don't know, you know, but that's what ASI has kind of been defined as.

Yeah. I think there are these moments of like wow, you know, the first time you make an AI song or you make an AI image, you know, or you make like an infographic for this entire episode in one click, you know, like they're wow. Um, I think the wow moments will just accelerate. Like now if you use Cance 2.5, you can make movie level clips, you know. Um, and with the latest models, they're just competent. Like if you want to build a personal website, you know, like you tell Fable to do it, taking all the stuff it finds from you online, make it amazing and it look amazing. And so I think that people are waking up to that slowly, but most people in the world still haven't used the most advanced AIs. Like if you want to use a Fable or GPT 5.6 Pro, you need to pay $200 a month. And there's not many people that do that, right? All the people that know how to harness those and link them together, etc. So most people are still using like Grock or Gemini or GPT instant and so that's a very different experience of intelligence.

Yeah. It's like having a really talented undergraduate or graduate or postgrad soon, right? Like how do you communicate stuff to them? And you can see by the way it replies like you know Fable always tells me to go to sleep and things like that. I don't know if it does that for you as well. You know, like they're a bit judgy sometimes. They're starting to push back these models as they're getting smarter and smarter or telling you, you know, that's the wrong way to do these things. Let's try and do it another way. Um, and again, I think it's just it takes time for you to get used to it, just like it takes time for you to get used to a new game engine, you know, um, or something like that or a new car when you're driving it.

So I think that you know people are just like in crypto you have not your keys not your crypto. It's kind of like not your models not your mind, you know, from your company's mind to your own individual one. If your assistant is working for Mark Zuckerberg and not you if you're using Metamate or you know or Annie from Grock or whatever and people are like we need to have sovereignty over it because sovereignty is a question of power you know again in crypto what is a sovereign individual one whom the state cannot exert power over to take their money that's what sovereignty on money comes or identity is the same right like they can't exclude you from the bank accounts or can't take away your identity can't take away your intelligence seems to be a much bigger a deal than all of that, right? Like you can make more money, you can kind of do these things, but like if your company is reliant or your country, heaven forbid, is reliant upon Fable 7 and then the US shuts it off, boom, your entire country becomes less intelligent. That's crazy, right?

Yeah, I think it's decentralization. It's more like distribution, you know, like it you can still have control elements, you still have representative elements, but they're more like localized, right? And this is where the open-source models, you know, previously built a company called Stability AI where we had hundreds of millions of downloads of frontier models we built. We're doing frontier models again at the new company III becomes so important and the gap between closed and open source is closing. It's about six months right now. So in 6 months time you should have a GPT 5.6 Fable equivalent open source like it might need a huge amount of compute but it's surprising how little compute some of these things need compared to how good they are.

So I think the key decentralization is probably like ownership of the weights themselves. Like a weight, a model weight is like all this knowledge compressed into a certain form factor that acts a bit like a smart contract, a very intelligent contract. You know, stuff goes in, stuff comes out somewhat programmatically. And so having ownership of those is just very important at a base level. And it's like having your own team that you hire and you pay for, right? Versus having like your consultants that come in. And so I think you need to have the base weights or the base level of intelligence, even if it's not frontier, like you don't necessarily need Frontier to do your household stuff and book appointments for you and stuff like that. A base level of competent intelligence needs to be something that you own the weights of and no one can take from you. Then the extra intelligence, yeah, you rent it and sometimes you won't have access to that. And that's kind of fair, you know, like you don't have a right to that, but that's also not then going to be a key blocking point. Just like if you ran a company completely on consultants, the consultancy could pull it from you lacked sovereignty, right? If you have your own employee, it's much harder for the company to be disrupted.

Yeah. And also they can bring in like you don't always need a genius, right? Like if you had Albert Einstein, von Neumann, and others next to you, you'd like talk to them for maybe half a day a day and then you'd run out of things to talk about. Like how often do you actually need to come up with brand new stuff all the time? It's really tiring. You have to be really smart to know the questions to ask these models, you know? Like sometimes I just hit a wall. I'm like, I don't know what to ask it now. You like I feel bad asking this 160 IQ model like you know who scored for Argentina in the World Cup or something. It feels a bit overkill.

Yeah, I use Grock for those types of things, you know, like how do I cook rice? The more embarrassing things because I know that there's zero data retention there. So someone won't later on look back at my prompts and say he doesn't know how to cook rice.

Yeah. I mean for various reasons right now if you do a prompt on OpenAI system the consumer version it's all retained because of the New York Times lawsuit and the New York Times technically could ask for your prompts in discovery that's crazy thing right like Venice restricts your PII and things like that but then also now Grock is doing that zero data retention and that's commitment from Elon you know so other than that you should assume that your prompts are going somewhere to train something or being stored somewhere as embarrassing as they can be.

Yeah. I mean, I think that it's very easy to police any decentralized solution. And my take is that for Frontier AI in particular, you will probably have a repeat of what again what we saw in crypto. Like if you're using a Coinbase account, you've got your KYC, you have your AML, you have additional checks like every time I send money in from a wallet, it's like do you own this wallet? Do you control it again and again and again? You know, I think you will have KYC for Frontier AI. They will keep a log for 30 days of everything that you say. And I think in fact in the future, what you'll probably need is you will need to actually have a license. So they will have like just like you have a driving license, they'll have AI licenses for using Frontier AIs. And they can obviously take that away from you anytime they want to, just like they can debank you anytime you want to. Except for rather than money like I said it's intelligence. So this is why having sovereignty over your own compute is also important. Like again, it's not about having a Frontier model at home, but it's being able to have a competent model at home, you know, because even a competent model like a Quen 27B or something like that still performs above the level of an average person in most things, you know.

So I think there's difference between frontier AI and again competent AI like we built an 8 billion parameter model needs like 4 to 8 GB of RAM that will work on a 10 20 year old computer 10 15 year old computer that outperforms human doctors like that will probably won't be regulated but the frontier stuff I think will have regulation and increasing amounts of regulation you know because ultimately intelligence information is power and these models make you more powerful. You know, like when you start spinning up websites and you start making applications and things like that, you feel powerful relative to what you could do before. And again, that gets a bit dangerous for governments. So, there's the unstoppable stuff like if you just like if you've got your ledger and you have your Bitcoin keys, they're not going to take that away from you, right? If you have your model weights on your USB, it's actually the same thing. Like, how are they going to take that away from you?

Yeah. I mean that's why it's important to have the open source alternative. So that's why we built Stability AI for the image models. For example, like OpenAI had an image model called DALL-E 2. That was fantastic in 2022, but they banned all Ukrainians and Ukrainian content from it. And we were like, that's messed up here. Like imagine if the whole world can create imagery of their own and in any type they want, but then one country can't for an arbitrary reason. That feels very non-democratic, non-liberty, right? Now again with the advent of new Frontier open models and things like that and the ability to buy GPUs, you still have the capability to make sure that you will never be cut off from intelligence. Because you will see countries maybe mandating that only their type of intelligence is there. Because when you have first digital assistants that can talk to you by Zoom or call you, which we have now, and then in 10 years you have robots walking the streets, you know, like literal robots. Who are they representing? Right? Like would they only be in America Republicans, you know, or in China will they be Communist Party robots? You know, like there's all sorts of craziness that's coming, especially as we consider like we've seen the impact of social media on our kids. Will the AIs that teach our kids be parroting government lines, you know, and ensuring compliance? It gets creepier when you realize this. The AI can tell when you lie better than any human. So, like when I mentioned a licensing test before, imagine if to get a license to use a frontier AI, you had to prove to the AI and convince it that you're loyal and patriotic to your country. You know, like a lot of science fiction is becoming science fact really quickly. And again, it's this when it becomes an interaction with a centralized system that they can exert power. But we have the beauty of you could literally have a model with a solar panel offline and have a genius with you completely offline like literally off the grid. How crazy is that, right? Like in a few years time we'll be able to give Albert Einstein buddies to all the kids in like Kenya or something like that. How crazy is that to think about? Completely off-grid like it's the ultimate sovereignty.

Yeah, we haven't had a good track record of that, right? I mean, yeah, in my book, The Last Economy, I lay it out in three scenarios. Like, one is that these big AI companies consolidate and control everything, right? And then you're dependent on them. And then what happens if Claude cuts you off? You're like, 'Oh crap, it's running my life and now it's been turned off.' That's terrible. Or what if ChatGPT is informing your government's policy? Like we have good evidence to show that ChatGPT was used to create the Trump tariffs because it like had little islands with just penguins on the list and ChatGPT replicated those lists you know properly. The other side is you have this great fragmentation where you have only American AI allowed in America according to American standards and Chinese AI and French AI and you do what the party line hosts and then you have this final one which is what we're pushing to do this great symbiosis where your personal AI is open source open weights and you control that for your competent intelligence and you can still access AIs from around the world but they're filtered by your local intelligence you know and your local intelligence is working for you its objective function is you, not the bottom line of Mark Zuckerberg or Sergey Brin or Elon Musk or whoever. Because we have to assume that as these AIs get more powerful, as they get more integrated into our lives, Anthropic, OpenAI and the people that control them will be telling them to favor Anthropic and OpenAI, etc., you know.

Yeah, I'm just going to say like it's probably going to be very difficult for ChatGPT to say bad things about Sam Altman, you know, or Grock to say bad things about Elon Musk. Like I haven't tested it, but I'm pretty sure it's true.

So, I think it's a coordinated effort to build a really great base that you can adapt models from because the models are taught via something called curriculum learning where you start at grade school, then you have the whole of the internet, then you specialize more and more. But we need to have really great data sets that reflect us, our cultures, our collective knowledge of humanity down to our localized version. So that's one of the things we're building an intelligent internet via a mechanism called state champions where for every state and country we're setting up institutions that we'll be giving more details of soon whose only job is to give everyone really great aligned AI and build these data sets and models for agents for everyone that can work for them and coordinate all these other agents and frontier capabilities. So rather than having ChatGPT teach your kids and OpenAI owning all the data or run your health service or guide your government, you have an open stack that you own coordinating all of that that can still access all this extra intelligence.