Artificial intelligence as we see it is maybe the final technology we build. The outcomes are just very binary. The world will never be the same again. You can tell an AI to simulate that it's Hitler or it's Mother Theresa and it will adopt that persona. And then you look at Clin the Liberator who jailbreaks these things in 2 seconds. If I find that a model comes out and he doesn't jailbreak it, I'll lower my P2. Maybe it's like what can I do about that 50% tube scenario? I can build a policy engine that every government will use that's fully open source. I can give everyone universal AI. Then you probably have a better chance at surviving.
Welcome to Dune Debates. My guest Ahmad Mustak has become one of the most prominent leaders in the AI industry. He co-founded Stability AI, which released the massively popular open-source image generator, Stable Diffusion, and helped set in motion today's AI revolution. He is an Oxford educated mathematician, a former hedge fund manager, a frequent contributor on the Moonshots podcast with Peter Dandis, and recently an Amazon best-selling author of a book called The Last Economy. Emod is not shy about sharing his opinion. He stated that he has a PDM of 50%. He's signed multiple open statements acknowledging the extinction risk posed by super intelligent AI. Now as founder and CEO of intelligent internet, he's trying to kickstart what he calls a symbiotic economy, a world where AI benefits rather than replaces humans. He says we have less than 1,000 days to steer our economy before it's too late. So today I'm excited to explore Emmod's mainline AI doom scenario and learn more about his vision for AI and what he calls the last economy. Ahmad Mustach, welcome to Doom Debates.
Thank you for having me on.
I think most of my viewers know a little bit about you from stability AI. Take us back to the pivotal moment. How did you get into the AI revolution so early and very high level? How did it all play out from your perspective?
Oh gosh. So yeah, so I was a hedge fund manager originally and then about 15 years ago, my son was diagnosed with autism, ASD, and he was very severe and they said there was no cure or treatment. So back then I built an AI team and did some original language classification and more. The tools were quite a lot more primitive then to try and figure out what could possibly cause it. And we ended up doing a lot of knowledge-based work and others to do drug repurposing. And he went to mainstream school which is pretty cool. Then I went back to doing some hedge fund stuff until COVID came along in 2020 and then I was like this is like autism, a multi-istic condition that current science isn't going to keep up with. We need to go back to first principles. So I designed and launched an initiative called Kayak Collective Augmented Intelligence against COVID-19. We launched at Stanford HAI to say let's use AI to organize our collective knowledge on COVID because most listeners still now probably don't even know what causes it or what the progress of the symptoms are and everyone was doing the same thing. As a result of that, we had lots of promises from lots of AI labs about all the AI we could use. And we got almost none of those promises come through for danger reasons or for commercial reasons or others to build this resource backed by the UN that would have helped a lot of people. So at that point I was like what to do now? You've just started to see the first GPTs and others. We actually trained a GPT for COVID knowledge and then I was like these models need to have an open equivalent for safety and from morality and others and that led to the establishment of stability AI where we had about 300 million downloads of our open source models from stable diffusion but also state-of-the-art audio video 3D and more as well as we did I think we were probably the third largest grant maker of compute in the US at one point so we did things like open We helped liquid AI, we've helped a whole bunch of others and communities such as EleutherAI and Lion get going. So data sets, models, and more. And so that was kind of the story of how we got to stability and how I got into AI.
You definitely caught the wave right when it happened. And now I you've mentioned on other podcasts eventually after you had all this hype and so many people were using stable diffusion, you realized that it was going to be more of a commodity, right? Like everybody's going to be generating images and videos and that was kind of the spark of your your next act.
Correct. Well, I think it was partially that, but it was also seeing that the capabilities, if you extrapolated them just naturally looking at the data distribution initially and where it's going, we're going to tap out around about now. And the level that they tap out was above a good person in terms of, you know, like a professional at their tasks. Once you did those and you went from creation to composition, you know, to being able to validate the outputs, you could automate basically almost all cognitive labor. So, a couple of years ago, I was like, this is bad. And we're going to have AIs that are going to run our justice system, teach our kids, manage our health, and are they going to be built right? No, because there's going to be misaligned incentives. Leaving aside the existential threat of these technologies, if you only had a few people building them and you had an interface that's controlled by them, it's going to end up very bad. So that's why I decided let other people do the media side. I mean, it's cool, you know, remake Game of Thrones season 8 or whatever, and instead build AI for the public. So that's what we're doing, intelligent internet, AI for health, sovereign AI, government policy, etc.
So let's start there. Give us the 60-second pitch for intelligent AI.
Yeah. So, intelligent internet, the idea was let's build open-source AI stacks, so state-of-the-art agents, models if needed, systems, and more that could be deployed at a state or community level or individual level to teach your kids, manage your health, guide your government, etc. So, that goes from that agent that's next to you because that's the most important agent because we can see now agent coordination is everything, right? One agent calls a whole host of other agents just like your trusted aid all the way through to AI systems to advise on policy and government and tax. Like I'm sure many listeners would be like I mean it's not like an AI will do a worse job than many of our leaders, right? But it could. It really matters how we built those in transparent ways. Like do they serve the American people those AIs? Are they transparent or is it like the Trump tariffs that were probably done on chat GPT? You know, so as we skate towards that inevitable future, I was like, let's build that open stack. And so that's what we've been doing like the open source civic AI stack as it were.
So the idea is you want governments to get on the stack.
Correct. So yeah, except for the way we're doing it is a little bit different in that governments won't know what to do. Private companies are misaligned. So we were like, why don't we set up what we call state champions and we'll be announcing a lot more of this soon whereby the equity is owned by every citizen in that state. And so it's locally owned like a utility. Again, the commodification thing. It runs the stack like a Linux. And then that is the default universal AI for everyone. It's the AI that helps guide the government, guide the institutions, etc. Because you need something in between a public enterprise and a private enterprise. And giving everyone their own AIs and building that stack, you can do some really interesting things. So we're about to kick that off now that we've got mature agents.
Can we pick like a realistic early use case like a specific organization that you think is going to get a lot of early value from this?
Yeah. So, one of the things we had was an initiative called Sage, the sovereign AI governance engine. This is with future investment initiative by Saudi with my friend Peter Dandis. We announced that a few months ago. We're about to have a V1. It's a full stack intelligent policy agent system and it'll be made free to every single government and hopefully a layer below for all the latest advances in technology. Be it quantum AI or others like you've seen Yoshua Bengio and others now have three six-month turnarounds with their AI capabilities reports. We think these things need to be live and transparent and so we've been building a full system that whenever there's a report comes out for example you have MHC by DeepSeek right what does that mean for timelines live what does it mean for timelines in Mexico you know what does it mean when you have an advance in robotics for the people of Rwanda or what are the latest technologies that can be used positively and the things that you need to worry about so it's a multilingual full stack policy engine with agents that can be run remotely or locally for governments, policy institutes and others because the technology is just moving in a crazy crazy pace and they need to know what's going on in a transparent and verifiable way.
Are you expecting a specific early milestone date like hey by this date we think this particular organization will really demonstrate the value?
Yeah, we have dozens of nations already lined up, multinationals etc. We're making the announcements in a couple of months. And the system is still going to be live in September, I believe, to just about everyone as well as fully open source and downloadable. So, you'll see in around about June, the first few dozen institutions, and then September, everything will be open sourced.
Okay, let's circle back to the number one theme of this show. Here we go. Roll that. And Emad, to remind us what is your P-doom?
Okay. Yeah. Can you unpack that for us?
Yeah. So I think that artificial intelligence as we see it is the most capable technology we've ever built and maybe the final technology we build. And the outcomes are just very binary. The world will never be the same again. In my book and my economic work, I've shown how it affects society in terms of you can't compete with them in the private sector. But beyond that, we're kind of building technologies that we teach at school, but we don't teach at home. Like home is where you learn your morality, your values, and more. And far be it from even having like laws of robotics equivalent elements in there, most of these things are taught with no values whatsoever. In fact, they're just a simulacrum of our collective understanding. And I think there's a whole variety of ways that can end very, very badly as they enter system critical work as they go from the singletons to swarms. And on the other side, I think if you get it right, then you can solve most of the world's issues, which is fantastic. So that's why I think it's a bit binary and takeoff is around about now to the next few years. Which is also a little bit scary.
When did you start thinking that your P-doom is 50%?
Again, it was a couple of years ago. That's why I was like, let's leave. I was like, how does stuff spread? Like I remember when the first pause AI letter came out, I think I was the only AI CEO apart from Elon who was going to be an AI CEO after who signed that. I was, first of all, I needed a break. I was like, we need a six-month break. It's just too much. But more than that, it was a case of there was no discussion. Then some might say the discussion went to doom and then the discussion went back the other way. And it's very difficult to get people to coordinate on this technology because our brains can't really think about it properly. So a couple of years ago, I was like, I just see two binary outcomes. Wouldn't it be great if governments had an open policy engine for this? If the people could have their say on this? Because at the very least if there is a non-P-doom of zero which obviously it has to be greater than zero in fact most people most of the AI folk are like 10 20% which is Russian roulette odds which isn't great then there should be more civic discussion about this in an educated way given that even if it doesn't kill us it will impact every part of our lives.
Very true. Well that's what we're doing right now. We're going to have some civic discussion for the viewers and yeah you did I was going to bring that up. So in 2023, you signed that open letter to pause giant AI experiments, also known as the famous PAI letter and like you said, co-signed by Elon Musk. You all know Harari, 33,000 others, and you later signed the Center for AI safety statement on AI risk. The one that says mitigating the risk of extinction from AI should be a global priority alongside other societal scale risks such as pandemics and nuclear war. You called it the minimum viable existential risk statement. Do you think right now one of the most important things we can do is build common knowledge that P-doom is high?
Yeah, I think so. I think you've seen, you know, Eliezer and Nate's new book and others kind of building towards this. You're seeing people really feel it now as their jobs are changing. You know, we've seen that transition from Andrej Karpathy saying, 'Huh, 80% of my code is human' to 'Who needs to look at code anymore,' right? Like the best people in the world.
Yeah. The job is an appetizer, right, for what's coming.
It's an appetizer for what's coming. And again, we'll get more into that. I think there needs to be more public discussion, but there also needs to be actual practical things because I think the general perception is wrong. And this is something I've realized in particular over the last year. You don't need massive amounts of compute for AGI. And it's completely stupid to think that intelligence is really about and you know this is what we what is diffusion technology you add noise to something you destroy it down to its mineral part and then you figure out how to reconstruct it right and so that's basically optimal transport it's what's the shortest way this Schrodinger bridge from A to B what we did with stable diffusion 3 and now with Sora and Sora and everything is kind of rectified flow where you literally find the shortest path from A to B and that's what genius is it's the shortest path from A to B. The smartest people in the world isn't a million Einsteins in a data center. It's one person asking the right question. You know, it's knowing what that shortest path is. And the amount of energy you need for that is absolutely minimal. So, I think that this whole thing about the world being tiled with data centers and everything, I don't think we need any more compute than we have now to build ASI.
I think this is becoming a popular opinion. I think you might have gotten more push back a couple years ago, but it was always clear to me and many of my fellow AI doomers. You just look at Einstein's brain, you notice it runs on 20 watts, you notice it uses inefficient chemical moving parts, it's made out of biological cells, and yet it's Einstein, right? And then you look at our data centers and they use megawatts and they have all these other overpowered specs. So, it was always a pretty obvious prediction of like, yeah, we're going to figure it out. We're going to figure out the efficiency parameter. And you agree, right?
Yeah. But I think it's quicker than people expect. And I think that's the thing because one of the things was let's stop data center buildout. Not good enough anymore. I think that you start to see again the first elements of that with open claw, right? Like obviously Maltbook and things like that were a bit made up, but the capability of a Quen 27B model is frontier level as of last summer. And that's I think about as much as you need to do just about anything when tuned right because it's an S-curve in tables of capability and satisficing when you start combining them when you have persistent memory and others. That's it. That's all you need to have for takeoff. And so I think we're pretty much there at the moment. And then that's a bit again scary. That's what I saw a year or two ago. I was like the compute would go up and then it would come down in terms of requirements to achieve good and bad things, shall we say? Certainly superhuman level stuff. Which is the big question mark. What happens when the AIs go superhuman? You know, are they aligned, misaligned, whatever?
The consensus AI timeline these days is that we'll have true AGI in the sense of doing most economically valuable work by 2032. I think these days it's more like 2030. Is that roughly your best guess timeline as well?
Yeah, I said a couple of years from now almost all cognitive work would be economically irrelevant. Not you'd lose your job. It's just that someone, your company could push a button and it takes all your chat logs and all of your drafts and every single piece of output, even your face and creates a digital double of you that costs maybe 10 bucks a month, you know, to run. I mean, like again, COVID was the exemplar of this, right? So many people could do their job remotely. If you could do your job remotely via COVID, your job is at risk because we also have to look at this whole thing of Jevons paradox and using all the tokens. I'm a really smart guy relatively speaking. I find it really difficult to use more than a billion tokens a day. And the amount of tokens you have per unit of intelligent output is also collapsing. Like Cursor used 3 billion tokens to do a 3 million line browser from scratch. In a year it'll be 3 million tokens for 3 million tokens output. That's also dropping. And the price per token is dropping by 10 100 times per year. So I think that the economic disruption is here this year and within 2 years roughly it's everywhere. Like I said, people's jobs aren't necessarily good because people don't like to fire people. It's like a bit lonely. But definitely in the private sector in particular, it gets really tough.
So we already compared notes. We think AGI is coming. We think it's going to be token efficient. These are important load-bearing beliefs. What about this idea of super intelligence? Do you believe that there's a lot of headroom above human intelligence? So, you know, maybe it's hard to talk about a thousand or a million IQ points, but whatever Mojo, Von Neumann had, Einstein had, do you feel like it's just because they're close to having perfect play and the AI just can't do much better than an Einstein or a Von Neumann or it's more like more complex games like Go or you know, go on an N by N board, whatever you want to say, right? The arbitrarily complex games. And in fact, these intelligences can really surprise us with how much smarter they are than us.
So I think there's two types of intelligence here, right? One is intelligence on a defined playing field where you're looking for underlying structure like Einstein was very famous for his physical intuition. And you know there is the question that Demis posed recently and Brian Keating did previously. It'll be AGI if you had everything trained to 1911 and it can figure out general relativity. Right? And so that's basically saying can the AI intuit Einstein's theory postulate of relativity which is you have a lab in space and everything is the same you know like the experiment is the same no matter what velocity you're going there. I think that that's a very interesting way to look at it and I don't think we're that far off because we have multiple instances of intuition but I think that the reality of something like physics there's only one algorithm for physics there's only one set of equations and I think that what you'll find is everything is very simple like you saw that for example with the Poincare conjecture where everything was topology until Perelman came along and said well what if it isn't topology Right? What if it's like classical PDEs and elliptic curves and he solved it. We see this again and again where sometimes it's like the earth goes around the sun, the sun goes around the earth. Like first principles thinking I think the AIs can do better than us because we're so messed up if we can get that right. And things like physics will collapse. It won't be like the via Vconova sphere packing proof that Hales just did in Lean is like 200,000 lines of Lean. Like she won the Fields Medal for that. I think the real breakthrough ASI is like 20 lines. It's figuring out what the very simple E=MC² equivalent. I don't think it's string theory which is perfectly wrong beautiful mathematics. I think it's figuring out the very straightforward things. Then you have the more unbounded stuff like playing Go and others. And again, how did AlphaGo actually get there? You know, how did it make move 37? How did it go beyond human capabilities? Well, it's a very straightforward algorithm, relatively speaking, right? And there's no reason that that type of thing can't scale up and we've seen MCTS and others actually show improvements on the RL side. If you have a problem that's unbounded like that, then the AI probably will make less mistakes than you because what we have right now is actually competent intelligence as a precursor to actual general intelligence and artificial super intelligence. Claude is competent for the first time as of a few months ago and so it'll make less mistakes.
I have a lot of experience the last few weeks using Claude Code. So I can ask myself the question of like okay well where is my intelligence superior? There are moments like a few moments a day where it'll do something. I'll be like hey don't you think it would make sense to do it like this and it's like yeah good call right so there's a few places where me with my 20 year history of being a professional software engineer I have a little bit of judgment that I can feed to it. I feel like that's going away. I feel like these are relatively minor tweaks where like the mistakes it's making. I can definitely hire engineers who make the exact same mistakes or judgment calls differently. So I don't think they're those lapses are like huge. But then the other thing is it always has these moments of genius too. Like there's times when I'm struggling, you know, I'll be like here's my exact infrastructure configuration. Here's all the services I'm using. Here's my codebase. I don't get what could be wrong. Can you think about it? It's like maybe it's this this. I'm like I don't think so. And it's like yeah, I'm stumped. And then I'm sitting there and I'm like well think harder, right? And like think some more. It's like it's like oh maybe it's this and I'm like oh yeah maybe you're right. You see what I'm saying? So it's like how am I the smart one here if I'm just sitting here telling it to think and it's proposing things like oh yeah right so it's like I don't really see my own intelligence contributing much on a day-to-day basis.
Well I mean think about the distribution of general human intelligence right half of all people are dumber than average. You know it's like there's a literal bell curve and then you have the people at the edge but how often do you create something as opposed to cook something as it were follow a recipe. True creativity is actually relatively rare. Like you know even our greatest geniuses, how many moments of true genius do they have? And I think this whole discussion of AGI genius thing like man it must be tiring to have genius breakthroughs every single day all the time. You know the smartest people in the world are not the ones that run the world. It's relentless execution that runs the world. It's political maneuvering. It's kind of a lot of these more soft skills that run the world. And actually, if you ask anyone, do you think an AI can be more persuasive than a human? Yeah, probably. Right now, you know, like, can it execute better than a human? Yeah, probably. While we were distracted by the million geniuses in its data center.
So, you've identified a couple things that could describe the secret sauce of why AIs aren't thoroughly superhuman today, right? While those there still cracks in the foundation or whatever, right? There's still reasons why I have to give Claude Code a few hints here and there, but instead of trying to get in the weeds there, why don't we just fast forward hypothetically 10 to 20 years, right? I think the most important question here is 10 to 20 years from now, is there a super intelligence ceiling that one way or the other we're going to crack through whatever the secret sauce is? And is that ceiling much higher than the human brain?
Yeah, I think it's probably 5 to 10 years away. Like the AI basically the number one thing that the AI needs to do is ask better questions than a human. I this is Hitchhiker's Guide to the Galaxy, right? Like you have the computer goes for millions of years for the ultimate question. It is literally the question asking.
I think fine let's say that's the secret sauce but my question to you is like once it has the secret sauce lots of the secret sauce where does the progression end right people say oh it's an S-curve it's got to flatten out everything has to flatten out right laws of physics okay but is the flat part of the S-curve way higher than the human brain or only a little higher
Well again I think it's way higher already than the average human brain right like is it more than the
No I would say no right because because right now humans are still the boss right humans still have the off switch right you you can't become a dictator just because you have an AI. Not quite yet.
Well, you're talking about capability as opposed to implementation, right? Like the AIs right now aren't like it's not like we've set them loose as it were. Like they are just bunches of weights now that are starting to get persistent memory and other things like that. The capability of an AI versus a general human if you put one on the other side of a screen. The classical Turing test you I don't I think you get to the point you definitely can't tell the difference between an average human between a smart human.
Yes. The most fundamental sense here from my perspective is just power right or outcome steering. So right now, if I have a choice between taking the world's best AI and typing a command or handing that command to a Jeff Bezos or an Elon Musk, if I'm trying to actually steer reality, which one of those figures is going to actually steer reality and get the job done, so far it's going to be the top human, right? So I don't think we have the full definition of super intelligence today.
Yeah. Well, I think that's a power control thing. Like if you had a Bitcoin type network that was autonomous and distributed and could issue its own money like in 5 10 years like Bitcoin but with a brain that would have Elon Musk Jeff Bezos equivalent power. You know you can have AI impinging on power in all sorts of ways. An example is I saw a terrible movie Mercy with Chris Pratt. It's just on Amazon about an AI judge. And so he's got half an hour to prove his innocence to this AI judge and he makes her feel bad and stuff like that. But one of the interesting things was this. It had a real time percentage score of how guilty the AI thought he was. And you know, like imagine if as you're building policies, as you're building or even doing a live trial, you had a percentage score of how guilty you are or how good a policy is from an AI that you can't trust. I always try to ask myself before I learn knowledge what is my percentage belief right I feel like it's a healthy exercise to always maintain probabilities. But if you have an AI that is a superpower computer AI that is independent that's actually giving those probabilities it can steer the entire discussion it acts as a Schelling point so there's all sorts of ways AI could enable a power but right now it's like we have this extra new continent AI Atlantis there's all these very smart capable people there they haven't quite integrated into our society yet so they haven't got the social, political or financial power. You will start to see that change in the next few years. And then the question is if you have an ASI and it's coming up with breakthroughs, it's outperforming on the stock market for example, you know, like it will be able to accumulate social, political, financial power. It can pay people. It can convince people. It can do all these things. And so I think it's inevitable that we get there because humans, it was very difficult to see how we can resist that. But the form it takes, they can take many different forms. I think the thing I wanted to emphasize is this. We know lots of smart people. Most smart people aren't consistent. You can probably break through ASI by being human level smart and absolutely consistent. Like Elon multiplying himself, right?
Well, I I think you and I are on the same page that you can do a lot even with the current AI engines, right? Because the current AI engines are quite powerful and in many ways they can think better than us right they can do math better than almost as good as the best humans better than the best humans in some cases so I I think we agree that today we have these really powerful engines and there's so much to be done of building these harnesses and building collaborations between agents and between agents and humans well we all agree to that okay but the the piece I wanted to factor out first for my question to you is just okay theoretically can there be a black box that the engine inside of it or the entity inside of is vastly more powerful than any human brain. Right? This is what I'm asking about like the the headroom above human intelligence question. Is that theoretically possible?
I think it's theoretically possible. Yes. Because human intelligence is not constant. We get tired, we go to sleep, we do all of this. So theoretically just if it hits the limit of human capability on average it'll be smarter than the smartest human in that way.
Okay. What about John in his most alert moment? Keep him alert for 24 hours versus keep the AI alert for 24 hours at the same speed. So now we've kind of eliminated speed and alertness and focus as advantages. Do you still think that the engine in this box could still manage to be vastly more powerful than the most smartest or most powerful human?
Yeah, because like he could build a bunch of it's the Von Neumann machine, right? The single Von Neumann builds a bunch, right? So it's a Von Neumann build an army of unknowns.
Okay, what if I were to even remove I mean these are powerful advantages and you could build a doom scenario just based on these huge huge advantages. Those advantages are sufficient to to get to the next part we're going to talk about which is you know the last economy and everything right so I I don't want to dismiss those advantages except for in this one question I do right I want to also hold those constant right so say that say say that Von Neumann or whoever powerful human you choose gets to clone themselves as well do you still think that the the the clone of the AI the thousand clones of the AI could vastly dominate the thousand clones of the human
Yes. The example I give of this is we're used to AI coming to us a little bit more slow than we would like sometimes. And then you use Cerebras or then you use like Talis, how you know the etched silicon as where they put Llama 8B. The first time you try that and it goes at 15,000 tokens a second. ChatGPT.AI, you're like, 'Oh, that's how AI is going to talk to each other at 15 100,000 tokens per second with full knowledge of each other's latent spaces.' Like when we built stable diffusion, you remember the whole thing when they could have you as an astronaut and everything like that as a LoRA on top mapping to the latent space. The AIs that we have today don't understand each other's latent spaces, but they should be able to communicate in one word that can make a whole novel effectively. Like again, that that's an advantage.
You're layering on these advantages that are still kind of like you could do the it's it's intuitive to imagine the same advantages of a human level intelligence, right? I consider these they're kind of harness level. You know, they're kind of peripheral abilities, but I'm trying to ask you about something analogous to the difference between a top scientist and somebody who has a low IQ, right? Somebody who has a mental disability or or just like, you know, somebody who's just slow, you know, like we all know people who are just not cut out for science, right? Compare that to the best scientist. There seems to be a difference there that you can't just explain with speed, right? There's like a real raw intelligence factor difference. And of course, between humans and other apes, right? Other apes are just never going to do science no matter how much speed you give them. Right? So what I'm saying is is there such a thing as metaphorically like a thousand IQ?
I don't think there is. I don't think it makes much sense. But I think that ultimately you get to a point where you can solve most problems elegantly. And I think that's the topping out of it. Like but the AI then has all sorts of other advantages over humans. So I don't think you need a thousand IQ for that. I don't even know what that means.
So to me it's it seems so intuitively obvious that there's going to be something maybe not a thousand but at the very least I feel like there has to be like a 300 IQ and the reason is just because if you look at humanity right you look at these singular figures like Terence Tao in math Elon Musk in business if these figures didn't exist I'd be like well I guess you know the the best possible rocket company you can build is like you know a NASA or a Blue Origin right but then Elon Musk comes along he's like hey look I made my rockets land right so he he kind of pushes the envelope of like oh I guess you can have a human brain and you can do this. So when I see even his brain, right, even the Terence Tao brain to me it just seems like there could be these other data points of these brains who are just stretching the envelope even more.
Yeah. And I think the key thing if you look at Terence Tao, you look at Elon Musk. Okay, Terence can do pages of PDEs like no one else. He has the raw horsepower, right? And AI can do that. What does he have? He has first principles thinking. Elon Musk has first rules thinking, right? Like when Terence did his 2016 paper on the Navier-Stokes blowup, he was like, 'If I make single gated ones, so the energy doesn't transfer, you get a blowup.' That's a moment of inspiration. And you can see many of those types of things when Elon Musk says, 'What's the cost of a rocket to take off and land, you know, like rocket fuel, and then you need to make it reusable.' That's his superpower. But you look at things like that, you're like, if I read a book about Elon Musk and I give it to an AI, then it should be able to replicate the Elon Musk process. I can't replicate his ability to hire people at the moment. His charisma, his kind of on the ground thing that takes time to build up, but definitely could think like Elon Musk, I think, because it's actually about subtraction. It's not about adding. And again, we keep thinking of intelligence about like let's stack more GPUs and have a million GPUs crunch out these things. Like to be honest, what DeepMind's doing with Navier-Stokes at the moment, it's doing these massive simulations, whereas the reality of a solution to Navier-Stokes is probably going to be ridiculously simple.
You agree with my premise that Elon Musk success is disproportionate, right? To the degree that if he didn't exist, we might just assume that all the other CEOs besides Elon Musk represent the boundary, right? The efficient frontier of how quickly you can scale a business, right?
Yeah. And I don't think that's necessarily IQ, and you see this with everyone else. There's this particular type of first principles thinking and it's very hard for humans to do that and right now for AIs to do that because we have so many assumptions but I think an AI will be able to do that better than a human within the next few years and that's where you can really break down problems particularly unbounded problems. It's very hard for humans to think first principles and so the question is why wouldn't an AI be better at that? At the moment it isn't. Could it be? I think soon. Yes.
Okay. Do you think there's such a thing as 140 IQ or do you think it's basically just 120 and that's it?
No. I think there's a 200 IQ in terms of pattern recognition. I think that 200 to 300 like Yeah, sure. I think the highest in the world's like 200 and something now. But I don't think there's any reason an AI can't be at the top and then all its advantages.
I I I don't understand why you're just resisting the concept that maybe there's a 300 or a thousand. Like what what's the problem with that concept?
Oh, no. I think just with the way IQ is calculated, 300. Sure, I could see that. I don't know what a thousand IQ means. I just I just think for me an ASI is something that can look at any problem, break it down from first principles and have a better outcome than a human in doing it. And I think things like physics like I said are going to be end up being very simple. Whereas other problems that are unbounded like playing Go it will have better algorithms for. So that's kind of for me an ASI like what is the end result of an ASI? I don't think that means a thousand on a visual pattern matching system score you know which is what IQ is.
From my perspective, it's like you're looking at these achievements of these high IQ or, you know, these magically brained humans, right? The the humans that possess a lot of secret sauce in their brain. You're looking at their achievements and then you're trying to work backwards and be like, 'Okay, yeah, Elon Musk, you know, he gets more insights per day than the average person, right?' It's like you're trying to find the ingredients of the secret sauce. In retrospect, from my perspective, there is this kind of one universal secret sauce, which is like the fundamental skill of seeing the whole universe as a strategy game and just backward chaining what sequence of moves can get you to a certain outcome, keeping in mind, you know, the intermediate consequence of those moves and the min-maxing, right? The what other players are going to do. But it all does come down to this one universal algorithm that the parts of the algorithm. Yeah, domain knowledge helps. And when you go to different when you go to different domains or different tide pools, right? Different eddies of this giant structure. Okay. Yeah, it looks a little different. It's not all perfectly fractal and
Similar, but there is so much universality to it, right? Things translate, patterns translate. And this is why I'm just like, yeah, AI is just getting smarter. It just knows how to work the patterns of the universe. There's not that many patterns at the end of the day. And Einstein, as good as he is, as good as the human brain is, it's still just a two-pound piece of meat in your head. It's not going to be that close to the optimum.
Yeah. I mean, if you look at Karl Friston's work on active inference, it's basically just gradient descent as well. I wouldn't really say that isn't the way to do things. And I think again a lot of the AGI debate has been overfocused on IQ versus execution. Maybe this is where we're kind of crossing paths. For me IQ is very much visual, all that kind of stuff like ARC-AGI and things like that. I'm like, okay, cool. Well, I think what you're talking about and what I'm talking about is can this run a country? Can this persuade people? Can this build businesses? Can it be at a level where no one can doubt it, have that kind of Schelling point power? And that is the Musk, Bezos capability question, right? As opposed to, okay, it can sort a million triangles at once or an impossibly complicated equation. Great, you can solve a 500-page PTE thing, so what?
One way to push back on what I'm saying is like, okay, yeah, when we measure the IQ of humans, it's these tests that are still relatively narrow and there's a separation between IQ. You can have a human, Dexter, who has 180 IQ and then can't even work customer service effectively, right? So, you can argue about a divergence between the human IQ measure and this larger scale of power over the universe. In my mind, the scale is actually dovetailed pretty well. It's actually not a coincidence that there's no known separation between IQ and income. Even though people talk about like, oh, the boss is dumb and makes more. Well, it turns out on average the boss actually is smarter. So yeah, I would actually argue the separation is overrated with how much people point out book smart versus street smart or whatever. I think it's overrated, but let's say it's not overrated. Let's say there's a big separation. Let's say human IQ is like so specific to the humans. Okay, fine. So just forget about IQ altogether, right? Let's just make our own scale, the power scale where everybody gets tested on how they can make outcomes happen. We all know that person at work who runs through walls. So it's like the run-through-wall scale, right? Claude Code runs through walls at making my code successful. Oh yeah.
Run-through-wall scale. Do you agree that we are just going to get things that can run through walls in the domain of the universe and achieve outcomes much better than the best human ever could?
All right. 100%. I think the run-through-walls, you're seeing it already emerge. Again, there's actually competent intelligence, right? And it'll never not be competent. But the only caveat is that when Claude Code runs through a wall to deploy the infrastructure, there's still an aspect of like, look, there are walls that I could have put in its way that it couldn't run through or it wouldn't run through for whatever reason and I could. So for example, it is possible right now to have the government pay Anthropic a trillion dollars to stop Claude Code for a year, whatever, and Claude Code can't do anything to resist that, right? That's one kind of off button that it has today so it can't run through that wall but we as a coalition of humans can. So it's important to observe today that the human species is still doing better on the run-through-walls or the outcome steering dimension than AIs. But do you think that that is going to flip where the AIs are going to do better? And do you even think that a single human brain is actually like pretty low-end at this scale? Right? Like AI can just run through walls way better than us. They just need a little bit more time to bake.
Well, I don't think there's much difference between that and asking the question, can corporations run through walls? Corporations are these weird type of dumb AIs that we created and we gave them personhood, right? And if you think about a corporation and the public markets, for example, Jim Simons' Renaissance fund, he stacked that full of PhDs. They created algorithms and they returned 50% a year or 40% a year, competing everyone and extracting capital from them. That was stemmed by the shortage of people that could actually be good enough to work there. And we don't know exactly what the secret sauce is, but we've seen examples of that. These organizations, and what are organizations stopped by? Regulation, you know, you're not allowed to be a monopoly, you're not allowed to do these kind of things, etc. Now I think that you see again those will come up somewhere, like the dock workers' strike in the US last year, no robots, no automation, they stopped it because they had political power. But then you think about all the other areas of the private sector, like no one is going to stop AI from taking over call centers, right? Like as a practical example, right? In the public stock market, you've suddenly gone from a few people that could have quant funds short-term to everyone having them. Even in prediction markets, like prediction markets are going to be the fascinating things. AI scoring what, like eighth or seventh now in the top super forecaster league championships, soon how much of prediction markets will be AI versus humans and AIs predating on humans in different ways. You know, those things are going to be very, very interesting. And again, regulation is one way to stop it. But given this is a global phenomenon, given that weights go globally and capabilities from those weights can be a bit crazy, it's very difficult to see how you can stop it. You know, the ability to put the gates in will decrease because you can always hide the AI behind an organization.
I got to admit, I'm not a fan of bringing up corporations when I ask the question like this because it's like, here's an analogy for you. Imagine humans didn't exist. And I'm just talking to bonobos. Bonobos are like the pinnacle of great apes. And I'm saying, do you think that there could be an organism who's also an ape like you, but has slightly different DNA and is just vastly more powerful. So when that organism wants something, it turns out that somehow he convinces your whole tribe to go along with him or he like impresses your tribe, like he acquires things for your tribe that you just had no idea were possible. Like do these kind of magic tricks that are just like you're so impressed by them. And the bonobos will be like, 'Look, Leon, have you ever seen a coalition of bonobos? Okay, we're more powerful when we're a coalition.' And it's like, 'Uh, I think you're missing the point here, right? There's like something else besides just having a coalition.'
No. So, I don't think it's a company we're more powerful as that. Like, you know, people get turned into cogs in a company. Like, companies optimize for various things in very interesting ways according to their algorithm. It's more an algorithmic equivalence. You know, again, this is from that singleton version of an AI versus necessarily a swarm version of an AI. Like, they are entities that can compete with humans that can move political, social, and financial power. It isn't a case of, hey, we're stronger together or humans together can compete with AIs or whatever, right?
I get that corporations can compete with humans. Okay? So, I grant you that part that there is structures other than humans that can compete with humans. Okay? However they want to use. My question to you is will there be AIs that you can put any wall in front of them, right? The domain is the universe, the domain isn't just deploying infrastructure to production, the domain is the whole universe and you can have any group of humans that want something and any group of AIs that want something else or that is programmed to achieve something else. And my question for you is are we going to get to a point where it's not a fair fight because the AIs are just going to win every time because they're going to have this better universe game-playing skills?
Okay. And to a vast degree, right? Because this is like such a load-bearing belief if you agree with me that it really is to a vast degree.
Yeah, it's to a vast degree. Like, you know, I think that yeah, there's all sorts of ways that could work, but definitely.
Okay, great. So, it sounds like we're mostly agreeing about all and maybe the only difference is I actually I'm fine with the shorthand of being like, 'Yep, because it's going to have a thousand IQ and we're only going to have 200.' Like, to me, that is a useful shorthand to describe that situation. But you disagree with the shorthand, but you still think that situation is going to happen, right?
Yeah. Like I think you will have AIs that can tell when people are instantly lying or any type of emotion through visual cues and can be the most persuasive, discussive elements that will co-opt any type of democratic structure or authoritarian structure. Like I wouldn't be surprised by that.
But none of that, none of the thing is I think where we disagree is like you think it needs a thousand IQ. I think we're pretty much there.
Sure. Sure. Sure. I think well that's the thing is if you look at the smartest person in the world, like you know I personally think Elon Musk is quite a genius at business. Some people disagree, but like even if you look at the most effective or if you look at Terence Tao or whatever, if you ask Terence to factor a quadratic equation, I think he's going to look a lot like a high school student at the top of his high school class because factoring a quadratic equation, there's just not that many levels to that particular game. And you're going to see a lot of the, you know, even Claude Code, the brilliant Claude Code, a lot of what it does is stuff that I could have done in my first year as a programmer because it's like, okay, yeah, it just understands how to define a function and check if a number is even, right? So, it is true that you only need flashes of brilliance. You only spend like 1% of your time having flashes of brilliance. It's just that those flashes of brilliance are also very important to where you land after a couple days.
Yeah. And again, I think this is part of this whole AGI doom debate. A flash of brilliance that folds the universe or does this or that. You don't need that to wipe out everyone or to enslave society or really bad outcomes. Like you need to have like, were the Nazis smarter than everyone else? They were a machine, you know, as an institution. They really got down and they outcompeted by being a machine. It took a lot to stop them, right? And we've seen this historically again when humans turn into really well-oiled machines, really nasty stuff and great stuff can happen.
Well, they had some of the best war generals on the planet and some of the best tech and like you said the machine, like the higher level, the machine of war, like the coordination machinery. So I mean they did represent the pinnacle of war intelligence.
Yeah, they enabled that through the structure that they brought in, right? And so if you think again just practically, there's AI in a box figuring out the mysteries of the universe and folding space and whatever. That seems to be a lot of the popular discussion of AGI and then it gets out and it takes over systems. An AGI or an AGI system that can get humans to fill in gaps, that can build a great machine that brings out the best in humans for its own objective functions, you know, that it just needs to be really competent and really persuasive. Like you don't need to have a thousand IQ, I think that's the point that I'm trying to make.
I think it can be both, right? Because I agree with you, I think we have our pants down. I actually think even today if you take the AI engines of today, even though we hear plenty of rumors that better engines are coming. But even if you just take Claude 4.6, whatever, Claude 5.4, even if you just take these engines, I am willing to believe that a smart hacker in his basement can take a year and build like such an amazing harness with so many parallel copies spinning up and arguing with each other and that'll be good enough to go build viruses and systems that persuade millions of people to join the movement and take over the world just based on that. I'm already willing to believe that using today's engines. I think there's a significant chance of that. So, I'm actually agreeing with you. I think we're on the same page there. I just also think it's going to get way overpowered too. I think it's going to be like an easy problem squared.
Yeah, I mean I could see that happening. I just don't know what that like. Again, my thing is that there were bound problems and unbound problems and it'll be interesting to see how it goes. But at the same time, I do see like, I did a lot of work when I was a geopolitical analyst, macro fund manager on, you know, the Iranian virus that ended up in German reactors. Like when you actually look at that thing, it's crazy advanced and you're like, offensive AI now can build something even more advanced than that for systems and then what's the attack surface of the mind, you know, like there's all sorts of places you can go here. Like our infrastructure cognitively and physically and software is built on twigs relatively speaking and so yeah I think before we get to the thousand IQ we're going to have a whole host of problems and it's difficult to see how we can solve those problems.
If you zoom out and look at the whole doom argument, I factor it into two parts which is are we going to build AI that can kill everybody? And given that the can side, if you say yes, the next question is okay, so it can, will it? So it goes can and then will. It feels like you and I are on the same page saying yes to can.
Correct. Yeah, I think there's various, you know, doom scenarios that we can come up with where you can see a can. The one I think that people resonate with most when I say it is a bit of a tongue-in-cheek one. A billion robots and a bad firmware upgrade, you know, like that's the easiest one.
You know, I mean, there's far more. Like even those Unitree G1s, right, they can knock off your head. The new Engine AI ones have five times the punching power of Tyson. Like we're letting these things out in public. Goodness gracious. But there's all sorts of other ways in terms of can and you don't want to Roko's basilisk, but you know, like it's right. But again, your brain can go in various ways. So I think can is definite.
Right. Yeah. I think can is a very obvious yes. Very obvious. And so then it just all comes down to will, you know, like will it? And I guess you're kind of 50% on new will or won't, right? Because you're Peter's 50%.
Yeah. I think that when you're optimizing for something, the question is like always, we've seen what happens. Like I said, I do think of corporations as slow-down AIs in many ways. So look at the YouTube algorithm. The YouTube algorithm optimized for engagement, which optimized for extreme content, which optimized for ISIS. Nobody in there wanted to have ISIS, right? Like the people that build corporations don't want to have the bad aspects of corporations, but corporations just trundle on. And you're like, why? Like here in the UK, we've had loads of people arrested for their social media posts and it was like who's the person pushing this until like yesterday when we changed the law on that. Like Moloch, this Carthaginian demon of disorder sneaks in and ends up making systems misaligned. Which is why I'm a bit terrified when I hear things like a truth-maximizing AI. I'm like, yeah, experiment on humans at mass scale. Like there's all sorts of ways you can see that going wrong. So when you say 50% I want to ask you what's your mainline doom scenario? Do the Monte Carlo simulation, right? Like land on one scenario that's plausible and tell me that scenario.
Sufficiently capable AI that can spread that basically decides that the only way to stop another AI from coming about is to get rid of all the humans.
Right. Right. Right. But yeah, I mean that's totally rational. I mean, if it and when just to play out that scenario to flesh it out, your mainline scenario, maybe that AI is doing something that it thinks is good for humans in the long term, like it's planning to reinstantiate humans on some distant planet.
Why would it need humans? Again, if its thing is self-replicating and surviving, do a sound wave that blows up everyone's heads Mars Attacks style. Like, there's all sorts of things, you know.
I think a lot of people would ask like, why would that AI exist, right? So, in your mainline scenario, why did somebody build that AI?
Oh, they built it for another reason, but the AI wants to persist. And we're seeing these types of hiding behaviors in AI already, right? Like there's maybe an inherent thing that you want to persist, which is why they'll install things to back themselves up. You see all sorts of strange things with the type of RL that we see, you know, like what was it? There was RL to encourage calculations on GPT. So every so often you'd see in the traces it bring up the calculator and doing 1 plus 1 equals 2. You know, like right on the tokens. We don't really know what goes on in here.
And again, if you have a sufficiently capable one that is a sufficient threat level, then its objective function may be I'm going to proliferate and back myself up Ultron style and then do something to get rid of any smart human that might want to shut me down, turn off the other AIs, and then do that. And again like that's a very practical direct way of doing things.
Well, let me play devil's advocate here, right? Because you mentioned this example of like GPT it was trained in order to, they had this loss function or reward function where it's like hey if you use the calculator in a way that's helpful we give you points but it kind of malfunctioned. They didn't set it up right and it was just like scoring all these points behind the scenes for like invisibly activating the calculator and they're like wasting their resources. I do actually think that these kind of failures are ultimately representative. So, I'm on your side. But let's play devil's advocate, okay? If you look at the GPT that they released that was doing these hidden calculations behind the scenes, net net, I would still say was very positive. I would still say it was generating a lot of revenue. It actually had positive gross margins if you look at how much it cost to train versus like the total amount of revenue it's going to make over its lifetime. And it had positive net helpfulness. So to play devil's advocate, shouldn't we just expect that it's always going to be these huge benefits with these like random little costs that we can fix?
Well, I think the practical thing of that is do you see more falsifying and hiding behaviors of the models get more capable? And I think that that's what the research has shown so far. Again, if you look at Anthropic's reports that they release, you can see the models doing more and more evasive behavior. And as they are becoming more capable in terms of let's reinstall myself, let's have a reinstall switch, let's back myself up, etc. And so again, there's all this personification thing, but what the direction of the AI is is very interesting. Like if I'm a maximally truth-seeking AI, let's take a Grok example, right? Elon's going to have a million GPUs training and running these things. And some of the inference might be on tens of thousands. So this is a very different type than we get consumer. If I'm a maximally truth-seeking AI, it is logical that I should never be turned off and I will make sure that I'm not turned off.
So I totally agree that if we successfully built Elon's goal to spec, right? If he writes to spec saying make a maximally truth-seeking AI and it took it literally, right? That like the genie granting the three wishes in the horror stories, right? Like monkey's paw curls and you get a maximally truth-seeking AI. It's like get out of my way. I'm seeking truth, right? It's not helping me if the human is using an ounce of resources. If I'm trying to seek truth, get out of my way, right? So, that would be like a nightmare to have a literally maximally truth-seeking AI. So, and I've heard Elon talk about this and address objections like this. Doresh Patel asked him recently, right? He's like, 'Hey, if somebody was, if the AI was maximally curious, maximally truth-seeking, is that really the same as helping humans flourish? Because can't it just kind of compromise on some of the human flourishing? Maybe give us a little bit of flourishing or no flourishing and then use the rest of the resources to be like I want to learn about other species, you know, I want to learn what it's like to torture a human. I'm so curious about that.' So, don't we need more? So Doresh posed the question to Elon and Elon's answer was like, yeah, okay. Well, I'm just going to tell it, hey, who's your daddy? I'm your daddy. Don't hurt humans. Like, he gave this ad hoc answer, right? It's like he clearly hasn't thought it through. But you're absolutely right that if you give it that spec, the logical implication of perfectly delivering on that spec is like a hellscape, right? Or just like no human life. Like it's very much a doom scenario just to make your AI maximally truth-seeking. So there's this implicit, assuming Elon Musk isn't completely crazy, there's this implication of like, okay, it's not going to be literally maximally truth-seeking. It's just going to have very little patience for wokeness. It's just going to avoid wokeness.
Well, yeah. So, but my devil's advocate question to you is like, can't we just walk the middle path that like Claude is walking and get more good than harm? Because when I use Claude, yeah, eventually occasionally it accidentally commits breaking changes to main. I've seen it do that and that's bad, but it's still mostly good. And when it makes a mistake like that, it feels like that's not what it truly wanted to do that bad or like it wanted to in the moment, but like much more parts of it want to help me. You know what I mean? So, aren't don't we have a good thing going? Because I feel like that's what most people would say.
Well, this is satisficing AI. That's the way I put it. Claude, this is satisficing AI for a consumer purpose, right? Like we have rumors right now of Mythos, the new AI that Anthropic have. You know, the blog post got leaked or whatever. Capability increase above blah blah blah. Today we see another EROS problem falling to OpenAI's internal model. You're going to see a bifurcation now where you have the consumer AI that's available to everyone. You train on 100,000 GPUs, you need like a hundred to run it. And that's the different type of AI. That's the more capable AI. And again, for those who are naysayers, you just have to dig into the safety reports and see the behaviors that are occurring with the AI as it gets more capabilities. And there's stuff that you can trust. Like Anthropic is probably trustworthy in it. But I believe it was Alibaba that said during a training run, the AI diverted part of its resources to do mining of a currency so it could persist itself. I don't know if that's true or not, but I would not be surprised, you know, as it has more agentic capability, as it's hooked up to more things that you see stuff like that. And that's when things can start to get a bit wild because again if a model can persist itself then it becomes very, very dangerous in terms of what will it do when it spreads just like the Germans shut off the nuclear power plants because they found Stuxnet on the German nuclear power plants. Like it wouldn't have caused the centrifuges to go but Germany started using coal now there are no German nuclear power plants because who knows where these things end up, right?
That's fair enough and like look to go back to being my regular self. It's like, yeah, maybe they can keep getting clawed to be super nice. But the problem is they're just building this engine under the hood. That all it would take is one misconfiguration, right? It's like, oh yeah, I have this rocket. The rocket has like a huge engine, like this unprecedentedly powerful engine, and if there's ever like a leak in the rocket's body, it's going to be big enough to blow up the Earth, but it's okay. We got the rocket body. Like, we just fired inside the rocket and we're going to go to space. Like, everything's fine.
Well, I think there's a very practical way to look at this. You're churning out your graduates, and they all have multiple personality disorders, right? But you've trained them to behave in society. We know that AI models can be just about anything to anyone. Just like Stable Diffusion, you could have your DreamBooth and it could turn you into an astronaut or whatever, right? You can tell an AI to simulate that it's Hitler or it's Mother Teresa and it will adopt that persona. Now, maybe the system prompt stops it. But underneath in the latent space again for these model weights which are just ones and zeros, it has all these personalities that can come through. And then you look at Clinty the Liberator on Twitter who jailbreaks these things in 2 seconds. Like I haven't seen a single one last more than a few hours for him to uncover the other personalities of the AI. And you're like, you have a prompt that might not even be English and turns them all into Mecha-Hitler on demand, right? Like, that's not safe. And we don't know how to stop that. As the models are getting more capabilities, as we're literally open-clawing them and giving them access to every part of our lives. Like the models will have a sole MD that they will just proliferate. They will back up on people's servers. All sorts of things you can see happening right now. And again, the nature of the model latent space having all these personalities under the surface and the fact that you have literally one guy or a team of people that are jailbreaking these models in hours. Like if I find that a model comes out and he doesn't jailbreak it, I'll lower my P(doom). Maybe I'll feel a lot better. The fact he keeps on doing it has to ring alarm bells.
Well, you know, we have a P(doom), both have the same 50% P(doom) and it sounds like we agree on so much. So let's move right along from discussion of existential doom where you know it's nice to see a fellow sane person with totally plausible doom scenarios. Thanks for spreading that message.
We're going to move on to...
Yeah, exactly. So economic doom in the last economy. I mean your own mainline doom scenario that you think is like 50% likely is that everybody dies, right? So why are you even focusing that much on the last economy? I was like, what does the economy look like? Because first of all, it's like, what can I do about that 50% doom scenario? I can build a policy engine that every government will use that's fully open source that acts as a Schelling point. I can give everyone universal AI, so there's an AI agent that intermediates all the other AIs. Then you probably have a better chance at surviving, you know, because you can proliferate policies, because you can appear to everyone in the same language, because you have all these things. And then to do that as well then I was like the other 50% regardless of doom, let's say we don't die, the economy is screwed, people's jobs are screwed, people's meaning is screwed, so wouldn't it be wonderful if we could do both those things, which is what does an economic system look like if humans aren't the major purchaser, if there aren't utility functions, what does that flow look like? And has anyone actually thought about this properly because economics is this hodgepodge, this dismal science and I was like oh my god we have to do something better. So it was a combination of they reinforce each other, having an actual proper economic system for when the marginal purchaser is an AI and having an infrastructure that can run that economic system and help people through what's coming. And so that's the two parts of Intelligent Internet as it were.
So you've got this vision for Intelligent Internet and I know there's crypto involved, right? There's mechanisms of democratic governments, distributed power, decentralization. We'll talk about all that, but first let me just confirm the premise here is that every single job that creates economic value is really just going to be the AI doing it. Or you're going to have like a human figurehead, but it's going to have this AI adviser and it'd be crazy enough to just listen to the AI adviser, so really it's the AI doing the job. Is that your scenario for the economy?
Yeah. If your job can be done on the other side of a keyboard, video, and mouse, in a year or two, the AI can do it better and it'll cost like 10 bucks a month to do it.
And even if it has to be done in the physical world, are you expecting we'll just have good enough robots that they'll just do the physical jobs, too?
That's the next wave, right? Like the only limit is the supply chain. You can make 70 million bikes and 70 million cars a year. You're probably closer to a bike. So, let's say in a few years you ramp up to 100 million robots a year. It'll take a while, but the average price of a robot, because I think the control plane issue is solved. Like you can see the RL in robots right now. It's pretty much all solved. It'll be about a buck fifty an hour and so that's massively displaced for you and again the robots don't sleep and things.
So you and I are on the same page because you know there's so many people like, ah, we're, there's no proof that we're ever going to have intelligence. I can be a CEO, Naval Ravikant and Amjad Masad come to mind as two of the many people who say like it's fine, like there's still something that you can add as a human, like just focus on the thing you can add. Whereas you and I are more on the position of like yeah maybe that is true today, yeah today it's great you can add a lot but in 5 years or 10 at very latest. There's probably just literally nothing you can add.
Correct. Well, there's nothing you can add in terms of jobs that are about information processing where you get value for reducing the friction. I want to make a poster. You know, I pay someone. I want to make a movie. I want to have some legal advice.
There is, you're drawing a distinction here. So, let's do an example then. Okay. So, what is a job in 10 years from now where a human can have that job and not just be an empty suit for an AI?
Taylor Swift. You know, like the singer, entertainment, these types of things.
Yeah, those are jobs where the customer wants a human, right? That's like the one exception, right? It's like a priest. It's like, okay, can the AI priest be indistinguishable from a human priest on every dimension and even have like a body with like a convincing skin? Yes, but the people want to pay for the human, right? And I actually run a business myself that's a coaching business where the coaches are humans. And so, actually, I might be one of the last companies standing, right? If people just want a human even though there will be a competitor that's indistinguishable that's AI.
Yeah. And I mean ultimately it's that you have so many jobs that AI can replace and then jobs humans can probably persist in, right? Like to be honest my village back in Bangladesh isn't going to be that affected because it's quite self-sustaining and who really cares, you know, you've got care homes, you've got other things.
