Today we put out the bat signal and called for an emergency pod because America just experienced an AI Sputnik moment. Kimmy K3 released yesterday, shocking the AI world with the largest open model ever, and it went straight to number one. This week they didn't just close the gap, they jumped the fence. Kimmy's always been a model that felt a bit different. That's why it was always top of the writing benchmarks, for example. K3 is actually a multimodal model. So it can have all sorts of inputs and it can understand things, which is one of the reasons it's so good at front end. Now it's a free-for-all between Meta and SpaceX AI on the American side, and now China and Moonshot is number three on that Pareto optimal frontier.
Frontier intelligence is now a totally perishable asset.
What are the American frontier labs spending their money on? I think the US government starts a strategy of constraining in some fashion Chinese open models from being used in the US.
We've had this bunch in the internet world that information wants to be free. Basically, intelligence also wants to be free.
All we need now is some kind of a global... Now that's a moonshot. Ladies and gentlemen, welcome to Moonshots, everyone. The number one podcast in all things AI and exponential. Your front row seat to the coming singularity. Maybe I should say to the singularity, which is now... to the present singularity... to the present, to the continuous singularity. Today we put out the bat signal and called for an emergency pod because America just experienced an AI Sputnik moment. But more on that in just a moment. Allow me to welcome my magnificent Moonshot mates. We have the full quintet with us here today. Alex Cap, Dave Blondon, Salem, and Emad Mostaque. I'm Peter Diamandis, your host and abundance provocateur. If your head is spinning at the pace of the singularity, good. Mine is too. And that's the point. Our mission here at Moonshots is to keep you informed, keep you up to speed on exactly what's happening. Most importantly, keep you optimistic with the extraordinary pace of change, the coming age of abundance. Gentlemen, welcome. Thanks for getting up early, wherever you might be, or Emad in your case, in the afternoon. I was up at 4:00 a.m. this morning, the benefits of jet lag, but I could have used another hour of sleep.
And I'm not... so... European siesta.
Yeah, I've got a workout scheduled right after this. A lot happening, gentlemen. A lot going on and appreciate everybody's time here. You know, before we get started, I want to personally say thank you to all our subscribers and our viewers. You know, I've had a chance, I don't know if you guys did recently, to watch and read the YouTube chat. And all I can say is we love you guys, too. You know, our mission here is delivering the news. And we spend an ungodly amount of time reviewing. You know, Salem and Alex and Emad, I got your texts this morning. Let's add this. Let's add that. So, so much going on.
I got to say also all the memes of Alex, all the memes of Alex explaining JSpace are awesome. So, keep memeing Alex every time you can.
Yeah, for sure. And some great appreciation.
See if you can figure out my JSpace.
Yeah. Well, can we look inside? We'll be able to see.
Yeah, we're going to get a readout. And see, a lot of love for you on the comments as well.
There's some wonderful people out there. You know what's incredible is most YouTube videos, they're just a kind of a flamethrowing festival and ours are completely the opposite. It's really amazing. So...
Well, no, I just again just absolute gratitude and I appreciate the fact that everyone, all of our subscribers and viewers here take the time to listen to the pod. And you know, we're constantly, we spend so much time with our entire team and the entire Moonshot Mates here just really trying to assess what's going on and deliver it. And we have these emergency pods. So, if you haven't subscribed and turned on notifications, please do. Gentlemen, shall we jump into the first story? It's a big one.
I'll just note that if we do enough of these emergency pods, at some point it turns into Moonshots Daily.
Yeah. Or continuous. I still think moving into an Airbnb together and just turning on the camera...
All right, let's jump in. We've just had a Sputnik AI moment that's waking up the US Frontier Labs like a quadruple espresso shot. Kimmy K3 released yesterday, shocking the AI world with the largest open model ever, and it went straight to number one. A little backstory here. Kimmy K3, and Kimmy is from Moonshot AI, a Chinese lab. And over the last year, they've climbed the leaderboard. They put out K2, K2.6, K2.7. Each one closing the gap against Anthropic and OpenAI. This week, they didn't just close the gap, they jumped the fence. Overnight, they released Kimmy K3, and it's a monster. 2.8 trillion parameters. And you got to remember the context here. China is doing this while under US export controls intended to starve them of the most advanced Nvidia chips. That's a big deal I want to discuss with you guys. They've completely engineered around the compute wall and K3 jumped 17 places to the previous Kimmy model, blasting past Claude Fable 5 to land as number one on the front end code arena. K3 has also ranked number one in six other domains: brand and marketing, reference-based design, data analytics, consumer products, simulations, and content creation. The full model weights are set to drop around July 27th, which means anyone on Earth will be able to download and run this on their own prem. How big a deal is this, Alex?
I think it's great for competition. Let me first as a preliminary matter point out some things that have perhaps been slightly less obvious in the coverage, the meltdown if you will over K3. The first is as Moonshot points out, they claim in nine of the past 12 months that Kimmy models, Kimmy model series have held state-of-the-art among open-weight models. So if that claim is indeed true over the past year, it's been basically Kimmy all along. I think that's very interesting. Secondly, taking a look at the published architecture since we haven't actually seen the open weights yet, but they're promised later this month. There's no magic in it, and that's pretty striking. One can imagine that behind the scenes in Anthropic or OpenAI that they've somehow, Sam Altman continues to tease at this, that there's some post-transformer architecture lurking behind the scenes achieving all of these performance breakthroughs. But taking a look at the published K3 architecture, there's no magic. It's still essentially a transformer. They've made obviously a number of innovations, but well-understood innovations concerning how they do mixtures of experts, how they linearize attention. They have their own special Kimmy brand of linearized attention, but it's still basically a recognizable transformer. And I think that the fact that a recognizable transformer-like architecture can almost match GPT 5.5 Max on the task-cost frontier, which we should probably throw up a slide for you. I think that's pretty striking. That does raise the question, what are the American frontier labs spending their money on? If you can just use a transformer to get this close, not like it's already on the cost frontier, but you can get close, like third place on the state-of-the-art for overall AI performance. What the heck are the American labs spending all of their money on? So, I derive great comfort in at minimum knowing that the transformer architecture is still alive and kicking.
Emad, your analysis here because you've been tracking this. We've been going back and forth on WhatsApp together.
Yeah. No, I mean I think Kimmy has been top of various benchmarks. Again, you can pick and choose. And they have had the largest open-weight models out of China regularly ever since they almost kicked off a year and a bit ago. I think as Alex said, the architecture isn't anything super novel. Like there are improvements like their Muon scaling that they did with UCLA and kind of other things. And they've actually been releasing breadcrumbs of all of these parts. I think what's key here is the underlying data. Kimmy's always been a model that felt a bit different. That's why it was always top of the writing benchmarks, for example. And what they've done here seems to be something extraordinary, which is when GLM came out, it's a fantastic model. It wasn't quite up to frontier, but it was text only. K3 is actually a multimodal model. So it can have all sorts of inputs and it can understand things, which is one of the reasons it's so good at front end, although we wouldn't have expected, again, it's number one in front end versus everyone. And I think this comes to something which I've said before, which is building great solid models is cutting-edge manufacturing. Like again, you will have algorithmic improvements and there are all sorts of things coming, but why are Chinese EVs better than Fords? This actually feels like the same thing. Right. Like they build, it's engineering, but it's also like the number one car here in the UK last month was the Jaiku J7 or Teimu Land Rover as it's been known. It comes out fully loaded, full spec for like 50K, you know, a third of the price. And this actually feels something very similar. They've known what the ingredients are, the raw materials. They're now putting in an incredibly consumer-friendly way. And they're just executing that manufacturing process with what they have because when you look at the architecture, you look internally, they're still on H800s, you know, they're like a couple of generations behind on the Nvidia chips, but then they built it to take advantage of Huawei and Alibaba's next-generation chips, which you can see by the static shapes and all sorts of other things as well. And they're just relentlessly going at the engineering and the usability, which is why the front end code I think is the one where they're standing out because they're just like, how can we make it have the most amazing outputs, a personal website to a game to other things, whereas the US labs are maybe looking in other directions and focusing a little bit on different things.
Yeah. I mean one question real quick is we've always talked about do we need another breakthrough beyond LLMs to get to AGI? Does this give you comfort that we don't need another breakthrough to really move forward?
Again it comes down to definition of AGI, right?
Yeah, of course. Don't get me started.
Six years ago, Peter. It was 6 years ago.
I mean, I guess the question is there's plenty of headroom still to progress these models.
Well, I think you have the base model here, right? But then you've got all these amazing harnesses that are coming out and the way that you're using the model to go back on itself. One of the things that's in the Kimmy blog post is that it actually designed a chip for itself for its next generation and it designed its own kernels for running as well. And so you move from this model weight to this whole ecosystem that the model itself builds that feels AGI-ish, right? That feels like recursive self-improvement. That feels like the ability to learn and adapt new skills dynamically by changing itself. So I think for most definitions of AGI, we probably don't need something new to optimize and make it super efficient. Yeah, there are various ways even what we know it could be more efficient than what we have here. Just we don't have enough quite compute for it. And new architectures could push us even further.
You could say attention is still all you need.
I like that. So, Alex, we've thrown up here the performance charts and we see Kimmy K3 sort of topping the charts in a multitude of places. I don't know if you want to comment on this and I want to pull you into...
If we could throw up the AI scatter plot, I think is probably the most instructive one. So, this is from the Artificial Analysis Intelligence Index and this is of all of the charts at this point. This is my favorite one because this one actually shows the cost per task as defined by AI versus performance frontier. So, one can sort of mentally look at this for those who can't see it. We see the frontier as sort of a jagged frontier going from lower left to upper right where in the upper right we see maximum cost per task and maximum overall score is still Fable 5. And then riding the Pareto frontier down and to the left from that we see number two on the frontier is still, as of a few days ago, GPT 5.6 Solve Max and now for the first time Kimmy K3 is number three. It's on the frontier. It's number three both in terms of raw capabilities and also the third point on the optimal cost-performance frontier. And I think that's totally striking. We went from a world where, as we mentioned a couple pods ago where there was this OpenAI-Anthropic duopoly to now it's a free-for-all between Meta and SpaceX AI on the American side joining the upper end of the Pareto frontier and now China and Moonshot is now number three on that Pareto optimal frontier. And that's so exciting for any enterprise to the extent it's willing and able to use a Chinese open, soon to be open-weight model to control more of its own destiny. I think this is just such a boon for enterprise sovereignty. It's a boon for competitiveness. We're living in the AI version of For All Mankind where the Soviets landed first on the moon and now the space race never ends. The AI race is now no longer ending with a duopoly and I think that's a total boon for the future.
Amazing. Dave, let me pull you in here. What are your thoughts?
Well, you know, Peter, you called it a Sputnik moment. If anything, that's an understatement of the implications of this. It's, you know, we had that Alex Karp rant on the podcast last week where he was saying look, you can't as a large enterprise, as a government, you can't just throw all of your proprietary weights, your proprietary alpha, all of your intellectual property over the wall to Anthropic and make that the basis of your whole future. But he didn't give you a roadmap to move forward. Here we are just a week later and it's suddenly a free-for-all. As Alex was saying, a free-for-all where anyone who reads these weights has the ability to get very close to the frontier and then fine-tune for any vertical use case beyond the frontier. And so it gives everybody in the world, every corporation, every government in the world, a way to catch up to the frontier without going through the US AI models. So, you know, Sputnik, yeah, Sputnik times infinity essentially. And the thing I don't like about this particular chart is because the left index goes to 100% and when you chart it out over the next two years, it looks like an S-curve. And so we're in this really steep part of the curve right now, but it implies then we get to 100% and then we've achieved the end. But this is actually an exponential where intelligence goes to infinity. So the benchmark saturates, but intelligence itself goes to infinity. And so now it's really, really clear. Just for everybody, you know, the way this works typically is nested S-curves, right? One particular technology tops out, but it builds the next technology that then begins its exponential ascent and so on and so on.
Exactly. Exactly. Right. Let me just say one other thing. You know, Alex and I have spent a lot of time working on this Keller Jordan speedrun. We talk about it a lot. It's a way you take a GPT-2 class model. You can find it online very easily. Just, you know, look on GitHub. Look up Keller Jordan speedrun. And it's a whole bunch of hackers and AI researchers who are continually trying to take GPT-2 way back, you know, five years ago...
In the form of Andrej Karpathy's nanoGPT in particular.
Exactly. And try to recreate it faster and cheaper, faster and cheaper. And if you look at the innovations in that repo, they've been able to cut the original cost of creating GPT-2 by 99%. So now it's 1% of the original cost.
And so everyone, you know, kind of doesn't pay attention to it because it's GPT-2. And up until today, it wasn't clear whether those same ideas would apply at frontier scale. Now, it's really clear that when Elon Musk takes his 16 billion Colossus 2 data center and builds a 10 trillion or 20 trillion parameter model for billions of dollars, there is a 1% cost version of creating effectively the same thing. Nobody knew until Kimmy K3 whether that was going to work or not. And now it's really clear that it does work. And so we're looking at, you know, 100x kind of innovations in the software stack, in the kernel optimization, in the mixture of experts, these fundamental breakthroughs that come out of China are giving them, you know, 1% cost. So, you know, I think Emad gave a great analogy to the car where you can get a virtually identical car for about a third of the price. Here we're talking about less than 1% of the price to create the equivalent product. So Sputnik, yeah, that's the understatement of the century. This is just... and that's why we're on the emergency pod today.
I have three points to make. I think it's not so much that Kimmy's beaten etc. Whatever. It's the fact that frontier intelligence is now a totally perishable asset. Like the shelf life is weeks now for anybody that gets to the very edge and any enterprise or government interested in that very latest cutting frontier model doesn't have time to actually evaluate it, do an RFP, look at other models, have a committee internally, think about whether to deploy it, and now you're three generations ahead in the model anyway. So now all the value comes in the architecture that can swap models, right? And that's going to be the next layer, we call that interfaces in our the Exo world. That's going to be where all the value resides going forward.
We need a new term for that. Maybe like the frontier liberation front.
Let me say one other thing for the hypergeeks out there. Emad said the Muon optimizer, but he said it very, very quickly. And anyone who's an enthusiast, look that up as well. Because one of the reasons this is happening is because when we built these original models, the very large-scale models, we took, you know, 20, 30 trillion tokens from around the internet, every word ever written by humanity, and just dumped it into the training set and said, 'Here AI, become intelligent given all of this information.' But when you look under the covers, the vast majority of that information is Taylor Swift's concert coming up and their wedding, like it's a whole bunch of stuff that doesn't actually drive the intelligence of the model significantly.
Yeah. Yeah. It's very true. A lot of those tokens actually might slow down the training, not accelerate it. And so purely by pulling out the garbage and stripping down the training set to the relevant subset, you can, it still taxes the model just as much but it reduces the number of flops, the amount of computation that the model's doing to get to the same level of intelligence. I don't think we're anywhere near done with that problem yet. So you can expect more 10x's to come out of just the Muon optimizer process and the training data set getting stripped down process. I threw up this tweet from a guy named Allaric that I found fascinating. It's for those not viewing this. It says from Anthropic, quote, 'Fable is an agentic coding superweapon capable of developing cyber and bioweapons at unprecedented speed and scale. We cannot in good faith release it without guardrails.' Right? This is the conversation a month ago. And China comes back and says, 'Laughing my ass off. Here's Fable but open source. Good bleeping luck.' So I am curious, how do you guys think about that fact that, you know, we were so constrained because of the guardrails and here's an open-source equivalent of Fable.
Well, the Frontier Labs have a major problem. They've got three fundamental massive constraints that they can't get around. One is compute and the availability of chips and all the electricity and power that's needed. The second is frontier open-source models that are as good as or in many cases substitutable without much notable difference. And the third is you've got government coming down on you going, 'We need to check before you release anything.' I'll make a thumb-in-the-air guess. The trillion dollars that OpenAI might have been worth shrank by about 50% when the government said we have to review all these models because now it's going to take time to get things out. I think this crashes it by another 50%. I would put the finger-in-the-air value of these Frontier Labs at about a quarter of what they were three months ago.
If that, I mean if I don't have to spend the money for the API calls and I can just use Kimmy K3 on my on-prem, why would I spend the money? Are they going to be hit by, you know, massive reductions in revenues?
Yeah, I think there's a couple of things here. Number one is reduction in revenue. Why do people pay for IBM? You know, why do they pay for non-Chinese cars for mission-critical things? I think having US on-call entities where, you know, things aren't going to go wrong will still sustain for a while. So I think revenues will still go up for OpenAI, for others, and this is why they built these forward-deployed engineering companies as well. And so I think they've still got a way to go. But, you know, you have the substitution effect again. This is just like Chinese industrial substitution. Why can't America build industrial things? Why do you have Chinese? Sometimes you buy Chinese, sometimes you buy American. And I think we'll see that at least for another year, but then it gets difficult. On the cyber attack security theater kind of things that we've had, you know, I've maintained that we would get to this point. And what does it mean? It means the only form of thing that you can actually do is cyber defense. Like this must be the absolute biggest category in VC right now. Like if you're a talented Stanford, MIT grad, build a cyber defense startup that goes into cutting-edge and every other company and says, 'Let's use this technology to defend against what's inevitably coming,' because the proliferation of these capabilities is going to increase but not quite as fast as we think because what actually happens, and you know, we've done some tests around this, is that GPT 5.6, the cyber version, Fable, etc., are trained on lots of CVE and cyber data. The Chinese models don't actually have that much of that, so they're not that great. But someone can train that data if they have it into there. And so we'll probably see cyber attack-capable open-source emerge, I'd say, in a quarter or two. So there'll be a bit of a lag there, but definitely for the types of big adversaries, it's going to get a bit crazy.
Dave, you want to jump in?
Yeah, for sure. I think we glossed over recursive self-improvement there. Peter, you asked the question of, you know, is this the tipping point. The view of the US government, we always knew it was going to be too late, right? It just moves too slowly. But the view was, look, when we get to a model that's capable of building itself, building the next model, we're not going to let that go out to everybody in the world so they can catch up overnight because there's never been a product in the history of manufacturing like a car. If you have your state-of-the-art car and you give it to a foreign government, they can't use it to make a better car. But AI doesn't work that way. If you have state-of-the-art AI and you give it to a foreign government, they can use it to actually catch up to you and create state-of-the-art AI. And that became clear to the government what, a month ago, month and a half ago, that Fable 5 was over that line. And so they stopped it. But the reality is that Opus 4.8 was over that line. And, you know, people in China could use Opus 4.8 to create Kimmy K3. And so that recursive self-improvement line was actually crossed earlier than Fable 5. And that's going to be obvious to the world now because all you need to do is have an AI that's capable of improving its own kernel. It doesn't have to. This is a point, you know, I've made on a podcast like months ago. People think that RSI is going to trigger when it's Einstein-level intelligence, but all it has to be able to do is improve its own kernel and get a 10x step up in speed, which nobody perceives that as being true AGI, but that's all it needs to accelerate itself by 10x. And then the 10x smarter or 10x, you know, higher parameter model will be some level of intelligence higher. A lot of people in academia were saying, well, look, we're getting diminishing returns with the parameter count. So a 10x faster model won't natively be 10x smarter. But that turned out to be wrong. You know, we're not seeing, we're seeing slowing, but we're not seeing flattening of the intelligence curve. So all the evidence now is that if you boost the raw speed by another 10x, you're going to see genius-level AI. And then that genius-level AI will boost its speed again. So I think when we look back on this in history, we'll say right around Opus 4.8 was the point where the little spark was enough to ignite a flame and then a flame can become a fire and then a fire can, you know, become a sun. And that's I think the way we'll look back on this moment in time. So the cat is definitely out of the bag. The current policy, the current US policy of constrain the next model, there's no way that's going to contain global and corporate proliferation of frontier AI.
Do you think the US government starts a strategy of constraining in some fashion Chinese open models from being used in the US?
Well, you know, in two weeks these weights are supposed to be open-weight, open-sourced and then we'll see. Like this probably, if they're rational at the White House right now, they're spending every minute in a debate on do we negotiate with China immediately and not release those open weights. And I really doubt they'll move quickly enough. I'm sure they'll, well, I'm not sure. We'll see what happens in two weeks.
Can I merge two ideas here?
You know, Peter, you talked about exponentials and the law of accelerating returns, right? I think it's worth drilling into that because if you connect that to what Dave just said, this is why we've been saying forever and a day on this podcast that this is unstoppable. Ray's original observation was once you have an information-based paradigm, you just keep hopping across multiple technologies. So we had vacuum tubes, relays, and then vacuum tubes in computing. At some point, you can only fit so many vacuum tubes into a room, but that architecture was used to design transistors. Transistors were used to design integrated circuits and you get these nested S-curves. And so what Dave is talking about is as these architectures, all the various pieces of the puzzle get all reinforcing loops inside them. Each of those is like an S-curve that starts accelerating the collective and it's unstoppable. And so it doesn't, there's no limit to where this goes. And this is why people are so kind of freaked out about the upper end limit of this. So important to connect those two dots.
Yeah. If I just say something, Peter, I'm just following on from Dave. So there was an important speech by Xi Jinping a couple of days, yesterday, God time flies, at the World AI Conference in Shanghai where he basically said, 'We are going to fully back open source as a public good for humanity and they're not going to regulate and stop it.' This is their plan. It's great for China for a variety of reasons. From the fact they have a billion people whose IQ is about to increase, you know, by having these tools. From the fact they need robots to solve their demographic thing and the soft power from putting a Chinese educated brain, a Tsinghua graduate, into every critical system in the world. But they're going to keep on doing that because they actually have a regulator. And from talking to some of the Chinese labs, it used to take 60 days for a model to be approved. Now it's like a week.
Amazing. You know, and just also announced a regulatory body that they've created which includes Brazil, different parts of Asia and Africa. I don't know if you guys saw that.
I saw that. Means obviously the new Belt and Road is now focused on AI coming out of China. It's a bizarre future where the Chinese Communist Party is saving American capitalism from itself.
It's so true. Let's also note that Yang Xilin was a CMU graduate and we could have given him a visa to stay.
Yeah, we're going to get to that story in a second. See, I'm just, this is an interesting chart here that shows the valuation. So Kimmy's valuation or Moonshot's valuation, Moonshot AI valuation is at 20 billion as compared to Anthropic at a trillion and OpenAI basically at a trillion as well. If they were public companies today, I think you would have seen like a 30% stock valuation drop.
I'll ask again, what are the American Frontier Labs doing with all of their capital?
Yeah. What are they spending their money on?
Actually, if you go into the buildings and talk to them and you have any idea at all, they'll give you the capital. They're desperate for more smart people to help because they're trying to deploy and change the world at this insane pace no one's ever experienced before. And they want to deploy that capital much more quickly than they can find smart people who have good ideas to use the capital. But it's a great point, like, you know, you're sort of saying it in an accusing way like what are you guys doing with your capital, but no one in the history of the world has ever had this much money pour into their building this quickly with no prior business experience. We're talking about CEOs that have never run a company before. It's like they're trying but I mean seriously can any human being really rise to the occasion of AI that quickly? So, but my point there though is if you're smart and you have good ideas, get into those buildings and propose your ideas. This applies to XPRIZE too. You know, they are desperate to move that money out the door into something productive that gives them a sustainable barrier to entry.
And I also think that the Frontier Labs are also asking themselves that question and asking the US regulatory apparatus that question. Anthropic regularly is sending out smoke signals accusing various Chinese frontier labs of distillation attacks and maybe in Anthropic's public mind, that's how the Chinese labs are able to do it through distilling and capturing reasoning traces. But honestly, like looking at the K3 performance, I'm not at all convinced that Moonshot is achieving their performance purely or even substantially through distillation attacks on Claude. It just doesn't smell right.
No, no, I totally agree. I think though that there's a tendency to underweight or undervalue the existence proof, like just purely the knowledge that a highly scaled transformer running with a Muon optimizer and simplified data, knowing that that works gives you a much more refined roadmap. You don't have to copy, you don't have to cheat, you don't have to steal every trace. You just have to know that that formula works and that cuts your R&D costs by 90, 95%. So I think it's just that simple. There's nothing sneaky or cheaty about it. It's just knowing you're on the right path.
I have the greatest value creation idea for ourselves ever.
Which is we, in 9 days when they drop their open-source weights, we release an open-source model called Kimmy 4 under the Moonshots podcast name and IPO it.
And instantly we'll be billionaires. So, you're saying what's better than one moonshot? Moonshots plural.
Well, you know, why not copy the copers? Let's go.
That's good. Actually, I think there's a good, I've got a good analogy for you, Dave.
Why do Americans pay more for drugs than everyone else? All the R&D happens in America. You pay the premium just like tokens premiums. And then what happens? You have generics elsewhere.
Yeah, that is a good analogy because that's like a 99% cost cut. It's much more akin to AI than cars. That's a great analogy. You know, Gavin Baker, our friend, wrote a brilliant post. You can find it on X about the implications of this for businesses.
And essentially, must read. Absolutely. But essentially all businesses, all stocks other than the Foundation AI labs are huge beneficiaries of this. And then, like you said earlier, the foundation labs are like, well, what's your future? What's your revenue model? What's, you know, why are you worth a trillion dollars? I don't quite get it. So you should see a really big reshuffling of valuations in the next week based on that observation. And then any corporation that has its technical act together, you know, there aren't very many of those, but any, you know, if you're a bank, but you happen to be a very good bank with brilliant IT and technical skills, or you have great partners and great vendors, you now have a clear roadmap to controlling your own destiny with your own AI, your own like JP Morgan AI. And so I suspect the markets will react to that if you, you know, put your hand up and say, 'Hey, we have a way to do this internally. We know how to do this with, you know, with our partners or however you get it done.' This is why we call it the organizational singularity.
And we're still seeing everybody who's using or trying to use Fable 5 getting downgraded every time they mention biology or mention something that is potentially on the edge. And why would you tolerate that, you know? So, in 9 days, what do we see? Do we see every, I mean, I'm as soon as it's available going to upgrade. I'm running Kimmy 2.7 on my Mac Studios. I'll upgrade it to Kimmy 3. Everybody will. So do we start to see sort of the wholesale US entrepreneurial base of capabilities on K3?
Well, I think this, I can think of an analogy of this which is Stable Diffusion. When we released Stable Diffusion, god, four years ago, time flies, you had these really restricted image generators that were a bit better but they were restricted and they had all sorts of arbitrary restrictions because obviously it's a bit dangerous to have it. You couldn't have likenesses. There was no way to get IP in there even if it's your own IP and more. And what happened? 100 million, 200 million downloads and a whole ecosystem that built around that and accelerated generative media as you said. And why are you going to have this model like I can't even talk about philosophy with it, it downgrades me, right? Like when you can have the fully open variant of it at even a fraction of the price that you can then customize, a whole ecosystem will build around this and other models and it has already been doing so and that's a real danger versus being locked into the single vendor.
Which is why I think the labs will go vertically integrated. Like all their customers are now going to be their competition and they're going to be like, 'Okay, I'm going to take you all on.'
Well, and that directly ties to Mira Murati and Inkling. Are we going to talk about that story, too? That's huge this week.
Well, we talked about it in the last pod, which was so two days ago.
I mean, and Mira just released Inkling, which is fantastic to see a US open-source lab. But the question is how many more will we get?
You know, how many more open-source, you know, sort of shocking Sputnik moments are we going to see? I mean we have a lot of Chinese labs pursuing beyond just Moonshot.
Yeah. So Inkling, it's just really telling about where things are going to go because it's designed for you to pick it up as a corporation and fine-tune it within your corporate walls to whatever your use case is. So if you're a biotech lab and you're researching and you don't want everybody to see your proprietary data, you take Inkling and you tune it internally. But the reason that's telling is because Mira Murati came from OpenAI. So if she didn't believe that pathway was viable, she wouldn't start, you know, say she wouldn't start Thinking Machines around that thesis. So it tells you that the people that are inside the best frontier labs believe that this process can catch up to the frontier. So you combine that with Kimmy K3 proving it and it's a different world next week. You know, the other thing that was weird in the market at the end of the week is that things started to reshuffle pretty dramatically toward the end of the week. But in the downdraft, the semiconductor companies also came down, but they're actually going to go the other direction. And this is the point Gavin Baker was making that this drives up the need for silicon, not down. It changes the whole software landscape tremendously. But silicon is going to be more in demand than ever before and completely sold out, you know, as we know.
Can we talk a second about the Nvidia embargo that we put for China. So here we see the highest performance models. Was the whole Nvidia, you know, sort of regulatory embargo unnecessary? Did it do what we've always done before which is just spark China's need to develop their own capabilities with Huawei?
Of course that's what happened. Of course we did everything that the embargo only incentivized the Chinese frontier labs to develop and cultivate new efficiencies that by the way were always there. To Dave's point earlier about the nanoGPT speedrun, there's this enormous overhang that isn't fully exploited in terms of leveraging algorithmic and computational and hardware efficiencies to train larger and more capable models. And all these export controls do, I think, is incentivize the Chinese labs, which are already feeling plenty of demand pull to compete with Western frontier models, to leverage those efficiencies sooner. And maybe on balance, although it's superficially bad for the West now that we've incentivized this new generation of much more efficient Chinese frontier models, in the end, I think it's net good for not just the world, but also for the US to have this fire lit underneath them by Chinese competition that's much more efficient, much more capital efficient, more weight efficient, probably more bit efficient. This is all a net positive as long as the US in my mind does not set up or fall into some ultimately protectionist regime of trying to prevent what may be construed as Chinese superintelligence dumping on the US.
As long as we avoid that, it's great.
Exactly what happened. That's exactly right. And I think the US learned a really important lesson in the Vietnam War. And then, you know, because that's over 50 years ago now, it's been forgotten again. And then you have to be reminded again. But in the Vietnam War, it was really clear that either you go to war and you win quickly or you don't. But what you don't do is send in a few troops and then send in a few more and then creep in and like you, nothing good comes of that at all. The embargo of chips on China was totally harebrained because it was enough to irritate but not enough to actually work. It's just the worst-case scenario and it sparked exactly like Alex said a huge amount of quantization research which is critically important and under-discussed that allows faster performance on cheaper chips and those innovations don't go away, you know, that's going to be around forever now.
I think we've got a, yeah, completely self-contradictory but it has some interesting outcomes. So the total amount of compute used for Kimmy K3 is the same as Inkling.
And you can tell that because it's the amount of dense weights and roughly we assume about twice the number of tokens trained because we don't have it. We're like, but how does that work? Well, you look at their architecture and it's a two and a half times in data to intelligence conversion through the advantages and data mix that they have because they've had to operate in these constraints. And we see that because the first model isn't as good as the second model. And for Inkling, you're going from a trillion parameter model to a 300 billion parameter model about to be released, which is actually better performance. So you see this with the labs and these labs have had to deal with the constraints.
But here's something really interesting I think. If you look at that slide Alex loves and we kind of chuck it up on the screen. So what they've had to do is they've had to optimize their inference for Huawei 910 Ascend chips, for the new Alibaba chips and others, 64 nodes in one because this is a big model like you're going to have to buy another Mac Studio or two, Peter, to serve this, you know, it needs like two terabytes of RAM. So you see where Kimmy K3 is there, that's because they can only use Chinese silicon to run it. They don't have Blackwells, they don't have V Rubins. V Rubins and Blackwells are designed for these really large models that have really small things because it's 50 billion active parameters against 3 trillion total. American companies like Modal, like...
Fireworks, like B10, will be able to serve this model 10 times cheaper than their Chinese competitors because they have access to the Nvidia and AMD big chips. And so like I said, it's a bit ironic where the development R&D suddenly has gone there, but there's going to be a 10 to 100 times price drop once this is optimized for the next generation via Rubin.
Well, that also, Emad, you're saying essentially the same thing, but that also unleashes a bunch of chips that aren't currently in circulation. They're underpriced, and also a bunch of fabs that can't make a GB300 but they can make an inference time chip that'll run the cheaper Chinese or the lower granularity Chinese model. So a lot of capacity for compute gets unleashed through that same process you just described.
If I could go up a level and go a little bit woowoo, right? We've had this mantra in the internet world that information wants to be free, right? Basically, intelligence also wants to be free. Okay, and essentially we've gone over the course of evolution from biological intelligence where you had evolution built in recursive improvement, and then we broke through that to individual intelligence to the person of a species to collective intelligence like markets or networks. And now we have AI which can scan across all the data to create a whole other level of intelligence. So this is not stoppable. And so any entity or domain or government or whatever that tries to constrain it always, always, always, always fails. And so it's just a fundamental law of nature that you cannot constrain this and it's just not possible. Why people bother is what really blows my mind. It's a very scarcity mindset to try and think about it this way. The faster we get to better intelligence, the faster we get to abundance, the faster we don't need to fight over anything.
I can't disagree with you, Salem. I wrote an entire paper arguing intelligence manifests in the physical world as maximizing future freedom of action. So here's to the Right Frontier Liberation Front.
Yeah. Well, you don't want to you don't want to.
Now, you remind me of the Monty Python thing where there's the Popular People's Front and the People's Popular Front of Judea.
This episode is brought to you by Blitzy, autonomous software development with infinite code context. Blitzy uses thousands of specialized AI agents that think for hours to understand enterprise-scale code bases with millions of lines of code. Engineers start every development sprint with the Blitzy platform, bringing in their development requirements. The Blitzy platform provides a plan, then generates and pre-compiles code for each task. Blitzy delivers 80% or more of the development work autonomously while providing a guide for the final 20% of human development work required to complete the sprint. Enterprises are achieving a 5x engineering velocity increase when incorporating Blitzy as their pre-IDE development tool, pairing it with their coding co-pilot of choice to bring an AI-native SDLC into their org. Ready to 5x your engineering velocity? Visit blitzy.com to schedule a demo and start building with Blitzy today.
Let me bring up a related subject to the story here that I have a pet peeve about. And it's this one. So, you know, the founder and CEO behind Moonshot AI, Yang Xilin, didn't learn his craft in Beijing. He earned his PhD at Carnegie Mellon, one of the best computer science programs in the world in Pittsburgh. And we basically trained him up. We admitted him, trained him up at one of our best institutions and then when he gets his PhD, he doesn't get a green card, he goes through the hassles of trying to get a visa and he goes back to China and he builds Moonshot AI there. Just a moment and talk about, I've stated publicly so many times that I think when anybody gets a PhD, they should get a green card stapled to the back of it. Why are we sending the most brilliant people who come here to get educated back home? Whether it's to China, whether it's to India, whether it's to Brazil, why don't we enable them to stay here and build? Gentlemen, comments on that.
Okay, so I did some research on this and I think the story is not what it seems to be. So a little bit of chronology first. So Yang Xilin, according to my research, he starts his PhD after undergrad in China, starts his PhD at CMU in fall of 2015. Okay. Then approximately one year later he founds a startup while a PhD student at CMU. The startup is named Recurrent AI. Where is Recurrent AI based? It's based in China. It's not based in the US. So one year into his PhD program, he starts a Chinese AI startup while still doing his PhD at CMU. That's interesting and that's a problem. This also runs counter the sort of a narrative violation for, oh, we wouldn't staple his visa or whatever and then he goes back to China. No, actually one year into his American PhD program, he starts a Chinese AI startup. Then he graduates in 2019 is my understanding. My understanding is he had offers from Google, Facebook, Huawei and others upon graduation in 2019, but he goes back to China because that's where his startup Recurrent AI was actually incorporated a few years earlier. And so I don't think necessarily this is the case where either the US was unwilling to retain him or even President Trump somehow through some policy was driving away this particularly talented Chinese graduate. He started his company during the tail end of President Obama's term in China.
And Alex, I appreciate the deeper dive that you did. Thank you for that. The point still stands and you know, see, you and I have seen this so many times right at Singularity University. Dave, you may have seen this at MIT. I mean the fact of the matter is a lot of the most brilliant students aren't given the opportunity to stay and develop here. Dave or actually Emad, what are your thoughts on that being someone not in the US?
So if I can just give my two cents. Yeah, I completely agree with it and here's this crazy thing. The math and the numbers are all there. What is the value of a PhD staying in America? It's actually quantifiable and there've been multiple studies on that. You know, Dave does a great job obviously of converting them into startups into innovation. And then there's the other thing that shoots in the foot, which is American companies can't invest in Chinese companies because of regulations and other things as well. Some of them are Chinese, but look at the trouble that Benchmark got in for investing in Manus, for example. So, I think it's kind of twofold, but I completely agree that if you've created or contributed to creating a valuable asset, most foreigners stay in America after they do their PhDs, but too many don't have a very direct path despite the math proving that they will add value to the American economy.
Yeah. I mean, another point just to make here is, you know, the AI race isn't only about chips and compute. It is about people. You know, key people are still driving the greatest value at least for the moment.
I would argue not just people but also to my earlier point it's about where the startups get domiciled. There is an alternative world where he through whatever immigration-oriented regs was deterred from starting his first AI startup in China while still an American PhD and we incentivized him to start Recurrent here in the US and I think there's maybe an alternative counterfactual world where Recurrent was American and then its arguably intellectual successor which is Moonshot AI also remained domiciled in the US and then he followed his own startup to stay here.
Yeah. Yeah. One thing that came out of the story is when people come from India to get educated in the US they overwhelmingly stay. When people come from China about 80% of the time they go back and it's just a difference in the local economy. You know, there's going back to India to start your company is a non-starter, is just so unlikely to catch. But going back to China, it's a thriving ecosystem, lots of support. So going back to China to start your company is actually not a bad plan for a lot of people. And so I didn't realize that until this report came out. But you know, as Peter was alluding to, we have tons of friends from MIT that came from China. And I don't want to put them all in one bucket because there's a really clear distinction to me between people from China that are Hong Kong, Taiwan, whatever that come over that don't really align with the Chinese Communist Party at all. In fact, they kind of hate it. And then you've got, you know, Chinese people that come over for an education. And in one case at BU, a very good friend of ours is the dean of computer science at BU. And there was this massive crisis because there's a concerted effort by the CCP to plant specific students into BU to gather specific knowledge. And they were given tasks. You have to go study this, learn it, and then send it back. And they didn't know what to do at BU. It's like, you know, these are effectively trained spies that got into our PhD program, but we weren't ready for it. What are we supposed to do? And you know, they want to be highly ethical so they don't want to just dismiss the students. So I don't know how they resolve that. So you got this really like that's a very different thing from the bulk of Chinese students who are, you know, they don't align with the CCP and they just want to thrive in the world and they're happy to start their company here or anywhere else. And don't forget when we looked at the frontier labs, I mean originally in the early days of xAI for example and in Meta like 50% of their research staff of their research PhDs were Chinese Americans, an extraordinary and the Chinese, you know, the Chinese every year in the math olympiad are at the top of the scoreboard. There's an incredible wealth of capability here that I think most companies desire to retain inside. Salem, you were going to say.
Yeah, two things here. One is, you know, the asymmetry of the talent I think is the really important part here. I made this point a couple of podcasts ago. 70% of the elite AI researchers are not US citizens. They're in order Chinese, Indian, Taiwanese and UK. And so, that's a huge problem. Stapling in a green card is the easiest thing we could do with zero friction to then give them incentive to stay here and build here. The US is massive asymmetric advantage for the rest of the world. It was better to build here than anywhere else in the world. And that's starting to become less true. And that's why people are going back to China, go increasingly back to India even to do things despite the kind of the friction that exists trying to do something in India. That is the part, the failure of the US to fix immigration is one of the biggest problems this country has right now.
Amen. All right, I'm going to move us forward here. I just want to put up this slide. You know, since mid-April we've seen 13 new frontier models launched an average of one every 10 days. Just, you know, comparing this to 2025, we had eight frontier releases over the course of a year, one every 50 days. A year earlier in 2024, we had six releases, one every 60 days, and it doesn't seem to be slowing down. And then, Emad, you sent me this morning this tweet from Elon. Thank you. I put it up here. This is Elon's tweet. Our two trillion model, which is better than our 1.5 trillion in every way, will finish initial training next week. It might be able to exceed Kimi, but with speed and token efficiency close to our 1.5 trillion, aka Grok 4.5. So, I mean, this is the number one piece of evidence that we're living in the singularity. The speed at which this intelligence is accelerating is insane. Peter, it gets better if you take Salem's list of frontier models and the dates and you regress an exponential curve to the predicted frequency or time period between model releases, which I did just as an exercise, you find that at the present rate, we're going to get to daily frontier model releases by January. By January, we're going to see daily new frontier model releases if this exponential trend continues, which basically implies continuous versioning.
So, I guess the question is, what does that really mean, right? What does it mean to have a new release? If it's a continuous process,
I mean, maybe it means that we'll have to do our daily Moonshots episodes about something other than point releases from the Frontier Labs. We'll need something new to talk about because it'll just be updated behind the background.
Like my son Jet said, 'Okay, so another release a little bit better.' I mean, like, 'Dad, come on.' Like, what's really new here?
Well, actually, yeah. We'll see later in the pod some use case demos, but I think those will take over because it's much more exciting when you see a tick up in the intelligence. You're like, 'Yeah, so what?' Like, 'Look what it made.' That's what really gets people's attention.
Let's take a second and just look at that because I skipped over it. But I think one of the things that's interesting here and I'll just play these, you know, is what we're seeing is gaming to like recreate your favorite game. And on the right hand side of the equation here, we're seeing a web browser, a browser-based web app simulating an Apple desktop. I think this is, you know, we haven't talked about what the implication to the gaming industry which is huge, right?
You took over your computer there.
But I mean what was fun the last 24 hours was seeing everybody sort of show their use of Kimi K3 and it's impressive.
Everybody becomes a creator, everybody becomes a maker.
One warning. So one warning, oneshotting a game is very different from building the entire ecosystem and the customer service and the marketing that goes around with it etc etc. So you really have to be passionate about that domain but the friction of getting a game launched per your personal interest or your fascinations or a particular type of game that you want is near zero now and that becomes really interesting.
Yeah. Yeah. Exactly. It's mentally taxing because if you take it to the limit, which is very soon, I can oneshot prompt to create anything. And then you're sitting with your corporate exec staff saying, 'Well, what do we want?' Well, we've never had the ability before. We never really think this way. So then you have to kind of stretch your brain to like, well, what's the purpose of our organization in the first place? Yeah.
Here's a thought. Historically on this pod, we've done calls to action to submit outro music videos. What about a call to action to submit an outro video game that people have just casually created?
Yeah. So, going to your point, Dave, I think having taste, having imagination, understanding what the public wants. I think these become the scarce elements. And you know for entrepreneurs out there as you're seeing this capability I think the entrepreneurial mindset and the ability to imagine something even greater. I mean what happens when you're unleashed in what you can make, right?
Yeah, and visualizing happiness is, you know, we're not used to trying but you know a lot of people don't manage their own happiness particularly well because they, you know, they have to suffer through their daily job, have to suffer through whatever, you know, mosquito bites and geography, what, like it's just you have no choice. Given choice, what would you do? And that's so liberating for the mind but because we're not used to thinking that way we're not ready for like there must be an infinite number of things. Like the one that's easy for everybody is medicine and biotech like at least I want to be healthy, you know, that's an obvious one. But what about all the other things that make humanity happy? We really thought through what we could voice prompt tonight. And it's really godlike. We are godlike in our abilities.
And it's the name of your book.
Yeah. Well, I mean, it's the idea is being a creator or a maker, right, versus a consumer or a taker. See, I saw your eyebrows go up. What?
No, I'm just agreeing with all of this. I think this is such a magical time to be alive. Everybody listening to this podcast, please think up some business idea, project, impact, project, whatever, and use AI to go build it.
Yeah. I just want to maybe just briefly, Nick Bostrom speaks about this a bit in Deep Utopia. Peter, you and I speak about this quite a bit in Solve Everything. I'll just outright suggest folks if listening, I would, speaking just for myself, I'd love to see an outro video game that you casually create. Maybe something in the theme of the Moonshots pod since evidently we've completely solved and cooked music video creation.
Be a first shooter game where we get to take aim at AWG or
Oh no, please. No, no, no. Ideally, a nonviolent outro video. Nonviolent.
Civilization tech tree. That's what you need.
Civilization tech tree game would be great. I want to hit one point. You know, everybody watching and listening here, you have two options when you hear about this extraordinary ascent of Kimi K3. Fear might be one and the other might be, 'Oh my god, what an extraordinary time to be alive, right?' Hope and excitement and just an abundance mindset. And rather than fear, realize you are being unleashed. Your creativity, your ability to do whatever you want, the ability to create, your passion, your purpose, right? You know, find, you know, what Salem and I talk about so much is finding your massive transformative purpose, right? Just to distinguish between two, a passion is something you love doing. A purpose is something you love doing that actually benefits the world, right? And so if you can connect with that and realize that you can without any background, I mean, I think this is one of the most important things. You don't have to be a computer scientist. You don't have to be an expert. You have to be purpose-driven. And if you use these tools, you can make a dent in the universe. You can improve, you know, humanity at an awesome scale. And that's what entrepreneurship is.
Three steps. Read Alex and Peter's paper, Solve Everything. Pick the biggest problem you dare to pick. Go download the Organizational Singularity cloud skill which is free and start building.
Yeah. And note for our production team too here. It's so cheap and easy now to do things like Alex suggested, you know, make a video game. We should collect and post some examples for the audience so that they can say, 'Oh, that's what Alex was talking about.' But it's, you know, just a little road map is all people need. It can be this long. If the audience doesn't send in amazing Moonshots-oriented video games as outros, I promise I will create a cyberpunk FPS, but it'll be a nonviolent FPS, if you can imagine that, oriented around... What are you shooting? You'll be tickling bunny rabbits.
No, no, okay. It'll be a cyberpunk FPS where we're cooking every problem. How about that?
It's a first person solver, not a first person shooter.
First person solver. Love it. Oh god, that's great.
You got to do the tickling bunny rabbits, too, though.
Okay, I'll tickle bunny rabbits. Fine.
I'm going to move us to our next story.
Please, let's stick with their music videos. They're great.
And Emad, this is one you sent over the transom that I added here. So, if Kimi K3 is the frontier going big, trillions of parameters in a data center, this story is about frontiers going small, small enough to fit on your smartphone. So, Bonsai 27B for billion is the work of Prism ML. It's a US-based AI startup out of Caltech. It's run by Babak Sabi, backed by Kesla Ventures, Cerbrus, and Google. It's the first 27 billion parameter class model to run entirely on a smartphone, not a stripped down version. It is built on Qwen 3.627B. Emad, tell us about this. Why is it important? We just talked about small language models with Liquid AI on our last pod.
Yeah, this is one of Dave's favorite topics, quantization, right? Prism ML and actually Tencent, which I'll talk about in a second, have had massive advances in being able to take a model that's been trained in a 16-bit architecture or an 8-bit architecture like Kimi is basically 8-bit, 4-bit, and take it down to ternary, which is three bits of information or binary.
So, ternary is three values, one, zero, minus one, approximately 1.58 bits. 1.56. Yeah, like so three values, you geeks, you know, this is really, really important. Pay close attention geeks. Really important topic.
Well, this is again the accuracy thing. So what Prism managed to do is they managed to get the model down to ternary which basically means I think it was 6 GB for the model. This 27B model which is really performant. I think it's basically GPT-5 class from memory. With a 5% drop in accuracy, they managed to get it down to 6 GB and with a 15% drop in accuracy down to 4 GB.
And you can get the accuracy back too by expanding the size of the network a little bit. Sorry, there's various things you can do.
And so this is a big deal because it means you have a 110-120 IQ buddy that can work on your smartphone.
Again, if without an internet connection,
Without connection, it's smaller than a video game. You literally have this level of intelligence in your pocket all the time.
But it's live. You can download the weights right now. You can run it on your smartphone.
Exactly. Then away you go. But this is the super interesting thing. When you reduce the bits, it also increases the speed. So from 16 bits down to three bits, it's a five times improvement in the speed. And there was another article or another release which is 10 cents latest model. And this is actually the old Wizard LM team who had to leave Microsoft because Microsoft wouldn't give them compute. It's very ironic. They managed to get binary compression. So taking it all the way down for their high3 model. So it's now the best on a GGX Spark or a big MacBook to take a 300 billion parameter model, the size of the new model that's coming out of Inkling, to work on binary with a 5% drop in performance.
Wow. Yeah. And now I'm going to... Sorry, Emad. This is so important.
I'm only going to say this once on the pod because we're investing a lot of companies that are working on exactly this. I don't want to tip it too much, but the implications of what Emad just said have there's one more step there which is if I imagine like all of this intelligence under the covers, the computation going on has always been matrix multiplications. So I have a number, I multiply it by another number, and then I add two of those together. It's called a MAC, a multiply accumulate. If one of those numbers is just 1, 0, or minus one, I think we can multiply a number by 1, 0, or minus one pretty damn efficiently now. So that's the efficiency Emad's talking about, but it also opens the door for new ways to compute. And what Alex has been saying for a while, if anyone listens through, we're going to discover new physics, but also new substrates on which we can compute. And we're going to discover that computation is possible virtually anywhere in crystals, in liquids, but the computation we're looking for is simply 1, 0, -1.
