The Model That Beat Claude & ChatGPT, The Nvidia GPU Crunch Continues, Beehiiv’s Social Tools

Thanks for having me.

New day, new model.

It's incredible. Six models in the last seven days. It's been Grok 4.5, Kimi K3. You know, previously we had GLM 5.2. It's the new series of models, the GPT 5.6s, the Soul, you know, the Luna, the Terra. It's been an extraordinary pace of model development and it's left us with a Chinese open-source model on the top of the leaderboard.

Well, you know, there have been murmurs of the great performance of Kimi K3 for quite a while. But I'm not sure that people expected it necessarily from Moonshot. I will tell you what has been the trend, which is that over the past year you've seen open-source and closed-source models had a big gap and then it started closing and closing and closing and then they were kind of tracking with open-source models just a few months or weeks behind closed-source models. And people were saying they're never going to crack it and it's because of distillation. They're just distilling, distilling, distilling. And so, but fundamentally, when you distill a model, it degrades. And so, you're never going to get better performance. But this is the first time we're really seeing the narrative that breaks that mental model that says that the Chinese labs might actually just be really good at developing models and not just distilling American intelligence.

Well, the way our platform works is that we have a user base of tens of millions of people that are coming on Arena to use AI for their real workflows. They're agentic workflows. They could do single-threaded conversations that are, you know, tens or hundreds of turns long. They're getting to do, you know, workflow automation. They're doing coding. They're doing math. They're doing all these sort of economically valuable tasks. And we take all those tasks and then turn them into a benchmark. And one of the benchmarks that we released for Kimi K3 has been Code Arena. And specifically, Kimi K3 is at the top. It's number one in Frontend Code Arena. So, it's the best at the sort of Lovable vibe coding use case. Now, the best model in the world for that, of course, for which hundreds of millions of dollars in inference spend will accrue, is...

Ahead of Fable, which is absolutely remarkable.

That's correct. And I will tell you the gap between Kimi K3 and Fable is not small. It's a win rate of in the order of tens of percent. You know, let's say 10% is the difference in win rate between Kimi K3 and Fable 5, which is, you know, it's not the largest gap that we've seen on Arena, but it is substantial and indicates a real delta in the change in performance. And now that comes at the cost of a model that is a Sonnet-level cost. That's a double-edged sword.

Oh, like Claude Sonnet. It costs the same amount as Claude Sonnet. It's $3 input, $3 output token. Same price as Claude Sonnet. So the significance of this is actually quite interesting. First of all, of course, the raw performance is amazing, but also the fact that it's no longer the case that an open-source model is, you know, 50% of the quality at 10% of the cost. It's actually that it's frontier-level performance at a reasonable cost that you would see for a mid-tier frontier model. And so I think the ecosystem is evolving perhaps in an unexpected way in the sense that open-source is actually also quite expensive, which speaks to the fact that the margins are actually not that high on inference.

Well, you know, it does sort of like — it's definitely a frontier model on a lot of the static benchmarks. So if you're looking at things like SWE-bench and Tau-bench and so on and so forth, it's sort of in the pack along with Soul and with Fable, although not necessarily leading on some, not leading on others, but still right there in the competition. In Text Arena, which is our benchmark for sort of ChatGPT-like capabilities with user interaction and so on, Kimi K3 is top 10, but not top one. And in fact, it's not even the top model outside of OpenAI and Anthropic. It's about tied with Soul, but it is beaten by Muskpark 1.1 and the basically entire bench of Claude models — 4.6, 4.7, 4.6 Thinking, 4.7 Thinking, and Fable — all better chat models for chat interfaces. So right now, for text, who's at the top right now?

So, so, so Kimi K3, from what I'm hearing from you, I mean, really — I mean, it's coding that has really impressed me. Of all the use cases you could use this for, that is the one that's standing out.

Too early to say, but we're going to have more to say on that over the next days and weeks about how steerable the model is and how companies are finding it easier and easier to use. Of course, the one thing that I will say is Kimi K3 is quite a large model and so it's not like you're going to be able to run this locally on your laptop. You are going to need a cluster. It's going to need quite a bit of infrastructure. And I would expect that many businesses will be relying on third-party inference providers to give them the compute they need even to do the inference on this model.

Well, I think it will cause a reckoning in the capital markets. And the reason for that is that it brings into question what the dominance will be of the closed-source models in a world where there's a narrative violation against the distillation story. Then what will happen is that people will say, well, open-source models coming from China, they're, you know, they have so many benefits. Businesses can incorporate them into their own infrastructure without relying on a third-party service, without worrying about privacy and data leakage, without worrying about, you know, evil third-party companies training on their data and stealing their businesses. There's so many reasons why people would want an open-weight model. And in a world where that can be done for free, then the question comes, why would we be paying for closed-source models that are worse for our businesses, that are less private, and that allow these companies that want to be in every single business in, you know, FDE for every single vertical? Why would we be allowing them to witness our business and see our data, learn from it so that they can improve and eventually one day steal our business? Why would we do that? And so then what's likely to happen is an accrual of value to these open-weight models, from which of course the revenue model is quite different and worse than the closed-source models, which could then cause a collapse, you know, in the way that people are seeing the compute markets because of course the compute markets are driven by the optimism and the revenue of the closed-source models, so on and so forth. So it could cause a cascading effect.

Absolutely. It absolutely matters. And in fact, we should all, you know, if we're patriots and Americans, be rooting for some American open-source models to come out as well. Of course, it's fantastic that we see the Chinese developing these models and it's socially positive net-net, but at the end of the day, because of the geopolitical risk, we are not going to see businesses comfortable building their entire stack on Chinese models.

No. Absolutely not.

Why do you think it's less of an issue?

I think the governments of both sides would disagree. What I see happening is actually more hawkishness on both sides. That the Chinese government is actually talking about export-controlling Chinese models and the US government is also talking about banning Chinese models because neither side wants to incur the geopolitical risk. And of course there's so much competition happening between the US and China, it is absolutely becoming higher and higher risk. We even have the US government banning US models from release, let alone Chinese models. I think it is actually a huge topic.

Listen, Inkling is the first open-source model to be released by Thinky. Actually, the first model period to be released by Thinky. And of course, it's incredible to see that we have a company that is leading the American open-source charge. The other contenders would be Neotron, RC, Reflection, Mistral, of course, in Europe, and there's a couple other players. But Inkling came out at the top of the US open-source race today. That said, it is not yet a frontier model by any stretch. It is, if you look at the wider field of top 10, including Chinese models, it is number 10. Number 10. So there's nine Chinese models above this. And so they have quite a bit of room to climb. Now, if you know anything about Thinking Machines, and it's sort of, you know, common knowledge, it's not something that I know specifically because of our business, but they've had quite a bit of restructuring, so on and so forth. People leaving, people coming in, restructuring their team. And so maybe to some extent it's impressive that they were able to do this in a relatively short amount of time since the current structure that they've had, six months or so. That said, there is a long, long way to go. And if you look at the field of closed-source models as well as open-source models, it is number 37 model. So we still have, you know, there's a ladder to climb, but let's all root for them because that is — we need a model within our shores to support the enterprises that want to avoid the geopolitical risk that we just discussed. If we don't have an American open-source model, businesses will simply be incurring greater risk and be less comfortable adopting AI.

Yeah. Well, I think that the strategy is actually quite a good one, which is to diffuse the model everywhere and then become the best place for running that model and therefore gain resources. And, you know, they need to have some kind of an asymmetric advantage, the Chinese model providers over the US model providers there. But because they have a lot of drawbacks too. So they are absolutely limited by the amount of GPUs that they have. They are absolutely limited by the fact that there's illegal regulations inside the US that keep you from shipping data to China. And so because of these facts, they need to have some way of attacking the market and it may as well be open-source. I think that's also compounded by the fact that there's a lot of central planning in China. And IP is treated differently in China than it is in the US and perhaps there's less of a focus on IP from what I know of the Chinese market than the US market. So those are all reasons why the open-source strategy would be better for them. And then how do you monetize? Well, there's two ways of monetizing an open-source model. The first is become the best place to run that open-source model. And then you can earn money through inference. That's what a lot of these companies are doing. And the second is through licensing. So you say, okay, I'm not going to run my open-source model, but I'm going to partner with inference providers in the US that are going to run this open-source model. And if they reach a revenue of above X dollars in running my model, then I'm going to do a rev share with them. And that's going to be in the licensing of the weights. And of course, because we respect licensing laws here, that is exactly what those companies are doing. And that's going to accrue back to revenue for those companies.

Of course, I think you're getting at the concept of recursive self-improvement, the idea that the models might be able to train themselves and so on and so forth. I don't think we're quite there yet. Although we are even seeing in our business that the AI — that the existence of AI is actually making our employees more and more productive by the month. We're seeing productivity per employee actually rise even at Arena at our small scale. We're 70 people and we are still seeing that effect happening. So I'm sure it's happening at the big labs as well. That said, I will say there's a funny dynamic, you know, first of all, you know, even competition aside with these labs, which is that the release cycles tend to be kind of bulky in the sense that you'll get a quiet week with a couple models here and there and then you'll get a week with 10 models. And why is that? I think these companies are all hearing about each other. Oh, Kimi K3 is going to release the following day. Oh, we know that we're not quite as good as Kimi K3, so we should probably front-run them so that we get a little bit of good press. Imagine if Inkling and Kimi K3 switch, right? If they switch on the release date and Kimi K3 comes first, Inkling doesn't — they don't release. They're not coming out with it.

Yeah, exactly. They don't look quite as good and so it creates this dynamic of a mad rush. Everyone hears that everyone else is releasing and so everyone is competing to be first, especially the players that know that they're on a weak foot. And not to say that Thinking Machines is — I think that the model's great. It's the number one US open-source model, and again, we should all be rooting for that. But it's also true of the closed-source model providers as well.

Well, I said this online and I'll say it again. I think we're going to see a lot of companies that are distilling — American companies distilling Chinese models now instead of the other way around. I'm, you know, I'm sure OpenAI and Anthropic have their own way of training models, but even beyond those two, you know, the Kimi K3 is going to serve as a base model for many, many US models and we're going to see that happening now. And if the trend continues that the Chinese models get better and better over time, it may pose like a serious threat to the closed-source business model because of all the drawbacks that the closed-source providers have.

Absolutely. I actually have huge belief in the Google team. You should see those people. They're really smart. Their team is like excellent. Excellent. I mean, it's also true of Anthropic and OpenAI.

I think they're good. I expect the comeback. Wait for it. Just wait for Gemini 4. I'll bet you it'll be, you know...

Yeah, that's right. That's right.