Is the open-weight, low-cost LLM business a good business? And if it's a good business, why can't some red-blooded American company step up and give OpenAI and Anthropic a run for the money? So, kicking us off, China ships two near-frontier open models in a week with Kimi absolutely crushing it. Next, Databricks raising 3 billion — I love this — a Series M at a $188 billion valuation. And then on top of that, we have RAMP releasing an OpenRouter competitor just as OpenRouter are supposedly about to get bought. This and so much more in the conversation this week.
The quest for equivalent models at a cheaper price just going to keep going up. At some point the people spending a trillion dollars a year are going to want some apps to pay for all this. If you're growing 10x year on year and you have any kind of positive and improving gross margin, it just covers all the nut. Growth for the last two or three years has been a very attractive place to make money.
Ready to go, guys. I'm looking forward to this. There has been a lot, as always, going down. I remember when news cycles were so much shorter. I don't know if you remember this, but like, you know, a $100 million round and it would be like the thing for a week. And now days go by and you're like, 'Wow, we're forgetting the Stripe and the PayPal,' which we'll get to, which is mega. But I'm going to start on the two near-frontier open-weight models that we saw in the last seven days from China. One of them being Kimi, which has got a lot of attention and a lot of press, and then the other being Qwen from Alibaba. How significant were the two model announcements that we saw today, and what should we be taking from their seemingly catching up or close to the frontier models we have in the West?
I mean, an eval is just an eval. So let's not take a bunch of folks on X who had someone in their engineering department look at some evals and write a tweet for them. Okay? Like, we're not saying something is similar in performance, maybe, but let's prove it in the field. Having said that, I mean, we can't even sign up new as consumers for Kimi because it's blocked. They have so much demand since this happened, right? Demand is literally — I don't know whether it's geometric or exponential — but it's so high we can't even... we can come back to this next week when it opens up and I can use it on the consumer side even better. But I mean, I think there's a lot going on, and there's a lot on politics, and it's an aha moment and a wake-up moment. On the other hand, it's not new. It's not new, right? I mean, if you look at OpenRouter data, half the traffic's through China-created models. Even 'China models' is a confusing term, right? They may well be hosted on — yeah, they may well be hosted in the US, right? And when they have open weights, they may be, for all intents and purposes, truly open-source models hosted in the US. But it's not new. It's just going to accelerate this. And that's why you see the stress. It's just, it's accelerating. And that 50% instead of being niche or for tech-forward folks or venture-backed folks, you know, in a year it could be everybody, and that is material — just being in the zone and even materially cheaper. It's just going to get more and more attention.
I totally agree with that, actually, Jason. I was curious to see what you'd say, is that there wasn't — Harry, you kind of led with the 'what do these new models mean?' I think Jason's take is exactly right. It's exactly what you'd expect. It turns out the five — and there's five main Chinese LLM companies and a bunch of followers — it turns out that aggressively funded companies with smart engineers are just going to keep cranking through and building new models. And yeah, they're not state-of-the-art compared to the frontier models, but they're, you know, six months behind depending on how you measure it. So yeah, actually no new news about that. But Jason's right, quite a lot of fun news about how parts of the US responded to that. We had the small-p political response. So that's one dimension. You know, the policy adviser for OpenAI, formerly from the Trump administration, making some comments on Twitter leading to a wonderful firestorm that we'll absolutely talk about. That's one thread. And then another thread is just talking about what these models start to reveal about the economics of a model company. I mean, Jason hinted at it. We lump all these models in together, right? But let's take two: some of the DeepSeek models you can run on your PC or your Mac or on a laptop, right? Conversely, Kimi K3 is, I think, a 2.8 trillion parameter model. It's a huge honking thing, and you know, you need myriads of GPUs just to run it. So they're not, quote-unquote, the same thing. That's much more comparable in size, and therefore in terms of compute capacity, to US frontier models. So there's a lot we can learn about. I think we'll kind of talk about the politics first and then maybe, oddly enough, talk about the inference implications and kind of as that goes into the opportunity for fireworks. So, lots of downstream implications, but zooming out, nothing amazingly surprising in the news that after three years of competent execution along a pretty defined trend, we now have three years and three months of competent execution around a pretty defined trend.
If we dig into the small-p in the political, how should we analyze that? We can talk about the tweet that you mentioned, which was — as I can't remember his exact title. Um...
Yeah, you go. You know, you keep going. No, no. And then Emil Michael obviously latched on to it, and I'm trying to remember, is it Dean Ball his name?
It's Dean Ball, and he is currently, I think it's out of policy or communications director for OpenAI. He just started there two weeks ago. Before that he was at the Trump part of the administration, kind of on AI policy, and before that a bunch of Hoover Institute type stuff. And he, you know, he set off a firestorm with the tweet, and then he did a little bit of 'Oh, I can't really post because I'm now at OpenAI, everyone was mean to me because I posted a bunch of stuff,' which was frankly a little naive. Because the two comments about the tweet: one is you're in a senior role at OpenAI; one, there was a hysterical tone to it, right? He used the word 'AI communism' and it was very kind of over-exaggerated. And then secondly, you know, when you start even hinting about — I mean, we saw this when Sarah Fryer did it — when you start hinting about significant regulatory changes that will massively benefit you, you got to expect that everyone's going to say, 'Dude, of course you're going to say that. You're, you know, that's your side.' And if you make the expensive closed-source product that sells for, you know, 10, 20 bucks and the Chinese are shipping something for two bucks and you say, 'Well, totally independently, just speaking as a common citizen, I think they should ban this shit,' you got to expect that a whole bunch of people are going to say, 'Dude, you're not talking as a common citizen. You're talking as the provider of the company who will jack up our rates the minute the stuff gets banned.' So, you know, it was a little naive not to expect that level of blowback.
Well, we'll see. At some level, first of all, I think that guy at OpenAI had been there like two weeks, right? Two weeks. Whether this was — whether he used that as a reason to go on this or...
As they say in the meme, Jason, two weeks so far.
Yeah, so far. Listen, I can't — I'm not a total expert. It's difficult for me to imagine the federal government's ever going to use a China-built model at this point in the US, right? It's difficult, and anything adjacent to that, it's difficult to imagine. Just simply, I mean, there's always in our whole history at tech, the ability of Chinese technology to penetrate many US buyers has been limited, right? It has certainly been limited in telecom and other spaces. So I think the quest — stepping back for a minute — the real question is how limited is it going to be, right? How limited are we going to be? Because it's going to be limited. The availability of China-built models to penetrate the US is going to be limited. The question is just how much. You know, Jesse Zhang had a Twitter article today or yesterday — I think today — and it was pretty good. I think people might have missed it because it's real data, which is what I like. But he said, 'Here's one of our most regulated companies. We have highly regulated folks. Just our token use here has gone up what looks to be about 2.5x since January.' And the reasons are really interesting. I mean, I've lived this myself. The reasons are having supervisor models track the agents so the agents don't make mistakes, running multiple agents in parallel so they don't make mistakes. The more regulated you are, the less forgiving you are of an error in an agent. And so it's like four times the agentic use just to have multiple agents regulating agents. If it's already grown that much in the first half of the year, you know, the quest for equivalent models at a cheaper price is just going to keep going up. It's just going to keep going up. And so, but we've always had cheaper, pretty good solutions from other vendors. It's not new.
Do you think Washington should move to restrict access then to these Chinese models? Or is Bill Gurley right in suggesting that we should let free markets do what free markets do best and we should not?
I didn't realize Bill had said that, though there's something very pleasing about that, which I'll mention in just a second, right? Because one of the fun things about this policy dispute, it brings out the hater in everybody, right? And you know, Dean Ball said what he said, and then two people who can be controversial came down strongly on the other side, and I support them both. The first was David Sacks, the former AI czar, who basically said, 'This is rubbish, stop.' And then the second one was Emil, whom you mentioned before — Emil Michael, I'm never sure of the pronunciation of his last name — who was a guy at the Defense Department who got totally sideways with Anthropic. I mean, what I like about that guy is that man knows how to hate, and one of his biggest hates for the last decade and a half has, of course, been Bill Gurley from his time at Uber. So I really find it... So if Bill and Emil are on the same side saying don't ban these models, then you got to know that there's got to be some truth in that. It's got to make you think, because that's an interesting lineup. But yeah, I think there — and I actually just saw literally as I came on, and look, this is the Trump administration, so things change every day. But Politico today basically saying some version of 'we're not going to ban these things on any significant basis,' which, as Jason points out, is very different than saying the White House decision support system will be run on Kimi, right? Even if it's Kimi hosted in California, I think we can take it for granted it won't be. Right? But conversely, if you're a decacorn and you're a startup doing inference on customer support queries for a very boring consumer product, there is no reason why you should pay marquee prices when something 10x cheaper is available, and it would be horribly bad policy to ban that. Right.
Well, look, the one thing I will say, just to add — and Bill Gurley, another rich, grouchy billionaire. Bill Gurley's probably got 30 IQ points on me, okay? And he's seen it all, right? And even at his grouchiest point, I learn something from it, right? I always learn from it. So, having said that, I don't think you're going to convince me there aren't some data export risks with China-based models. You're just not going to convince me, based on what I've done with all our agents in building. And if you're not going to convince me, I don't think you're going to convince 99% of the world that there isn't some security leakage issue. It's already scary how much of our data we put into these closed-source models in the US. It is scary. Here's Elon saying 'scam Altman' every day to create distrust, right? We cannot understand what these models do. They are connected to the internet. We cannot even — even if we have Fable read it and have it read it itself. I don't think you're going to convince most of us there isn't data export risk. And so I think that's going to lead to tighter constriction than this 'you know, leave everything open so we can compete' — in my portfolio company's benefit — argument. I think every CIO is being told right now, 'Oh, don't worry, if you host it on-prem, you remove any security risks and the back door that could potentially be there is then removed.' Why would you not be reassured by that on-prem solution? Why would you not be reassured? Rory's more of a historian here than me. You can mock our regulatory bodies, but they're here to answer those questions for us. Is it safe to drink that cup of coffee? The American Heart Association, I think, just said six cups are safe now, right? This week. Now I know. Now I'm cool, right? I was a little worried about my caffeine consumption when we did... No, no, seriously. I mean, I'm not sure they're right. Who has said my data is not being exported through the most complicated, borderline self-aware software of our lifetimes? Who can say that? Especially — and I admit this can be triggering — there is a history of data export risk with Chinese products. These are companies that are arguably run by the PLA. I'm not — I'm just saying my lifetime of experience says I'm not confident there isn't. And just the internet telling a CIO, I don't think is good enough. And if I were a CIO, it wouldn't be good enough to me, as long as I thought my job was on the line. I don't want to take this risk, CIO of some Fortune 500, Global 2000 company, unless everyone... I don't know, man.
I don't think it's triggering to say that there are IP risks in this. I mean, at the risk of being kind of level-headed here, the data is very clear that, you know, technically important US companies, Boeing for example, suffer continual cyber attacks, many of which are attributable to sovereign state actors, including China. It's a thing. So we're not being, you know, we're not being sensationalist or alarmist. You know, it would be naive not to put it on the table. You're right on both counts. Second point, I'm thinking about: the problem proving a negative is — you know, if you have an... remember these are open — I occasionally say open-source incorrectly — they are open-weight, which means you can see the weights and you can run it yourself, but you don't have technically the full definition of open-source in the context of an LLM, which means seeing the underlying training data, which you don't, right? But you have the open weights. The question is, if the model is being run in a trusted US inference company, Fireworks, some of those guys, right? And you could get into a long technical question: look, what can it really do? Could it initiate tool use on — I mean, it probably speaks — could it initiate tool use on the customer side whereby the model sends a command back to the customer to exfiltrate their data? You can imagine being able to use these models fairly comfortably and being fairly certain that you have blocked access and this can't happen. So I mean, I think in logic you could satisfy a technologist that the risk is not there. Whether you can satisfy a politician, whether you can satisfy someone who's just afraid of what they don't know, is, Jason, to your point, another question, right? I mean, and I think you're right, you have seen things like Huawei has effectively been prevented from selling to any cellular networks in Europe and the US because of this, as yet unprovable fear. So it's not crazy that there will be some level of, I think on the government side, some restrictions. I think a blanket ban would be massive overkill, to be very clear, right? But I think the interesting question it raises is this. Well, two questions. First of all, it's also worth pointing out that while we're talking about banning Chinese open-weight models, the Chinese administration are talking about preventing those companies from selling those models to the US. Like, just like we don't let them buy Nvidia, they're not going to let us buy their open-source models. Which is kind of totally zany. We think they're trying to sell it to us and we don't want to buy it, and they think they shouldn't be selling it to us because it's so powerful. So that's kind of just weird in of itself. But I think the really interesting question here, and it gets to Thinking Machines, is: is the open-weight, low-cost LLM business a good business? And if it's a good business, why can't some red-blooded American company step up and give OpenAI and Anthropic a run for the money? And Jason, it's the point you made. If this is — I mean, where's Grok? Where's Gemini? Thinking Machines had an announcement last week. They announced a model. They didn't position it as, you know, state-of-the-art frontier, but they kind of, I think they made a comment on something that you can build upon. Inkling, I think it was called. So, one of the — and you know, at one point, Meta looked like they were going to go down this route, right? Is there a business? How can you make money? It's an interesting question. Can you make money as a maybe not completely open-weight but a low-cost US provider of these models and be competitive with those guys? Because you know, the open-weight models in China are getting 50, 70 billion valuations. Like, it's not Anthropic, but I wouldn't turn down a $50 billion outcome if someone could make a convincing case to me that a US company could do this. So I think that's one of the interesting questions here.
Maybe it's because the dirty little secret is a lot of their advantage is distillation, which you can't legally do if you're US-based. So I do wonder, Jason, to your exact point, if there is a market for 80% cheaper intelligence, and that's roughly what we're looking at in terms of when you take into account the cost of inference, the difference between the bundled product that is a frontier model — you know, IP plus inference — and an open-source model where you dissociate the IP from the inference cost. If you're looking at an 80% cheaper opportunity and there's the mass demand for that, when is someone going to try and fill that demand in the US? I just, I don't know.
I've asked so many people why we don't have leading open models in the US. No one's actually given me an answer. We're still waiting for models from Reflection, which I think is kind of one of the hopes that we have. I completely... I was offered Kimi today, by the way, Rory, at 20 billion. It fell into my inbox: 'I have an SPV for you. We do Kimi at 20 billion. We're oversubscribed, but we'll make room for 5 million for Harry.'
Yeah, that's because we say such nice things about them. Thank you for your check. But no, look, we're going to gloss by it and I don't have an answer, but it's a huge freaking question, right? There's this new category, too, called LLM intelligence. Two companies existing as premium products, their combined market cap is $2 trillion. The combined revenue at this point is probably 100 billion, plus or minus. There are four or five other companies in the US that are capable and have proven their ability to build something roughly comparable. None of them are taking advantage of this. And there's five Chinese companies that have proven their ability to build something roughly comparable and they're, you know, cranking night and day to take advantage of it. Where are you, Google? Where are you, Reflection, as you say? Where are you, Thinking Machines? Where are you, LMArena? I mean, the fact that there's four or five potential competitors — it's just fascinating.
Or are you asking them to dance? I'm asking them to ship, you know, a story, and I'm kind of throwing it in here as a wedge. But when we talk about all the different models that we have on offer, one of the big kind of gossip stories or breakouts this week in terms of news stories was The Information suggesting that OpenRouter is in talks to be bought by several different acquirers. And then on top of that, we have RAMP introducing their routing, model-provider product. How...
Is it a great time for OpenRouter to sell? I think it's — I think them leaking the story was very savvy.
Why is it a great time for them to sell, Jason?
Because the market's in flux. Everyone's figured out they need this. OpenRouter, like a lot of folks, was early and benefited from it and deserves it. Right? This is a repeat founding team that saw that there would be value to having a fairly heterogeneous mix of models that, when we started this pod, probably made no sense at some level. It probably seemed too nerdy and too niche and too cool-cat developer. Who's going to need — like, yeah, sure, it's cool, but you know, guys like Rory and me, we're going to stick to the big guns, right? And everything broke well for them, but it's still a niche product that more and more people are going to build variants of themselves. And is this the plumbing they will pick? If you're on a lot of platforms, if you're on adjacent platforms, if you're using Databricks Gateway, they have their own harness, they'll figure this out for you. There will — I don't know whether RAMP's competitor even makes sense. Like, but my point is it's something that's going to become embedded in so many vendors that if I could sell for a lofty multiple of my last round, I might check out at five or six billion. Like, it's just because you've achieved a certain amount of victory in a market that's going through radical change and becoming part of everything. I might take the offer.
Jason is answering the question, why is it a good time to sell? And you know, frankly, as you yourself, Jason, have said, and I say it too, the private market liquidity window opens so rarely that it's always a good idea to pay attention when it does. So I think it's easy to understand that side of it. I actually think the interesting side of the discussion is the other side: why would someone want to buy, right? And I think when I saw that article, I was like, yeah, that makes sense. If you think about the last conversation we had, you know, what's in the zeitgeist right now, it's this whole idea of can I get escape model dependency, manage my costs, have a whole load of alternatives easily available to me, right? If I'm someone who makes my money as a hyperscaler hosting, especially someone that doesn't have their own in-house model they're pushing like Gemini. If I'm maybe Amazon in particular, who's made a business of saying I'm going to support all the models. If you had maybe even Microsoft now that the divorce is coming true from OpenAI, you know, maybe this would be a great product to own if I was a cloud hyperscaler. So I will admit I had that moment of — because you often see, and it's step out, you often see this in this kind of market. You have these businesses where intellectually over a 10-year period, you can say, 'Hey, margins are going to be tough in that business. It's going to be compressed, and maybe it won't at scale in the end be an amazingly valuable business because it won't be able to extract margin.' But on the other hand, when growth is so quick and people's urgency to adopt technology is so fast as it is right now, if you have a crucial piece of the plumbing at just the right time, when two or three people need that piece of plumbing, you can find yourself in a very interesting position in terms of M&A, right? Because it may well be that — and this is where the finance guys miss it — it may well be that the NPV of the company on a standalone basis is, you know, a couple of billion, not huge, but the value to it right now to a hyperscaler, if they could shift 10% market share in the enterprise to them over the next half a decade by saying, 'Dude, we are the cloud, we are the model-agnostic people, and we'll make it easy,' could be interesting. So I remember thinking, I wish I was in that — I mean, which is always how you know what a venture guy really thinks. It's like, damn, I bet you they could at a good offer right now. It's just interesting. So yeah, I'm with you. I think it's an interesting time to sell, interesting time to buy. I mean...
Rory, if you were on that board, would you sell at five to six billion?
You know, it's always hard to... I mean, my first comment is whenever you get an offer, I always do the same thing. I say to the founders: one, the window's open. It doesn't open often. We should take it seriously. Two, I'm going to support you whatever you want to do. Three, if there's concerns that you have that you've been sitting on and not telling me, now would be a good time to share so we can make an informed decision, right? And then go away and think about it, and I have a whole process of how you talk to them about it, right? I don't think you should pressure people into selling. I think your job is to give them whatever experience you have to bring to bear, and then they'll make the decision. Because you know, in the end — and it's so funny because founders agonize about this when they think about control and they think, 'Oh my god, these people are going to make a sale' — even if as the VCs we have board control, 70% ownership, and a drag-along for the founders, Jason knows this: if the founders who are core to the business don't want to sell, it's not going to happen. So the first thing I tell the founders is, it's your decision, which I think is very empowering because it takes it away from 'people are going to make you.' You know, that's the beauty of being private, unlike being public where you don't have that degree of freedom. We could come back to that later. So yeah, I would say to them, take this seriously. Do you think you can be worth 3x this in three, four years? Do you think that's worth it? But yeah, I would definitely say take a day out of your life and think about this long and hard.
The way Harry phrased his question was very VC-centric: 'Would you sell at six billion?' — what, I can't do the accent — 'Would you sell at six billion?' Okay, I hate the term 'triggering,' but that triggered me. That triggered me because this is a very VC-centric way to think of it. I've got an asset. What am I sitting at in my last round? 1.8 billion. Okay, there's going to be dilution. There's time value of money. There's IRR impact. Is the six billion worth it for me? I think for a founder, when you start to get into nosebleed offers in absolute terms, it has to be 10x to go for it. It's not worth it for 3x. It is not. Okay. Let's say I own 15% of OpenRouter. Okay, for the money. Okay. How much am I going to take home? Okay, at four billion, I'm going to take home 600 million. Okay. Now, one of the founders I think is super rich, right? But put that aside. I got four million in the bank. 400,000. 40,000. I'm going to walk away with 600, 700 million. 3x for you — it doesn't... like, maybe later in life. It's now 10x. And building something — and this is another trite VC — is building something truly generational. You kind of know as a founder when you're on that path. Okay. And so the VC, 'What, should I do it for 2.8x on my last round?' You know, it's just — it's the right way as a finance guy to think about it. Don't get me wrong, but it's a terrible way for a founder to think about it because there's way too much risk for not enough money. Like, going from again, going from 40,000 in the bank to 400 million versus 800 million — it's irrelevant if there's risk. And sometimes there is a handful of work. There is a handful of work in those next three years. It is a handful of sweat and a handful of market change and a handful of people that quit and move on and a handful of competitors that are looking pretty good and they may pass you. 3x not good enough, man. Got to be 10x.
I'm confused. What are you saying? Are you saying to sell at six or are you saying no? Which is odd because...
I'm saying if it's financial, sell at six. If the 18 is not enough, it's got to be 60 to be worth the risk for most founders. It's not enough gain. This isn't clearing the prep stack. This is clearing my life stack.
I understand what he's saying. It was weird. I didn't think you were going that direction, but as often happens with you, and I listen to the whole thing, I'm like, I get it. I think what he's saying is this, Harry. Right. You know, when you face that sell decision, you don't not sell because you think you can make twice as much in a year, right? Because you just never know. And so in the end, even though I didn't think I'd agree with him, in the end, I did. It's like what I think he's saying is, let's leave aside the 'what do you want to do with your life' questions. From a return perspective, you know, don't think incrementally. If you have a chance to sell a company for six billion and make 600 million, and you think you can run it another three years and get, you know, 1.2 billion, that might — risk-adjusted — that mightn't be worth it if your current net worth is $40,000. And that's actually good financial advice. I mean, it doesn't — in other words, if you turn down a big-ass offer, you better be sure it can be way bigger. You better have high certainty and high bigness. I think that's a fair comment.
Yeah. If it's 10x, this is my net advice. If you know in your heart and soul you are building a company 10x bigger than this, right or wrong, like I don't know, then go — then effing say no and go for it. Here's a few more shares. In fact, friends, let me reload you. But they only best at 10x.
It wasn't what I thought where I thought you were going, because I actually thought you were going to say, yeah, something that you also — don't sell also just because financially you feel you should. I mean, because the other thing you're not taking into account is that depends on the person. Some people just love running that company and frankly don't want to sell. It's their life's work. And I also think you have to respect — I mean, my point, you also have to respect that. I mean, again, this gets back to the 'there's no one answer, the founder decides.' I've known people who are like, 'This is my first hit, I'm going to take it,' who read exactly that logic. I am not...
I'm going to make 60 million bucks. I don't have one million buck and maybe I can make 120 in four years but I'm taking the 60. And I've also known other people who to a rounding error have said this is what I want to do for the rest of my life. Why would I take that money? I've got I've made three or four million in the secondary. I got my house. I'm done.
Can I ask more more more thinking is we said it doesn't matter RAMP but ramp doing their own data bricks doing their own. I just interviewed the founder of fireworks who announced their 17.5 billion valuation. Like is there any value in this layer if it's as commoditized as everyone? That's why I would sell with my limited knowledge but as a user right as a customer I would sell just because I think there's some commodification at a minimum right
sometimes you're l I mean as to your job by this team but sometimes if you're early you can gain a lot of traction in something that becomes somewhat commoditized it's just it's just the way it goes and if you and the perfect outcome is to sell the moment it becomes commoditized but before everyone fully realizes it. That's when they'll give you the money, but that's before the value decrease rather than increase. And it my gut is it might be. It might be now. That crossover moment. And I think Rory kind of made a version of that point. It might be now. It spaces commodify. It doesn't kill everybody, but it might maim you, but it also makes it attractive. It also makes you attractive to acquirers for a window and then that window closes. The commod the commodification window closes. And um probably why Curser wasn't dumb to sell at 60 billion. I think we can agree that's true. I don't love the commoditization description. I think it it it it's an overloaded term, but yes, I
well an included feature and more and more folks will include your function some version of your functionality in their product, right?
That's that that's exactly it. Where Yeah. which segus to the next topic,
Yeah. And inference in general. Yeah. I mean,
take it away, Rory. I'm going to butcher whatever context that you want to take it on.
No, no, you do. No, you do first cuz I'm I'm just
Are you sure? I'll lay the framework and then you can just destroy it. Steamroll away.
Fireworks uh a leading inference provider uh announced their latest round which was a $1.5 billion round done by um uh index Gavin Baker uh Nvidia Lightseed 20VC. Amazing firms. Um they're incredible. Uh thank you very that they're really [__] good. Lynn is amazing. Uh doing over a billion in AR got there in three and a half years and they announced around 40 trillion tokens a day up from 15. I mean I think the story is inference. Yes.
First of all I agree. Yeah inference is a hu it goes back ironically to the prior comment on openweight models. it this kind of standalone inference is a big business right and you know obviously you know inference is both something that's done within the the frontier model companies where they do their own inference and people like Microsoft and Google provide the capex and provide the compute for that but people like fireworks and base 10 they and file they all make their money offering a variety of these open openweight models to third party developers and enterprises that want to use open source models to do AI, right? And it, as I said, the two trends go together. They're exploding because the open source trend is exploding. So if you're base 10, if you're foul more media, if you're fireworks, if you're together, this is your marker than your moment, right? So yeah, I this is cuz this is how you access those because I can tell you one thing. We can going back to the discussion about open weight models from China. It's one thing to decide to use an openweight model on fireworks in the US. What you're not going to do is be using the API back to China even if they'd let you. Right? So this is a onetoone linkage between the open source the open way trend. Right? These are the companies that are benefiting massively from that trend and it's not the only kind of route for inference. There are you know inference for US-based models etc etc. But the vast bulk of it is oh my god I'm sourcing Quen. I'm sourcing Kim. I'm hosting Quen Kimmy. I want to use it as Cursor. I want to get someone to provide me some inference. These guys exist and you know they have lots of customer skew at the high end. I believe you know companies like cursor are probably big customers of all these guys. At least they were until they were acquired by um acquired and probably still are right. So yeah, I mean it's a great candidly. I think you know it actually goes back to the point I made earlier. some businesses where and I think you know you can look at and say oh my gosh the cost con you know you have margin compression in your future because you're buying your compute from the neoclouds and you know you're offering this product and are you going to be scrunched and margins you know were probably slow for a while but now the beauty of it is demand is massive so whatever compute you have today whatever compute you've already signed up for and these guys sign up for compute from the neo clouds in general are bu are starting to build their own. Whatever compute you own now, you can charge way more, which means what looked like a grow a lowish gross margin business has now probably become a very attractive business. So, not only are they probably going grow growing 5x to a billion, but they're probably growing 5x to a billion with expanding gross margins.
And just just to add some details there, Lynn said specifically that they were at mid30s in margins and that would move up as they eat more of the stack and they do plan to move into the data center uh layer themselves.
Yeah. And just to be clear for people, what that means is Yeah. And that's exactly where I thought they'd be and good on them, right? 30%. In other words, what they're saying and this is going to be an and this and I agree with that sentence. It also means that the challenges I hinted at are there in the future, right? Because what they're saying is if I'm buying data center compute and then effectively selling data center compute with hosted LLM, at some point I'm going to want to own my own data center assets to have more control of my destiny, which means vertically integrating downwards, which also means a ton more capex. So these are going to become way more capex intensive businesses. There is a risk of commodification here. Even though even with massive complexity and massive capex,
there is one one thread of the Twitterati who has said for a while like um all this stuff's interesting but ultimately it's the you know the application layer is going to be the most interesting. It's going to benefit from all this. Everything's commodified, right? Um all the good investments sure seem to be in the infrastructure.
Even the ones that look good in software, the numbers pale in comparison anyway, right? The the absolute numbers pale. So, I'm waiting for the for for the for the era of the application layer in uh AI and um making bets and seeing some good stuff, but I don't believe it's here yet. I actually don't believe the application layer is here yet.
To to put again, Lynn said in the show, she expects to double by the end of the year.
And that's and that's just a slice of the market. Listen, people have gone all back. You know, when we started the show, uh it felt like vibe coding applications run a muck. Everyone thought you'd replace your sales source. You even had a guest the other week who was I forget saying how great it was. He replaced Salesforce. Who cares, right? Um didn't kill software, but where where where is the software renaissance? I I mean the revenue is there. We've talked about leaders, right? But it it's so trivial compared to the infrastructure. It's so trivial. It's almost a rounding error uh the application layer.
And just to dimension that because I agree, Jason, I mean look, I'm an app investor. It hasn't, you know, you look back and you what's going on here? You've got companies like, you know, um, fireworks doing a billion dollars. There's very, there's very few apps companies doing that. And, you know, zooming out a million miles. You my mental model is I divide the AI world up into three buckets. It's the making AI, the infrastructure layer, right? And you're right, the spend there is 8 $900 billion a year. Then there's the two foundation model companies themselves and they're doing plus or minus hundred billion dollars a year, right? And then taking those guys out, rounding up every other apps company, right? You struggle to make 40 or 50 bill. You struggle. You start with cursor at four because I think coding is an app. You know, you by the time you're chucking in Harvey, you're adding two 300 million, right? It's amazing. I mean, just the difference in spend. And you know, at some point, the people spending a trillion dollars a year are going to want some apps to pay for all this, right? But right now the volume has it's been front end loaded on the infrastructure side and at some point the revenue has to match it. But right now info has been the place to be. Like there's probably more money being spent on training data for the foundation models you the merker surges and that than pretty much any app company outside of cursor. In fact, probably the sum of all the apps companies outside of cursor, right? You know, are probably less than the amount that Entropic and Open AAI are spending on training data, which is just amazing.
That I can guarantee when you look at McCoring 2 billion in AR,
two billion for Merco surge, another billion, you get to four or five billion and you know,
I mean yeah maybe if you start throwing in on the other side the the consumer products like Higsfield you get to roughly the same place but it's astonishing the scale of the investment versus the scale of the apps at this point means that all the actions on the info side for now if we bring this all together we you know we mentioned fireworks at the start Lynn said in the show the future would be every company having specialized models with their own data um we mentioned Harvey there who've been building their own models Jason I'm just intrigued In the last week, you spent time labeling data, building your own model through that data. Any lessons, reflections from the last few days, labeling data, and going through that process that you've been through?
I've been building this Aentic recruiting app just to recruit from the Saster community. It's been fun. I've learned a lot building it, right? Hopefully, it can ship in the next week or two. But, but to really get it great, it needed it needed labeling to make its Now, I'm going to put model in quotes, right? it it uses sonnet and opus but so there's different definitions of model and it was good and but man once I started labeling all of this it it got exponentially better right um built my built my own it built built our own little labeling tool and so you need your own micro model whether it is some sort of reasoning layer that you build on top of uh claude or or chatg or kimmy or shmimmy like it's still your own model even if it's not technically a model right because you you have your own reasoning layer with with your with your own rules, your own weights, your own and um but you want more you if you have the resources you want to go further than that, right? You want to your big M model as soon as you're at a certain amount of scale and it's not cheap all in, right? You are going to want to have your own model, right? Like like like a Harvey cursor. So some version of this I think and the folks that are going to want to you at any application level folks that are going to want to use just the generic models is just going to decline to to to to prototypes, right? Prototypes and proofing.
Yeah. Or absolute state-of-the-art kind of small parts of the overall task. But agreed
parts. Yeah. Little parts, right?
Like I mean again, you're going to want to use the expensive tool for the expensive parts, right? And you're going to want to use the cheap tool for most of the parts. from the customized tool to your usage.
But man, the outputs are just or order literally an order of magnitude better once you do it. So every everyone wants your own model. Um and um so I I do think whether whether that always benefits fireworks or not, it doesn't matter. as long as they pick up the some of the bigger the bigger end right that scales it's uh uh it is yeah the don't the generic models are great but it is amazing how much
how much better you can do than them for any specific workflow you can do epically better
would that change your confidence on the data labeling market a lot of shade is thrown at at it as an investor in Mor I definitely see it is does that change how you think about
Personally, I'm totally I totally get it like having a subject matter go in and answer 20 questions about a disease about a history. I mean, it's a lot of professors and teachers that they have there, right? That that model, right? The amount of power you can get in a domain by having a subject manager answer just 20 or 30 questions, right? 5 minutes, 10 minutes. the amount of power you can add versus the the generic LLMs which are a sea of mediocrity combined into one giant LLM. Okay, every mediocre history professor, every every mediocre doctor that doesn't even know what what caused your runny nose is in the LLM. But if you get the the best people training on the best answers, it's a step function. I'm less smart on the the this seeming low end of the model, right? This commodity thing that people made fun of Mccur, but I ain't making fun of it anymore. I I tell you that much. Um, and these models are are a sea of mediocre all combined in a giant soup that gets better. These domain experts are so powerful in tuning your model to get the the better output. So powerful.
I I I think the answer though is really a derivative of the big question which is you know because your statement your companies are going to want their own model is probably true right and the real question is not that the real question is will that be additive to the rough trajectory of the foundation models as it's established today in other words coming you know at or close to 100 billion combined revenue growing nicely or does it start to take away significantly Because you know to answer your specific question if the foundation models continue to grow and we just saw an article information that you know for for all the training data companies the vast majority of their revenue comes from the foundation models to which your correct response is no [__] Of course it does, right? If that continues to grow and you have an additive market in enterprise of all these companies you JP Morgan building the JP Morgan Morgan model on top then you know it's net expansive and net expansive is by definition good and reduces customer concentration and I think that's what people like are forecasting right if on the other hand you know which is hard to contemplate today if these enterprise models if these openweight models really impacted the growth rate of anthropic and open AI Then obviously when your 80% customer slows down it would have a significant impact on your growth rate right but you know if it's any consolation Harry if that happens worrying about your merur valuation will be the least thing people are worried about because you'll see an implosion of much bigger market cap entities right and you that's that frankly is the billion dollar question you know can these two foundation models maintain their growth trajectory which is starting to become profitable at least in the case of open of entropic in the face of all this openw weight competition in the face of this push back um on costs and basically kind of push for ROI if they can maintain this trajectory for even another one or two years then everything's fine and everyone's fine and right now the data says they are if you start to see slowdown on those two ARR growth rates then you know all bets are off because the pressure because the amount of commitments they've made assuming that 10x growth rate continues will mean that even if it slips to a 2 or 3x growth rate there's going to be a mad scramble
bets on yes or no answer will open impact that trajectory for anthropic and open AI in the next 1 to two years
yes Harry it will impact it might impact at 1% or 50% what you're really saying what the question you're really trying to ask is does it produce a sustain you know does it reduce that growth growth rate to sub 100% within one or two years. Right? And the answer to that question is I genuinely don't know. And if I did, I'd be trading that stock. Because if let me be clear, if you know the answer to that question, that one question, you know the answer to the entire direction of the US stock market for the next two years because all the hyperscala RPO, all of it is a function of the commitments they've gotten from the the hype from the foundation model companies. And yeah, you can say if the open models, open weight models explode, there will be demand for inference. And yeah, you will have this kind of transition from oh, I sold it to open AAI but I should have sold it to um I don't know cursor or B base 10 or someone else and the capex will get repurposed, but it will be a big ass dislocation and I just genuinely don't know. I mean it's the million-dollar question.
I think the tough the really tough part I mean it's Captain Obvious, right? is uh can they afford for it not to? And what I mean is look at what's happened with Fable this week. Okay, Fable went from you can't use it, it's not secure. Then the government let you use it. Then hey, we're going to turn it off except for variable usage on June July 15th. Now it can be 50% of your whole usage for the month. Why did they change when they don't even have enough capacity to serve it? Competition, right? Competition, right? So it if listen if they price Fable at sonnet rates I think they'll own the market.
I'm oversimplifying because you don't need fable for anything but literally so the question is can can they afford to compete and this is the stressor right? Of course they they could have 17 variants of the model at 17 price points like like uh that's not the issue. The the issue is the because they have to they have to pay to train these damn models um and other reasons. They have just this this high cost base and they're subsidizing it with venture capital, right? We whether we call this venture capital or not, private private capital. And so, but listen, you just cut the price of Fable 5 by half tomorrow, you don't need these Kimmy shmies, but can they afford to? And and and if they and and and over what schedule and the fact that you can use Fable for half your credits is pretty telling, right? They're they're pushing it as far as they can, but that's the limit today. They can afford to compete 50%.
And we don't have time for it because I do think we should spend at least half the show on stuff other than, you know, a AI model companies, but I think Jason's insight is correct about price. And if this was a software product with no gross cost of goods sold, that's what they do. I mean, Microsoft, I mean, and this is one of the big I've seen a bunch of articles on this. Again, it's as you'd say Jason Captain Obvious, but it's worth emphasizing in the great software wars of the last couple of decades, someone like Microsoft was able to take an you just use price ruthlessly because there was zero cost of goods sold and they just bundled the browser in with the operating system, bundled all Office in together, didn't cost them anything, and it just wiped everyone else out. But as you're pointing out here, there are real even at the margin, even after you fully paid for your training costs, there are real physical costs to serve these models. And you got to cover your nut. You got to cover the marginal cost of the model of the ser of the inference which gets you to I don't know two, three bucks kind of blended average token. Then you got to recover the cost of the training and you got to recover it pretty damn quick because it only lasts, you know, 12 24 months before it's obsolete. And then on top of that, you want to make extraordinary profits because you're being valued at 20 times revenues. And if you're valued at 20 times revenues, you better be like Microsoft with 40% operating margins. So when you look at all that, you're right. You know, it's kind of back to the thing I said, you can squint at that and say, "Oo, there's lots of things that could go wrong here." When you look at that those fundamentals as yet the thing that's saving you right now if you're growing 10x year on year and you have any kind of positive and improving gross margin it just covers all the nut right the minute that growth rate stops Jason to you if the only way you can keep that growth rate up is by lowering your kind of price per then your gross margins start to deteriorate instead of continue to improve that in itself would be a different ballgame. So you are right price could solve it but it would be a painful way to solve it.
Yeah, you have to start building your own chips and building your own everything, all the stuff you're trying to do to solve this problem. Um, but I think it's just a I mean, not to be, I think it's just a pricing problem, right? I mean, there's a bunch of issues underneath, but um, and I would argue they've already bundled it. Like the consumer apps of Claude especially, but also CHB, they bundled everything. I can get $10,000 worth of tokens for 200 bucks and I can just do just about anything in it, right? It's just outside of the consumer, it's not it's not massively bundled and subsidized, right? Just cuz you're still
Yeah. You remember the old days in software, Jason, when you'd have to say, "I promise I'm only using this for personal use." You remember that? And licensing, right? Well, if you're saying, "CL, if you're telling those nice clawed people that you only using your personal subscription for personal use," they're going to find you, dude.
They're going to find you.
Well, yeah. It's just It's just Fable is very good.
Yes. It that's why that's why they're going to find a way to charge for it.
Rory, you were like, "Are we going to get away from this like AI stuff at some point?"
Yeah. Yeah. Yeah. So much we What were you hoping to talk about? Like
I mean a vertical dentist company like