EP033: Cerebras (CBRS): Thematic Winner in AI Compute Shortage with SemiAnalysis’ President

One call and response in a small local language model isn't going to be valuable in 10 years from now. Pretty much every person in our firm has the ability and capability to be a software engineer, be able to do things at a speed and rate that are just impossible. The analogy that we best like to use is that Cerebras is like an F1 car and Nvidia is a bus.

Claude Code is the inflection point. We spend north of $10,000 a week in API and we think the ROI is very high for it.

Cerebras is pretty interesting. I guess I'm not shocked. Well, I am actually a little shocked to see the price action, but if we're going to talk on the technology side, I think the reality of what's happening is that we're in a compute shortage. And we were talking about this within the team. You know, if you hear the classic venture capital pitch of like, oh, we just need 2% of a very large market, when the Cerebras venture capital deal was happening, I don't think the market was as large as it is today. And let's just say I'm going to throw a number out. Let's just say it's a trillion dollar market. Okay, 1% of that actually happens to be extremely large. And so I think that that's kind of what we're watching here is that Cerebras has a niche technology that I don't think, you know, we had this long article that finally came out about the company. It doesn't fulfill most of the needs of inference today, but you know, there's enough demand that you can kind of find a place for it in the big scheme of things in terms of compute demand. So yeah, that's kind of the high-level thesis of what's going on. I can kind of dive into the technology, kind of where things are going in terms of inference. There's a lot of places we can take it, Doug. But yeah, I want to let you prompt me.

So, Cerebras made an intrinsic bet on something called SRAM. I can't tell you what SRAM is off the top of my head, but it's kind of a logic-based, it's at the highest tier and the fastest possible memory. The vision that Cerebras had when they originally started this is that what if we made the biggest possible, fastest possible memory, kind of like how HBM, as many have heard, is very popular, but an even faster level of memory, which is SRAM. But they made a bet on what ended up being a dead end. SRAM scaling has stopped. I don't think we have the time to explain why that's happened. But specifically for how the density of the bit lines in the logic process for SRAM, there's no more density to be gained. So essentially going to a smaller node isn't going to make your performance any better. And the way they had the wafer scale has like 50% of the die area is SRAM. So if it's not going to get any denser, they're pretty much stuck at this level of memory. And obviously when they first made this product, they're like, 'Dude, a whole wafer of memory, of course it's going to be big enough for an LLM.' And what we found is that LLM ended up being much bigger than anyone ever expected. When Cerebras, the company, first started out, this wafer-sized chip was almost inconceivable amounts of compute, but now that we've gone to today, we've come to see it's not enough memory. And so that's kind of the core technology problem, but that SRAM is extremely quick. So if you can fit the model weights on the wafer, what's going to happen is your inference is going to be crazy fast. But if you can't, then you can't inference it. And so that's kind of what we're seeing. There's no truly large models, meaning larger than a trillion parameters, that are able to be run on these wafers at all. And so that's kind of a problem. For example, the partnership they have with the Groq Spark that we think is a GPT-OSS 120B, which is like a 120 billion parameter model that is being run extremely quickly, but that's never going to have the performance kind of at the leading edge. So if the market is big enough that there's, let's say, a tail of smaller model providers, meaning let's say 500 billion parameters that can be fit onto the Cerebras wafer and be inferenced extremely fast, that is kind of the niche market that they can fulfill. But we're really worried that the really large market of extremely large models, which is where the frontier of capability is, won't be able to be serviced by them. And what we see in market is that people prefer the highest capability.

I'm going to put some big asterisks on it because things have been changing quite often and rapidly. I think that there is a world where Cerebras has a right to win, but we don't really know where the model architectures are going. One of the hottest trends right now within production inference is essentially breaking all of the big pieces within inferencing one of these models into their subsequent parts and doing a little bit of specialization. So the first one was called disaggregated prefill decode, which effectively there's two stages of serving, prefill and decode, and they have different requirements. And so what if you made one part of it served in this chip and one part of it served in this chip, and they better suited each requirement, which means the total throughput went up. There's a potential that the Amazon deal specifically could be a disaggregated prefill because there's a lot of flops, meaning a lot of logic, but not a lot of memory bandwidth off the chip on the Cerebras chip. So that's an example where maybe they can do some kind of hybrid architecture. Maybe there's a world where state-space models or Mamba or something like that happens and there's a hybrid architecture and they can have some part of it. I think the reality is that Cerebras is a viable alternative. OpenAI clearly has a lot of demand and we just don't think it's in the main technology tree of where things are going, but it could definitely be a really compelling branch. And then also the other context of this is after Groq happened specifically, and Groq, I'm going to be throwing a lot of things at you. I've been kind of workshopping some of this in terms of so AFD, which is aggregated feed-forward network disaggregation or something like that, where you push the feed-forward network off of the transformer. I know this is crazy. It sounds like I'm an insane person. That's an example of a disaggregation that's been happening as well. We originally thought that that's what they were trying to do, but I think it's unclear where Cerebras' exact niche is going to end up being in the very long run, but it's clearly a viable technology and we're in the biggest compute shortage of our lives. It's a hot IPO. The stock popped. I think there's a lot of things working for it in a good way, but I definitely, you know, I'm not on the snake oil. Like, obviously, it is a good or interesting technology bet. That was a very bold bet. I think that the bet didn't come completely true. And so they have, let's just say, I'm trying to maybe use some analogies, like let's just say they made a big bet on the size of a railroad line and now we're all using this other type of railroad line and there's this legacy thing that kind of works for this other alternative version of the world that we are in. And clearly it's worth something when everything in chips is so expensive and valuable, but it's not quite, it didn't beat Nvidia. And so that's kind of how I'm viewing it right now.

So, it does beat Nvidia in this niche speed inference, but there is a cap. We wrote about this in our article. There's a lot of hard, like essentially, let's just put it this way, it cannot serve you 5.5, I believe, but it could serve you 5.1. And for example, it could not inference, if you're using Claude, it could not inference Opus 4.6 or 4.7 based on our estimates, but it could inference Sonnet 4.7. So there's this world where maybe what happens for really fast inference is these lower-end models get the product, but we find overwhelmingly in the preference of people who want to use these products, people go to the best model possible. Sometimes they use the lower models, but most of the value is in the highest end. And so there's going to be value for Cerebras in really fast lower end, but it's not going to beat the capability. And the question, it's unfortunate, like we don't know what is going to win, but our impression is that, like for example, Mythos, right? The reason why Mythos is so good is because it's a big, it's a very large model. Likely will never be able to inference it. And if Mythos is going to be what is all these like cybersecurity, bleeding edge, extreme research cases that really do matter, and the capability frontier does seem to matter, right? Like all this vibe coding, agentic, the agentic explosion, I don't know what to call it, like the Claude Code thing wasn't possible in Opus 4. It was Opus 4.5 that made everything, like essentially that new capability frontier made this whole new market possible. And I think Mythos will probably have a new capability frontier that will make a whole new market possible. But the problem is Cerebras is not even able to be at the current frontier of capabilities, and Nvidia is going to continue to be able to service this at a larger, higher volume way. And yes, so there's a really good, and this is all on the SemiAnalysis website. Cerebras is really, really, really fast, but it can only serve a very small amount of users, right? Versus, and this is like probably outside of the scope of this podcast, it's like it's about batching versus speed. And so total throughput actually for an Nvidia system will be higher for multiple users. Meaning that you can have 30 users going at a pretty good speed versus one user going at a crazy speed because you could have this big model serving many, many, many people because of the size of the memory domain that Nvidia gives you versus a smaller model could only serve one person. The analogy that we best like to use is that, you know, Cerebras is like an F1 car. It's extremely fast, but it literally can't, you can't take it off the track effectively. So you're stuck on this one type of track, okay? And it could only serve, let's say, one or two users, one user, okay? And Nvidia is a bus, and in the big scheme of things, if you want to move people around, you're going to need a lot of buses and trains, right? One bus is going to be able to move 60 people much further. And maybe it isn't at 100 miles per hour, but it's at 60 miles per hour. You're 15 minutes later, but you managed to move the total volume orders of magnitude more. And so the reality is today, when it's about getting raw tokens out of the gate and people paying for those tokens, Nvidia is still the king. Now the real threat and competition we are starting to see from Nvidia is from custom silicon programs from the hyperscalers, which is the TPU, the Trainium. But is there a world where this niche F1 car deserves to exist? Yes, but it's just not going to service like the vast majority of the market. And so yeah, it is faster on a one-to-one comparison, but you know, if you throw in a GB200, you move the batch size to one, you're doing tens of thousands of tokens or thousands of tokens a second. We have all these numbers, I think, on inference. This kind of relationship between one or the other is kind of a little complicated and a little bit more nuanced than people think versus seeing, hey, fast token, whatever that fast token experience you have, it can only service what, one very few amount of people, and that's the big difference.

You know, it's funny, the CTO vision of the world, maybe that's true, but I think what I found so far in following the AI space is that best of breed wins. It's become very, like, capabilities matter. You could have said that last year and then now you're using, you know, Opus. Like you could have been using, let's say, Llama 70B or, you know, 140B or GPT-OSS model, and the reality is you can't actually make a web, like you really can't actually code a website in one shot and be able to do consistent, persistent work. It would be a pain in the ass. Yes, you lose, you know, you lose your data sovereignty, but the reality is you're just, you're capped at a level where it's a call-response chatbot, right? And what we're seeing in this explosion, like where the demand actually is coming from, is from Claude Code and this agentic IDE stuff. And the reason why is because it's not able to do, like the analogy I've been using is like that simple call and response chatbot is much closer to a TCP/IP. It is like literally bouncing a packet off of each other. One packet and one call and response in a small local language model isn't going to be valuable in 10 years from now. The thing that's going to be valuable is how do you, you know, how do you orchestrate hundreds of thousands of packets into a website? And so what we're seeing is that your packet quality has to be a lot better. And that capability tier with the Opus 4.5 and, you know, GPT-5.5 level products is what is able for websites to appear, and that's where most of the value and capability is. So maybe there's a world with, you know, local LLM from, let's say, a Chinese open source provider that you have your own data is valuable to you, but you're never going to, or it's unlikely you're going to have the utter cutting edge of frontier performance. And what that means in a lot of ways is like huge risk. I think the best example is cybersecurity. We're starting to see cybersecurity, it feels like a blow up every single day. My personal belief is maybe that's bad for incumbents, but maybe it's going to be a little bit more nuanced than that. But with Mythos, we're finding these bugs in these OS that's been unpatched for 10, 20 years. We're finding like Project Glass Wing is telling you that if you are not at that cutting frontier, you're going to be a sucker effectively. You know, some guy with a, and you know, Mythos is trained so it's not supposed to be able to be, let's say, you know, an adversarial attacker, meaning it can like hack into your system. But if you have, you know, your competitor has a better model than you, and let's just say you're two lawyers. We're going to do a lawyer, okay? These lawyers don't think about anything, and you're using your local language model, and you have access to all your data and all this stuff. But at the end of the day, these guys have, you know, this Mythos-class model, and you guys are making briefs, and these briefs, and you know, these lawyers are doing some cognition, but let's just pretend it's all the models, okay? This guy is going to crush the hell out of this guy. And so what's going to happen is you're going to lose. And when competition matters, it's going to push to capabilities, and then everything will be kept flat at that capability tier. And so I think that, and there's going to be cases where this doesn't matter, but I think we're still in a best of breed world. And I think that the rationalization for on-prem local language smaller when you own your own data is something that will happen when the capability curve flattens out. And until then, which I think we're pretty far from then, best of breed's going to win. So that's my personal belief. I think it's going to be nuanced, but, you know, I think my history and track record of being on the right side of things in the longer term has been pretty good.

It's actually interesting, you know, you can just opt out of that. I mean, and to be clear, there's going to be something in five years where OpenAI and Anthropic were training on it anyways, like kind of like, you know, the OG Facebook, Cambridge, what is it, Cambridge Analytica, is that the, that's the original, right? But I think that in... Yeah, that's clear. What happens is all information wants to be free, and we're just bringing up this higher and higher baseline. But yeah, I definitely think you can see an example where more net output is happening, and that example where you have the lawyer stuff where it's like, okay, before maybe I'm still up because the lawyer on the other end is like, yeah, it's kind of LLM, and this is how it makes these mistakes. But let's just say in terms of how informed you were versus, let's say, five years ago, it's night and day. And I think that that's the big difference is that you're just seeing all this stuff. This wouldn't have been possible. You can't just talk to a lawyer. I mean, you could just talk to a lawyer for tens of hours and be like, okay, well, what about this case? What about this case? What about this case?

Yeah, it's more, you know, more efficient market now. More access to information, a lot cheaper. It's better for everyone there.

We could talk on the stock setup specifically. This is not a recommendation buy or sell. I'm just trying to be as objective and level as I can be in terms of what I see as the setup. I think the broader context really matters. I think it IPO'd at a pretty good time. You know, SMH, which is like the broad-based semiconductor ETF, is up 50%. We had like literally of like whatever a four-sigma move. I think since the April bottom, it's been like, I don't know if you saw this, it was like a Sharpe of 10 or something like that. It's like, it's crazy. It's been up only, no downside vol. It's a straight line all the way to all-time highs. And so what I think that that is, what you're seeing is Cerebras, for better or for worse, is levered to one thing and one thing alone as a pure, pure, pure, pure, pure, pure bet, much more so than Nvidia. Hey, it's a smaller base. It's going to have higher beta. And so I think what the setup for this stock is, it's a function and proxy of the unrelenting demand into the AI infrastructure space, and it is a really solid outlet that kind of explodes. And so that's kind of what I think of the stock. This actually, you know, I wasn't around, I was alive during the, you know, I think I was barely alive during the 2000 era, but I think that this to me feels like the first of the, probably the first of many or definitely a few AI IPOs and how pure play clearly de-risks the demand in the market. If I was someone who is sitting on the sidelines with a private company that might not be as mature, obviously they have the backlog. They have a relationship with OpenAI that really helps anchor it. But it tells you that the answer is the demand for equity in these companies is higher, and CRWV is an example where it exploded. And so I think on CoreWeave side, I think people have always been core skeptical on a lot of the, how can you just rent something and have a good business model? There's a lot more pie in the sky here that I think can get people much more excited. And that's kind of how I think of a semiconductor versus an asset owner that sells something on a lease. One has some meaningful upside in a big market, but it's fixed at a contract rate. They don't have, you know, torque tokens or the value that's up on top of it. At the end of the day, they are going to have some kind of fixed gross margin. Cerebras has in theory a potential to win in almost an unbounded market. And I think that that's what makes the equity story better. I think that this is an interesting checkpoint because if I had to guess, this means that, and also we want to even take, like let's just even broaden our scope outside the United States. The IPOs in China specifically for GIU and GLM, which are two Chinese open labs, those have done amazing. Like they were, they're like 500, 600% year-to-date or something like that. Those are the first public companies that were labs that are available, and they're not even at the frontier. And clearly, you know, equity demand of risk-taking is huge. And so I think that what we're going to see is we're going to see more IPOs. As you know, SpaceX, xAI, Cursor is what we're calling it or something like that is going to come out any week now. That's going to be, that's going to be a next leg or probably the next big IPO because they're going to do a meaningful offering. But after that, we're going to be talking about obviously Anthropic and OpenAI sometime this year, maybe at the, I think it's going to be at the tail end of this year, but that's going to be, you know, that's like the main event, and we'll see how much unrelenting demand for AI will exist. And I think that that's going to be a huge, huge outlet, right? We're talking tens if not hundreds of billion dollars that will kind of be siphoned off into this market. But right now, what I can tell is that the $5.5 billion of demand for Cerebras was easily eaten up, and it's telling me that there's demand for more. And I think Cerebras is not the top dog in the big scheme of the game, and I think that that is going to bode really well for the top dogs in the game. And so I think this is a setup for what will be a pretty frenzied IPO window coming soon.