Hello and welcome to another episode of the OddLots podcast. I'm Jill Weisenthal.
And I'm Tracy Alloway. Tracy, I have to say unfortunately I don't have AI psychosis. I'm certain of that.
I'm pretty sure I don't have AI psychosis. I do have to say unfortunately like the amount of time now where it's like it feels like AI related questions and there many of them are sort of like swallowing up the other thoughts that I have in my head. Whether it's questions about which model is best and why and what are the economics of inference and how much training is pre-training versus post-training for each model. All like it's just sort of like this blob that's growing that's taking up more and more of my thoughts.
What is your definition of AI psychosis? Cuz one would argue that maybe thinking about AI literally all the time would be a form of psychosis.
Well, let's just say like I'm not the type who thinks that like I don't like think that the AI is a friend for one thing. I'm not in love with the AI models. I don't think that in collaboration with chat GPT that I'm stumbling on a unified theory of physics and things like that. So like
But you do spend a lot of time inputting instructions, pressing the button and seeing what comes out.
And seeing what comes out. I'm just saying I think I'm aware that I'm talking to a machine and that we're not establishing any great breakthroughs of which we are collaborators and partners and friends.
Recognizing you have a problem is the first step towards healing. Joe, seriously though, there's a good reason to think about AI more and more, which is that a huge chunk of not just the market but the real economy is now revolving around AI. Right.
Totally. So anyway, again, within the AI conversation, there are a lot of subcategories. One of the subcategories happens to be another OddLot's favorite topic which is chips. Of course, chips are used in multiple different ways. There chips are used in different parts of the AI supply chain. Different types of chips have different roles. And so, we have to learn more.
We have to learn more. And I have to say I'm particularly interested in the company we're about to speak to. Partly because the two things I know about them are number one, they just had a huge IPO, right? raising something like $5.5 billion at kind of insane multiple. I can't even do a price to earnings multiple because they're not profitable yet. But I think just on a sales basis, it was like 67 times forward earnings, which is
Pretty juicy, pretty hot. And the second thing I know about the company is they make giant wafers. Yes.
Which is just a fun image to have in your head.
That's right. So if you were thinking it's like okay there is a hot entrant in this space what is their differentiator well one fact about them is their chips are just enormous about the size of the dinner plate one might think you're reading an onion article but in fact it's real and apparently it actually has some real technical advantages so
And it's different to what everyone else is doing so everyone else is I guess doing this sort of like modular networking thing where you get together a bunch of chips and you connect them together and That's how you get more compute, more memory, more power basically. But this company has done something different in the form of the giant wafer.
The giant wafer. And if you figure that to get maximum performance, you sort of want to lessen the distance between things, then put it all on one wafer. Anyway, we're going to learn a lot more. I'm very excited to say about giant wafers and more. I'm very excited to say we do have the founder and CEO of Cerebras on the podcast, Andrew Feldman. Truly the perfect guest. So, Andrew, thank you so much for coming on the podcast on the week of your IPO.
Well, thank you so much for having me. What a pleasure.
Absolutely. Why don't you just start us off? The big giant chip, they're apparently real. They're as big as a dinner plate. What is the technical reason why this actually makes sense as a superior form of architecture for at least some aspect of AI?
Well, I think larger chips process more information in less time.
Okay. And that that produces faster results. And everybody had gone to bigger chips. Nvidia had moved from 400 square millimeters to 800 square millimeters over the course of five or six years for this exact reason. And in the compute industry, wafer scale, which is building a chip, this big
For those who are, by the way, for those who are just listening, Andrew is now holding up the chip. And yes, it looks it actually looks bigger than a dinner plate to be honest. But is that is a big
Beautiful. 50 is think it's 58 times larger than any other chip that had ever been.
And what it did was it allowed us to use a different type of memory.
A type of memory that there at the beginning there are two types of memory. There's memory that can store a lot but it's really slow.
Okay. And there's memory that can't store very much per square millimeter but is blisteringly fast.
Okay. And historically all graphics processing units used this memory that could store a lot but was really slow. And that's the reason they do inference so slowly. So if you're using Claude right now or you're using anything but chat GPT, what you'll frequently feel is you'll enter your prompt and you'll wait for an answer.
Right? And that's because the memory is slow and they have to move a ton of information from memory to compute. Now by going to wafer scale we could use this fast memory. Now we couldn't make that memory store more information per square millimeter but we could add square millimeters. And so by building this big chip we were able to stuff it to the gills with this fast memory. And that's why we're 15 times faster than the fastest GPU. That that's why on some problems we're 50, 100, even a thousand times faster than graphics processing units.
Wait, can you explain how you actually manage to do this? Because I know there have been previous attempts to do wafer scale and I I seem to remember there was even like an early attempt in the 1980s or something to do it. How were you able to pull this off?
Yeah, it was an ambitious undertaking, that's for sure. Every previous effort in the 75 year history of our industry had failed, including Gene Amdall, who who's sort of on the Mount Rushmore of compute in our industry. He failed sort of spectacularly in the mid 80s at a company called Trilogy. Not only that, but after we succeeded, people who had visited us, who'd been in our labs, tried to copy us, and they also failed. And so what we were able to do is solve a set of really fundamental problems. And those problems cut across a wide swath of technology. They cut across lithography. So we had to collaborate closely with TSMC. And they turned out to be a great partner. We had to make inventions in material and packaging. That's how you put a processor, how you put a piece of silicon on a motherboard, deliver power and IO to it. We had to make inventions in power delivery, right? When you build a giant chip, you're going to deliver way more power to it than if you do a chip the size of a postage stamp. We had to invent ways to cool it. We had to write new types of software that ran on it. All of these had never been done before. And it was a decade long process.
It took us five years and about $500 million to deliver the first one. And it's been an extraordinary run since. In December, we signed a deal with OpenAI north of $20 billion, one of the largest contracts ever signed in Silicon Valley. And then in March, we signed a deal with with AWS where they would deploy our systems in their data centers, in their AWS data centers. And so, it's just been an extraordinary run, but it took a long time. Yeah,
It took extraordinary engineering.
And there were certainly long periods of time when it wasn't clear we were going to make this work.
Obviously, you've hit this remarkable milestone. You have in fact IPOed and so forth and right now markets valuing your company at 64 billion dollar early days of the IPO. Just for the listener to understand the chips are are they solely an inference as opposed to you know in training. When we think about AI, I think about okay, there's training, training the model, and then answer giving. That's the inference. Are the chips for just for inference?
So, a couple things I I think you framed it exactly right. Training is how we make AI, and inference is how we use AI. And so, what happened was that in sort of 2025, in the first part of 2025, the models we made were smart enough to be useful.
And there was an explosion of use. and we use AI by doing inference. So there was this sort of tidal wave of demand on inference and that has continued in 2026 and we think it will continue for for years and years to come. And so that that's the what had happened in 2015 when we began thinking about the company. We knew that AI was on the horizon and it would eat a huge amount of compute, right? And we we made sort of two fundamental bets. We bet that it would need dedicated silicon and right graphics had needed dedicated silicon. That's how you got the graphics processing unit. Mobile compute had needed dedicated compute. That's where you got ARM processors. We made that bet. And we made a bet that modifying the GPU architecture wouldn't be right. You needed to start with a clean sheet of paper. And so what we started with was a new vision. And that vision could do training and it could do inference. And it was orders of magnitude faster at both. But right now what we're seeing is such an explosion in demand for inference that a lot of the business has met as inference even though we're just as fast at, you know, the same amount faster than than GPUs on training.
That's interesting. Maybe we'll get more to the theoretical training market a little later. Just real quick on inference. Ben Thompson who writes a newsletter about tech he wrote a piece in which he distinguishes between answer inference and agentic inference. So answer inference is like you know format my resume or whatever or write me an essay on X or Y or answer some questions and then aentic inference is like okay here's this thing that's going to go around. Do you distinguish and do services for you? Not producing visual answers. Do you distinguish between those two? Is that a real divide in your view? And can your chips do both?
Our chips can do both. I think it is a divide.
I think speed matters equally in both.
I think if you are engaged with the AI, if you're writing code, which is agentic, if you're writing code or you're doing work, nobody wants to wait. I mean, we could just turn the question around and say, well, how big is the market for slow search? Zero. How big is the market for dialup internet? Zero. Why is that? Because nobody wants to wait, right? So, if you're engaged with the AI, speed is of the essence. But if the AI is doing agentic work and your competitor gets three times, five times, 10 times as much work done in 20 minutes than you do, you're going to get smoked. And so this notion somehow that Ben proposed that speed isn't very important in agentic flows is dead wrong. That speed is important in all aspects of productive work. And that your ability to get more done in less time is a fundamental advantage that that accrues over time. Right? If while your competitor is doing one unit of work, you can do three and in the next time they do one unit of work, you do six.
Right. This adds up over time and you beat them in any line of work. And so speed, which is sort of our specialty, is is important across the board.
What do giant wafers and speed in general actually mean for I guess the economics of tokens? Because one way I think about it, I have this sort of vision in my head like, okay, if I'm out shopping for toothpaste, I know I need toothpaste every once in a while and I go into like a CVS, a store, I get one thing of toothpaste and then maybe a week later I get some more toothpaste. Or I could go to Costco and buy a giant thing of toothpaste and take it home probably at a cheaper cost. And that's sort of how I think of the giant wafers. Maybe it's a bad analogy, but what does speed actually mean for the cost of tokens?
Well, I I think there are a couple observations. I I think people have chosen so far to price speed a little higher. For example, Anthropic offered a premium service in which they offered tokens twice as fast and charged six times as much and they sold it out and they couldn't meet the demand. Now, just to give you an idea, we're 15 times faster than they're twice as fast. And so, people value speed because it allows them to do more work and they value their time. And when you can do more work in less time, you are making people more productive. That's why people have chosen to price them at a premium. They don't cost more to make. In fact, in the GPU architecture is an extremely good architecture and extremely efficient at building very slow tokens. And if you don't mind slow, the cost per token on a GPU is extremely low. But the GPU has a characteristic that as you try and go faster, the cost and the power used per token increase. Sort of like as you go faster in your car, your miles per gallon decrease, right? So what happens is as you try and get fast enough to be useful, fast enough to be interesting, fast enough to keep users intelligence focused on this product, they become extremely expensive and extremely power-hungry. And so the question is is not just what people are paying for a token, what people are choosing to price them at, but what they actually cost to make. And GPUs make very slow tokens very cheaply. And they're unbelievably expensive at fast tokens. We make fast tokens vastly less expensive than than GPUs. And we use a tiny fraction of the power.
Let's say we stipulate that this is all true and everyone wants the fastest and everyone's like you know what this is the solution that the Cerebras technology one big chip this is really where it's at. How much of like your market share for the inference market when you look out next year, the year after etc. How much is your market share going to be dictated by your ability to get capacity at TSMC fabs? How much is that a gating mechanism for growth?
You know, TSMC is is a huge part of the supply chain.
But we have some real advantages. There are three areas right now that are limiting vendors in building AI compute. Number one is HBM memory is this memory we described earlier that can store a lot but it's really slow that's made by three companies approximately Samsung, SK Hynix, and Micron and it's under unbelievable supply pressure it's extremely difficult to get there are very long lead times it's unbelievably expensive right now we don't use it. The second part that's limiting is a process inside of TSMC called CoWoS. And this is the process that that Nvidia and other GPUs use. We don't use it. The third thing is that at TSMC the factory that is under most pressure is their 3 nanometer factory. We don't use it. We use 5 nanometer. So we have managed to avoid some of the the most binding supply constraints. Now TSMC still has to give us a meaningful allocation and they've been an extraordinary partner from the get-go and they are the greatest manufacturing company on earth by far. A fab is sort of a modern pyramid. It's an unbelievable thing and and I highly recommend you or or any of your your listeners if you get a chance to go to Taipei get go and see them. They they are just extraordinary.
You can actually. Yeah, you can do fab tours. you can go and they have a museum of innovation and it is an extraordinary thing.
They are the sort of the national champion of of Taiwan.
Yeah. But I I think today TSMC has given us as many wafers as we've needed. Business today is constrained by data centers.
And that's the grand irony, right? You invent technology that has been unbuildable, never been invented for 75 years in the history of compute. You write software that is extraordinary. You built a product that is vastly faster than the incumbent. And what are we all constrained by? Buildings, right? Data centers right now are are everybody's constraint in the entire industry. Powered buildings. So, real estate, it is an amazing thing right now. And that is true sort of across the board.
And that that will not change for the next 15 or 18 months for sure. I mean, since we're talking physical constraints, I guess I should ask you, we did an episode about helium recently, the helium shortage given the situation in the Strait of Hormuz. And one of the things that helium is used for is lithography on semiconductor chips. Has that affected you at all? Is that something that you're monitoring?
We monitor, but there's not a lot we can do. And there's plenty of stuff to worry about that that we can affect. We obviously are in communication every day with TSMC. We're in communication with our entire supply chain every single day and we we stay abreast of of the various issues. But it it has had no impact on us and we we put that in the bucket of things that our manufacturing partners worry about also and that we can't help, you know.
So in addition to manufacturing these chips, you actually I I didn't realize this, you have your own cloud. We do and or you have your own cloud services which I have a bunch of questions about that. You have your own cloud services through which a user can actually get access to various open source models and so forth. It looks a little bit sort of visually it looks a lot like the open router interface roughly the same environment except it's all like the open source. What I'm something I'm curious about and maybe you can speak to this you know in traditional software opensource one nice thing about open source is you don't have to pay for it. So it's free. It's a little bit different when we're talking about there's no really such thing as like free AI software because even if it's like free, you still have to like pay for the depreciation of the chips and you have to pay for the electricity to run them. So there's no real such thing as like free open-source AI software. But what I am curious about in your experience as a cloud vendor are the open-source models cheaper on a per unit of intelligence basis if we had some way of saying levelized cost of intelligence which I don't know if the industry has yet. Are open source models cheaper per IQ point whatever we want however we want to measure intelligence.
Yeah. I I think in the closed source world, you're paying a lot for that extra little bit of intelligence, right? The open source models, there are no open source models that are as good as the closed source models. Okay? Think of it as 3, 4%, 5% different. Okay? Something in that range. And it could be a little more, could be a little less, but the cost to you using them. Yeah. Right. You can jump up right now and run Kimi K2. It's a one trillion parameter model. It's an open source model on Cerebras. We're 10 or 15 times faster than than others. And what you're paying for is the cost of our power and some cost of the compute that took to calculate it. What you're not paying for was the cost to train it, right? And that's a battle that is underway in the market. You have OpenAI with their coding software. You have Anthropic with their coding software. And you've got companies like Cursor and Cognition that are using open source. We power OpenAI and we power Cognition. You have a battle underway between closed source and open source. And I think that the winners of that battle is yet to be determined. What is clear is that the closed source is is strictly better by a little bit. By how much varies and it's more expensive.
Yeah. I think we've talked about this before, but like I've heard of a lot of big companies in the US who have been like very quietly shifting from some of the closed source models to the open source models like the Chinese ones like DeepSeek. Is that what it's called? DeepSeek and Qwen.
