Cerebras CEO, Andrew Feldman on Why Raise $1BN and Delay the IPO & Why NVIDIA’s Worried About Growth

Things are moving at a rate that 6, 8, 12 months out everybody's unsure. It's so fast. It's so big. There is unbelievable demand and nobody knows where it will go in the future. The question of depreciation is how much faster are future generations than the current generation. That's the actual question on depreciation. People often say we don't have enough power in the US. And this is strictly wrong. We have plenty of power. It's in the wrong places. Risk comes in financial markets where people fundamentally underestimate risk. No company ever went bankrupt by paying extraordinary people too much.

I must be charming.

Harry, I'm okay with any questions, naive or otherwise. So, I'm happy to do it anytime. I read your LinkedIn posts, your Twitter posts. I'm rooting for your mom. I mean, all good. All good.

Well, look, it was the largest raise ever done in our category. It was done at the highest valuation and with the premier investors. So, at late-stage investing, you're looking for the likes of Fidelity. They are the what would the English call it? The sort of Oxford or Cambridge of investing, right? I mean they are the premier public market investors and when they choose to lead a round it brings Wall Street a great deal of confidence and so we were really happy to partner with them and with the treaties to lead the round and then we were able to get enormous participation from Tiger Global, from Valor, from 1789. So that's point one. I think point two is that we now have sort of the dry powder to really push and to take the opportunities in front of us to build out our manufacturing to the scale and scope we want to add new data centers. We added five this year in the US to add more data centers and we have more big ideas. I think incremental improvements, make-believe gains achieved by dropping from 8-bit to 4-bit, those aren't going to get us to the promised land in AI. We need, we've got real work to do as a community and I think this funding puts us in the catbird seat for that.

We still have every intention of going public. I think it's very common in late stage to do a pre-IPO round if you can get it done very quickly, if it doesn't distract you and keep moving. I think there were so many opportunities in front of us that gathering the capital so that we could continue to prosecute these opportunities was sort of a no-brainer.

The first thing, Harry, is we are in a stage of the market where the claims are enormous, right? Where tens of billions of dollars are being done here and there and nobody's reading the fine print that it's over five years and it's up to this is the great sort of CYA word in marketing history is it will be up to 100 billion over five years right well up to means it could be 30 could be 12 could be 40 right it won't be bigger right he could pick a lot of big numbers and it won't be bigger then so I think as you read these deals, you have to really think about the time frame over which they're being done, you have to think about whether anybody is actually counting, right? Lots of people are saying they're going to bring hundreds of billions of dollars of jobs to the US and this and that. I mean, in eight months, has anybody got a little spreadsheet like nine jobs plus one factory? I mean, who holds anybody to account? And the answer is nobody. And so I think that's number one. I think number two, what this signals more than anything is that there is unbelievable demand and nobody knows where it will go in the future that it's so big and happening so quickly that they don't know. I mean we have customers coming to us and saying we would like between five and 40 million queries per second. Well, how do you not know by a factor of 35 million queries per second where your demand's going to be? How are you unsure by an order of magnitude of what your queries are going to be? And the answer is things are moving at a rate that 6, 8, 12 months out everybody's unsure. It's so fast. It's so big. And so I think you should think about sort of these announcements as sort of options on the future, right? That's really the way to think about it is in an unknown environment, how can I take an option on the future? I don't know if I'll use it all, but I'll pay something for the future rights to have some capacity. So, that's a way to think about it.

It's brutal. I mean I think there's a very interesting question about in extraordinarily rapidly moving environments what the right planning cadences are and what you really need is good planning changing rules rather than good planning right if you can only we have to make big bets we're making five and seven-year investments in data center capacity we are making hundreds of millions now on term of billions of dollars of bets in supply chain. And those are not three-month bets, right? And so I think what you need to do is use different rules that have historically been used. You plan more frequently. You have a shorter view. You take options on the future and if the future moves against you, you lose the premium on the option, right? You pay a little price to secure some capacity and if you don't use it, you just go, 'All right, that was sort of a way to manage uncertainty about the future.'

100%. I've been wrong. I mean the look if you would have said a year ago, two years ago, pick a time at which it would have been conceivable that OpenAI would get the valuations they're getting pick a time right it wouldn't have been conceivable right 3 months ago, six months ago, nine months ago when was it a reasonable like the day before right and I think That's true with the demand we're seeing. It's true with valuations on companies we're seeing. It's true with the rate of ideas entering the community.

You know, I would say this. I would say they're always grumpy people who like it'll never work and you know you'll never beat Goliath and you know truth is most things don't work and most of the time Goliath wins right but there's no alpha in that there's no money made for you or me betting on the biggest of the big dogs to continue not to lose how uninteresting is that. And so I think of course right if Nvidia keeps growing at the rate they're currently growing 11 years from now everybody on earth works for them right do the math right I mean however is it possible that our economy looks very different in five years is it possible that the things we value are very different that we have reorganized around AI. We've seen a major bump in labor productivity. We have benefited dramatically and the economic pie is much larger. I think that's not only likely, it's almost certain.

I hope he's long on them then. I don't pick public market stocks. I think I don't like picking public market stocks. I think in the public market you can lose money on good companies, you can make money on shitty companies and that for me doesn't sit well, right? As an entrepreneur, as a David in the battle with Goliath, I want to make money when we build a great company, period. But can they continue to grow? I think we are seeing some things that big companies do as they begin to worry about growth. I think use your balance sheet more and your technology less. Right? This is something that historically large companies have done as they feared for their technical prowess.

Well, I think it was designed for nobody to understand it. If we want to make something very clear in an investment that we've invested this amount at this valuation, the deal is done. Now, if you want to make something more difficult that's up to this amount over an unbound specified amount of time at no valuation given or a valuation specified, but it can change, right? It wasn't designed for you or other analysts to anchor on different things and that's a very reasonable thing for both of them but it's just it's not an analyzable thing what beyond the fact that Nvidia has chosen to try and lock up a portion of OpenAI's demand by investing in them. That's about as much as you can say.

I don't know if it was meant to be confusing. I think it was meant to be.

You call your mom by her first name. Hold on. Wait a second. Let's go back to the important thing. You call your mom by her first name?

Okay. You don't call her mom? I've never called my mom surely. I mean, I never call her. I mean, that would be mom or something else, but not.

I certainly don't think it was. The truth is is there may be and there likely a huge number of moving parts that make it impossible to clearly describe without giving out more than they wanted to give out. But you mentioned pre-announcements that like B300s, B200s, timings of such. Are we thinking about chip depreciation in the right way? I just, you know, again, I just did a show with Jonathan. He's like, 'Hey, we actually think about them on like an 18-month time cycle to maybe two years.' and I was like wow that's quite quick are we thinking about it the right way and how should we be thinking about the amortization of chips.

We are in unprecedented waters. I think people are clearly still getting value from H100s and that's more than two years right so you know if you say it's a two-year depreciation you're empirically wrong right I mean they are and I think people are still getting value from A100s though not on the cutting edge and so that's closer to three or four years and could be as long as five or six. The question of depreciation is how much faster are future generations than the current generation. That's the actual question on depreciation because what you're with depreciation, you're saying at some point it's no longer worth using a part that's fully paid off because there's a new part that's so much faster, uses so much less power that it's better for me to retire it. That's the actual sort of the underpinning to the depreciation question. If I have a data center and it's 50 megawatts and I have this much capacity in it at some point even though my chips in it have been depreciated and I'm running them at zero cost, right? power plus zero depreciated cost. It makes sense to move them out because the new one, the new chips are so much faster, so much better, use so much less power, I get so much more dollars per and so that's the question. And if we don't as an industry continue to build extraordinarily sort of better parts generation after generation that then people don't move from one generation to the next. They last longer. You depreciate them longer.

I think the question is in that case whether you read people's marketing material or the actual performance results. I mean certainly the people's marketing material would lead you to believe that generation over generation they're huge gains. A little bit of engineering digging probably leads you to the conclusion that you're getting two, two and a half x per meaningful generation move. Not more. Right? If you compare apples to apples, 8-bit to 8-bit, 4-bit to 4-bit, if you compare actual performance, you know, you might have more flops on the chip, but your memory bandwidth didn't improve more than 2x, so you can't get to them. These chips are a solution. And if you make one part fast, it's a system. You make one part fast and the other part you don't move doesn't move as far forward, it becomes the new bottleneck. And it doesn't matter how many flops your chip has. If you can't get data onto and off of the chip, those are wasted. And so the question isn't sort of oh how much faster is the chip it's how much faster is the solution and that includes memory which has for inference is the fundamental limiter for the GPU architecture and so it doesn't matter how much faster the chip goes it matters how much faster the memory bandwidth is.

Well, I not only is it fair, it's the reason we went to wafer scale. So, let me explain what your friend said is strictly true in that SRAM is blazing fast and low capacity. HBM is a flavor of DRAM. It has high capacity and it's very slow. Now, Nvidia and all GPUs, including AMDs, chose a big capacity memory that is slow because it's perfect for graphics. You don't have to go to memory very often. You can hold a lot, you don't go very often. SRAM is blazing fast, but it can't hold very much. So the problem on traditional chips is if you put memory on the chip, you are using space that could be otherwise used for compute, you have a fixed amount of real estate. And so if you put half memory, then you have half your real estate available for compute. And so our idea was that if we sort of if we built a chip that was the size of a dinner plate, we could stuff it to the gills with fast SRAM, overcoming the limitation of SRAM, which is it doesn't store very much by putting a huge amount down by using more silicon area. Now, if you're an SRAM solution today in a normal-sized chip and you're trying to do a trillion parameter model, use four or 5,000 chips. What a mess. You know how many cables that is? Do you know the impact to the AI? It's a horrible mess, right? And it limits you from doing things you want to do with the AI. Like speculative decode. It has all sorts of painful challenges. The other hand, use one of these or two or four, right? And it's simple. It's easy. And this is what your friend said. Exactly. Right. And the reason we went to build a bigger chip so we could fill it with this fast SRAM so we could get over the traditional limitations of SRAM that couldn't store very much by using a lot of space, by using a huge amount of silicon area. So your friend is exactly right. You might listen to him again in the future.

It does, doesn't it? Yeah.

Well, what we were missing is that for 75 years, nobody could do it that building a bigger chip had proven impossible before we did it. Nobody in the history of the computer industry had been able to build a chip bigger than about 840 square millimeters in the 75 year history of the computer industry. And many people had tried and failed. And after we did it, Elon tried at Dojo and they failed. And so it's really really hard and our strategy had never been done before, never been successfully sort of yielded and so while it was obvious it was hard.

No, we're faster on both, but the software challenges in training are real.

It means that when a new model is built, all right, and everybody reads about it in a publication, it was done on a GPU. everybody to train it. They take the recipe that was originally done for the GPU and they have to move it to the recipe for their hardware, whether that's a TPU, whether that's an AMD GPU, whether that's another dedicated chip like ours. You have to move it. And that's a harder software lift. In inference, the truth is, nobody cares about CUDA. Nobody even cares about PyTorch. All right. What they want is an API. And so it's literally 10 keystrokes to move from a GPU-based solution on OpenAI OSS 120B to our solution. It's 10 keystrokes. That's it. It's nothing. And so I think the answer is that while we are faster at training and we are faster at inference, it's easier to demonstrate inference, right? You just put up a side by side to show that you're faster than a thousand B200s. You know, you got to get a B200s. You need to train them for six weeks, train the model for four weeks or six weeks. You got to stand up a cluster of our machines. You got to it's a bigger lift. So I think the market right now is easier to move people off GPUs in inference and the number of people doing inference is vastly higher than the number of people doing training.

The rate of growth of inference is the number of people who use it times the frequency of use times the amount of compute needed per use. Right? It is three different variables multiplied by each other. The problem is they're all growing fast and that produces some mind-numbing effects. More people are using AI once they start using AI they use it more frequently and what they want to do with it is bigger and more complicated. So it uses more compute and so you have three variables. the size of the markets, the product of the three, all growing fast. And we knew that going in. We see that and it still takes your breath away.

I agree with that. No, no, I agree. I think the reason I'm 100% sure that we are underestimating the market is because of that premise. I think Sam Altman said it very well in terms of how people use ChatGPT, but he said, you know, essentially the majority of people use it like Google, a Google replacement, and actually the younger people use it as an operating system for the future, which is the right way to do it in his mind.

I think and this is there's a famous there was a I was just going to put up a post on this in 1988 Robert Solow who won a Nobel Prize in economics. He asked this question. He said we see computers on every desktop and everywhere we look except in the productivity statistics. And you say, well, that's a really interesting thing to say. And there was another economic historian who jumped into the fray and he did this huge study. His name was Paul David and he wrote a very famous paper called the computer and the dynamo. And what he studied was the adoption of electricity in the manufacturing sector between about 1880 and 1955. And what he showed was at the beginning electricity produced very little productivity gains. It was basically used as a backup for belt driven systems. And it wasn't until they reorganized the shop floor to take advantage of electricity that you got this huge jump in productivity. And if you roll that forward to the computer, what he was saying is look, we used computers to do things we're already pretty good at. that we replace a typewriter, we replace general ledger accounting with spreadsheets. We were good at those things. You didn't get a big jump. And what he predicted and then happened immediately thereafter, by the mid-90s, you had a huge jump in productivity. We had begun tying them together. We built the internet. We had the first parts of a cloud. and all these things used compute differently in ways that had never been consumed before and you got this massive jump in productivity. If you use OpenAI and a variety of their competitors the way you use Google, you'll see a very modest jump in productivity. If you use them in a fundamentally different way, that was Sam's point. You'll see a huge jump. And if we use if we reorganize ourselves around AI, you're going to see massive productivity gains. If we use AI to replace things we're already doing, Google or something else, you're not going to see very big jumps at all. And so what I know is that transition takes time. And what he pointed to was a demographic, right? Younger users are using it in a different way than older users. Older users are replacing something they already had. Younger users are using it in a way that never existed before as an operating system for life.

Yeah, it's feasible. Is it desirable or good for society is a different question. It's feasible. People often say we don't have enough power in the US and this is strictly wrong. We have plenty of power. It's in the wrong places, right? It's not where we have people or where we have fiber optic cable, right? We have a ton of power in West Texas in natural gas. We have a ton of power in upstate New York in hydro. We have a ton of power in lots of places. We don't have people there. And so, the problem is one of a mismatch between where all the power is and where the people are or where the buildings are or where the telco fiber is that we need to get data to and from the data center. So, that's the first observation. The second observation is sort of one of a community and that's that to the extent we consume this extraordinary amount of power, we have an obligation to deliver amazing things. Right. And that's not all of us. Right. I think we have an obligation to deliver drugs that are more efficacious, to deliver better health care, to make aging less painful, make the looking after of aged parents or sick parents. I mean, you go through society's ills and woes. If we are going to consume this amount of power, the burden is on us to deliver value for it. And if we use it and don't do that, then it's not a game for society.

It's a very hard question. It's a very hard question. And I, you know, I'm one voice in this. The problem with markets is that they do a lot of things that aren't productive in order to get one that is very productive. Ghibli may or may not have been net society gain, but maybe the technology that is used in Ghibli is used for X-ray crystallography and later is fundamental to finding major scientific breakthroughs. I mean that's the messiness at any given point in time, right? you can point to in a thousand poppy strategy, which is what a market is, right? A market has a lot of bad ideas to get a few good ones, right? That's your business. Your business is investing behind a lot of big ideas, most of which fail. And the market is that large. And so in that environment, you can always point to, oh, that was a bad investment, Harry. Why'd you invest with them? I mean, they blew up. you can always say it after the fact. Oh, look look look at that. They're using a ton of energy for that. That's not useful. Let's see. And so I think the answer is we need to be sure that at a societal level where we use government dollars, where we use tax breaks, where we use permitting breaks, we are sure that we are giving these disproportionately to projects that matter to society.

I think it's confusing. I think on net is probably more to help. The Biden administration was misguided and afraid. And I think the Trump administration I think to his credit they surrounded himself with some smart people in the AI space and I think they have on net it's been positive.

No, it's not unavoidable. It's a very reasonable decision for countries that don't have lots of alternatives. I mean, you know, Canada has more falling water than anywhere else on the earth, right? the opportunity for Canada to develop the cheapest power on earth is mind-boggling. There is cheap power in lots of places. but in order for countries that wish to and don't have the natural resources of Finland that has geothermal or Iceland that has geothermal or Canada that has falling water nuclear is a very reasonable and cost-effective strategy especially over a several decade view.

I do think of this idea that to consume the resources we're consuming, we have to be sure that we produce some extraordinary outcomes. I worry the opportunity is so big that as a community we're running sort of helter-skelter at it that actually sometimes instead of running where you trip and fall and graze your knee and chip a tooth, if you stopped and thought and marched, you might get further over a 30 or 60 or 90 day period.

The risk there is that not that they consume that much that they are that much value. That's not the risk. I think they're that much value because the future economy that we believe the future economy will reward that. I think the issue is that people then think the S&P is a safe investment or a safer investment than it might be. The risk is the mismatch right in the mental model people have right risk comes in financial markets where people fundamentally underestimate risk when risk is priced properly. All right. Then what happens is your outcomes are not surprising. But you know I if people continue to think the S&P is sort of an index of the global economy or and it's not it's 30% or 50% seven companies. Then they're exposed to sector risk that they weren't signing up for. They thought they were diversified and in fact they're heavily dependent on a very narrow sector and that's a risk. Do you know what I mean?

Yeah. that seems to me to be a challenge in the new world order and all the advice that sort of the pundits give you know a diversified portfolio when the world changes and that portfolio is not you're you keep holding it and it's not diversified anymore because of consolidation then there's real risk.

Look, I think they've proven themselves to be the greatest company of the first quarter of the 21st century, right? I mean, that they've proven themselves to be the extraordinary company in the first quarter of the century. And I don't know if 4 trillion is right, but I think a very big number, maybe it's too low, is right because of what they've achieved.

I think if you go back to planning, if you've got customers demanding 5x your capacity, I mean, you probably didn't

Get your planning right. Right. You probably should have planned better. I think there are bottlenecks at every level that are meaningful. I think the first one is expertise. I think we have fundamental limitations in AI expertise. We're not making enough AI practitioners. We're not making enough data scientists who understand data pipelines. We're just not, our universities aren't minting enough. And our challenges in the US with immigration don't help that. We have historically sucked the best and the brightest first on J1s to come to our schools and then H1s to stay. If that is not our policy, we need to make them right. If the government decides that that is not the way they want to build a workforce, instead they want to build it out of people who live here, we need to do a better job of training those people. We need to do a better job of teaching them in K through 12. We need to do a better job of educating them in our universities in order to make the number of engineers we need to meet this demand. That's a bottleneck and it's why the best and the brightest are getting such extraordinary compensation.

There are engineers who have skills that no number of other engineers working together can achieve. There are scientists who have ideas and who have brains that can't be replicated by lots of other talented people working together. Ought they to be paid more than world-class soccer players? I have no idea, right? Maybe, maybe not. But I mean, inherently, yes. From an economic rationale standpoint, yes, the value generated from a chief scientist at OpenAI, if they add $50 billion of enterprise value to pay them a billion dollars is worth it. That's what we have to think about. I mean, we've, I don't know, we paid Charlie Sheen $2.5 million an episode for Two and a Half Men. Two and a half million an episode, right? I'm pretty sure that there are lots of people whose net productivity to society is above that.