We can't build data centers fast enough to keep up with demand. We have a $25 billion backlog. If demand stays high, we are going to continue to see memory shortages for at least the next several years.
I'm so excited to welcome a dear friend, Andrew Feldman, founder and CEO of Cerebras. Last week, Cerebras went public. The largest semiconductor IPO ever. The price went from $185 to $311. They got over $5.5 billion. Today we deep dive on the future of chips, the future of US-China relations, the future of data centers and energy. This was an incredibly wide-ranging conversation.
I think it has been Nvidia's strategy to try and create competitors for the traditional hyperscalers. They have funded and backstopped and overallocated to the Neo clouds. They have created a dependence which is probably not healthy. So over time, the history of our industry is a massive reduction in the cost per unit compute for hard problems. There is no upper bound to how much faster you want to be.
Andrew, dude, it is so lovely to have you on the show. I have to say I was quite emotional last week when I saw the IPO because you are one of the kindest, greatest people. I love getting to know you. I so appreciate our relationship and so to see that culminate last week with the IPO was really special. So congratulations for last week, dude.
Well, thank you so much. Those were really kind words and it was a really exciting day for the company and the team and the people who believed in us and who backed us for a decade and it was great. Thank you for saying those nice things.
Not at all. And I was thinking like in terms of this conversation how I wanted to structure it. And I always go back when I have amazing people like you on the show which is Eleanor Roosevelt's kind of statement of you know not very intelligent people discuss other people, mediocre people kind of discuss current events, and then intelligent people discuss the future and ideas. And so I thought I'll grapple with my own ideas and wrestle with your incredible brain to help me understand where we're at and where we're going. I want to start with on the one hand we look at the landscapes there and it's like oh my gosh an AI infrastructure bubble and then on the other hand we look at you know Jensen Huang last night who comes out and says we're going to be spending 3 to 4 trillion on AI infrastructure by 2030. How should I balance the oh there's an AI infrastructure bubble with this appreciation of 3 to 4 trillion spent by 2030.
I've been thinking a lot about this. I think when you look at other bubbles and you look at bubbles in the past, and I was in one in the late '90s when we built out an enormous amount of fiber optics. And you sometimes have economists who maybe think it's relevant to look at 1880s, the building out of rail. I'm not sure that's relevant, but what I see is that there was a tendency to believe that if we built it, they would come. The infrastructure buildout was way ahead of demand. And that was true in railroads. That was true in fiber optic cabling. And in a strange way, that is the exact opposite of where we are with AI. The infrastructure buildout is behind demand. We can't build data centers fast enough to keep up with demand.
We have a $25 billion backlog. Nvidia has a backlog. AMD has a backlog. Others have backlogs. They have backlogs because we can't get data centers built fast enough. And it's not that we're building on the come. We're not building ahead of demand. We're building behind demand. And that is a very different observation than those who say there's a bubble. I don't think they've really gotten their head around the fact that we are trying to keep up with demand, not the other way around. And I don't think that's a characteristic of a bubble. When you are trying with your infrastructure to keep up with what people want today, not in the future, today, and their demands are growing over time.
Is it ultimately a good thing that we are metered in our ability to build out data centers because it almost tempers the demand if we were able to? Gavin Baker said actually kind of the delays and the permitting and the challenges that are incurred today actually help because if you were able to have it all today all of demand would be met with all of supply and that would actually be a challenge.
Well, look, I think sometimes the world is like I was in my 20s the first time I went to Vegas and went to the buffet, right? You eat so much you feel sick for days, right? It's all in front of you and you just gorge yourself. I think the market can sometimes be that and I think Gavin is an extraordinarily thoughtful sort of guy about this. I think we are being metered.
We also know that the reason you put meters on a freeway is because it makes the freeway traffic smoother and it avoids hiccups. That's exactly what metering is designed to do. And so I think he used that analogy extremely thoughtfully. One of the advantages that OpenAI has was that and I think one of Sam's brilliances was that he saw an exponential growth and he saw what that would mean in a year or two to the demand for compute and he wasn't afraid by it. He went out and took action and perhaps others couldn't believe it or they were looking at the same demand sort of steep exponential growth and were like well we can't need that much. You can't need tens of gig that my mind hurts if you do that. Whereas what OpenAI did is they went out and they're like, we're going to contract for it here and here. We're going to get power. We're going to get data centers. We're going to sign up for hardware and an ability to believe your data in an exponential growth environment out a year or two or three is a superpower.
Can I do you get rewarded for that insight if you can just buy it from Elon now on demand?
I don't think they can buy the same thing from Elon on demand. They bought down rev gear.
I'm sorry. I've learned from doing this show for a long time. I can ask stupid questions. They bought down rev gear. They got H100s. They didn't get the B200s. They didn't get the most current. They are a generation and a half, maybe two generations behind. So this was not a great deal. It was a good deal for Elon. He had them sitting around, but they were forced to take action in a deal that I think was not the ideal deal they wanted. It was a deal that was available.
Going back to what we said about kind of the delay in data centers and data centers being a constraint. I just hear everyone say well memory memory is the shortage too Harry and that's why we're seeing an increase in cost you know four 5x in certain cases. Is memory is that true how should we think about memory being the shortage as well.
Well, look, what's happening here is there is such extraordinary growth in demand that it is putting pressure on all parts of the supply chain. Memory after TSMC which is right after fab, memory is number two item in this, number two item that's needed. And what's happened is there are only three companies that make the memories GPUs use. We don't use that memory. But that HBM is made by Samsung and Micron and SK Hynix and they couldn't keep up and so the prices shot through the roof. I mean Micron producing numbers where they have 80, 85% gross margins. I mean they're getting software gross margins on making memory. Yeah, I think it's extraordinary. That is a limitation for all GPUs but not us. We don't use it.
Again, going back to my idea of what the future looks like, what should one expect from that? Does it ease? Does it ease over time? What happens to the cost?
Well, I think the challenge here is that these are extremely lumpy items, right? You can't just add a little bit of manufacturing capacity at a fab. You have to build a fab for $40 billion and takes five years to build. So if you see demand explode, you cannot respond quickly. All you can do is fill your factory. Once your factory is filled, you got to build another factory, right? It's a step function in your ability to meet that demand. And the step is huge and takes years. And so if demand stays high, we are going to continue to see memory shortages for at least the next several years.
Do you think we will see a peaking of demand? You've seen so many different.
Not if AI continues to improve in usefulness. I mean what's happened here is and this is something that I haven't heard others sort of talk about is somewhere in 2025 the models got smart enough to be really useful. Before that Harry these were sort of a novelty. AI was like cool and then nobody used it. Remember we make AI with training and we use it with inference. And so once the AI we made 2025ish first half got smart we began using it and this explosion in demand that Jensen described and that we very much agree with is happening. That's because people are using it every day and they're using it on more and more problems. They're using it on harder problems and it is sweeping through different demographic groups. It's not just 28-year-olds in Silicon Valley. It's my 85-year-old father. It's my 11-year-old niece. It is sweeping through demographic groups and they're using it all the time. And that is what's driving this demand. And so if we continue to find ways to make the AI the frontier models smarter and more useful, we'll keep using it the demand will continue on this sort of exponential curve.
You've compared past cycles before in this conversation. And Sarah Fry said about kind of cloud providers can be similar in some perspective to what we're seeing today in terms of frontier models and she said that last night. To what extent do you think you see the commoditization there and they essentially become utilities versus differentiated providers with meaningful moats.
I think it has been Nvidia's strategy to try and create competitors for the traditional hyperscalers. I think that has been a strategy of theirs. I think they have funded and backstopped and overallocated to the Neoclouds. I think they have created a dependence which is probably not healthy. But I think the truth is that what AWS and Azure offer is extremely useful for most enterprises.
They offer credibility and legitimacy. They offer security. They offer layers of different software for different parts of your organization. If you'd like to enter in the AWS world, you can enter with Bedrock. You can use tools like SageMaker. You have a collection of different ways to enter and you can store your data there. You have your S3 instance. I mean you can have an entire offering and I think that is really valuable to a segment of the market. I think there might be other segments of the market they're like give me cheap compute. I don't care about anything else.
And in that case your strength as a hyperscaler becomes your weakness. You have the security you have the other layers of software and you have some of the costs that are associated with that. And if people don't want that, if you don't care about leather seats, right, and there are leather seats in the truck, there's extra cost in the truck. And when you buy the truck, right, you find somebody who's got a truck that's got vinyl seats, right? I mean, it's exactly the same. Our business just because it's wrapped up in technology is no different than any other business. It's segmented. There's value. That value comes at a cost. You have to make that value. The hyperscalers make the value. They make the value through software, through security, through having rules about their data centers, about the security, physical security, the various security checks they put in. Those are enormously valuable to most parts of the market, but not all.
You said about the cost there. When we look forward, how do the costs of your business change significantly over time? We spoke about the cost of memory going up five. If we look at the cogs in five years time, how do you think they will look most significantly different?
Well, the cost of the increase in memory has been very good for us because we don't suffer it, right? This has given us opportunity, right? We use SRAM and there's no shortage of SRAM. The cost of SRAM hasn't changed. And you know, no SRAM maker because TSMC etches it into your chip while they're making the logic. There are no extra margins to pay the HBM maker. And so, we have been advantaged in this environment. We have been advantaged by the fact that there are constraints on CoWoS at TSMC. We don't use CoWoS. We are advantaged by the fact that we're at 5 nanometer and the 3 nanometer node is the most oversubscribed. And so our supply chain is advantaged on these dimensions. But they and others are paying the price. The price of GPUs has gone through the roof.
And so to my question on cogs do we see like a plateauing of cogs in terms of it can't get cheaper and this is the stable state. Do we see a meaningful reduction? How do.
I think what happens over time, Harry, is that we all of us. We improve our designs. The designs deliver more tokens per unit time. They deliver faster tokens. Now, we are 15x faster because of architectural reasons. We will continue to improve over time. Nvidia, they will continue to improve over time. I believe the gap will widen between our performance and their performance. But all of us, the whole industry, us, Nvidia, AMD, Qualcomm, ARM, everybody's chips will be better in three or four years than they are today. They will produce more per unit power and they will produce more per dollar cost. So over time the history of our industry is a massive reduction in the cost per unit compute.
I was chatting to a friend who is a phenomenal mind and he said that Google will become the lowest cost producer of tokens because they own the full stack from TPUs to data centers networking power procurement. Do you think that's right that that full stack ownership will lead to their highest margin lowest cost ability?
There are pros and cons of that strategy, right? The pro is you have everything from the ground, right? Land all the way up to tokens. The downside is you can only sell your TPU to yourself. And historically, volume mattered a lot. And so your market is constrained by your own demand.
Whereas if you were able to sell to the whole market, you might have more demand and be able to drive down the cost. It's an open question. Google is threatening that argument. I think your friend's argument is reasonable, but there has historically been a challenge if you only have one customer yourself for your hardware. And that has historically limited the size of the opportunity landscape for you.
Do you think they should sell to external customers?
I think you are already seeing them step outside of their own data centers for this exact reason. Right. What it says in your friend's construction is our ability to sell hardware is constrained by our ability to build data centers. Now one can imagine a world where you don't want that constraint. You would like to be able to sell hardware to anybody's data center.
And so I think these arguments are extremely complicated. Rarely unfold in a simple form. But it is true that when Google or when Cerebras puts our equipment in our own data center, right, we have a significant advantage over a Neocloud because Neoclouds are buying hardware with gross margins of 70, 80% for Nvidia. So the hardware in those data centers and then they have to make their margin. That's not what Google's doing. That's not what we're doing.
Does that mean they're dramatically overvalued when you look at a Nebius or a CoreWeave or any of the others?
I think CoreWeave has been an extraordinarily innovative company. I think they've solved a series of financial challenges with really innovative sort of financial engineering. They were the first to use debt in a very innovative way. They get enormous credit for that. They have been extremely good at sort of rapid deployment which itself is a really important skill in this environment. I don't know about the others but I think all of us have challenges as our business grows and I think they have produced really interesting things through creativity. Now, it's different creativity than what I have. But they've gotten paid for real innovation in financial thinking.
Speaking of real innovation, I saw the post about you running Kimi K2.6, 6.7x faster than the next fastest GPU cloud, right?
We posted it while one bozo at an analyst firm was on TV saying we couldn't do it. I mean, if ever there was a sort of example of being empirically proven dead wrong to have these numbers posted while you are on TV saying they can never do it. It was perfect. I enjoyed that. I'm a collector of examples of people being dead wrong.
My wife has a list. My wife has a list of when I'm dead wrong. So, I've sort of embraced this and collected.
You should be a venture investor, my friend. With a portfolio of 30, you'll be dead wrong a lot.
You have, if you're lucky, 80% of your portfolio where you were dead wrong.
You should do if you're doing it right. Yeah, that's right. That's right.
I agree with that. How important was that for you? And is there a stage where actually it doesn't matter being that increment more important like 6.7 times. This is so much more important. It's not 20% more important.
That's right. You know I think for hard problems there is no upper bound to how much faster you want to be nor the value of speed. If in three minutes we can solve problems that take others 20 minutes, then think of all the extra problems we get solved. And think of if I'm your competitor and I'm solving your hard problems in 3 minutes and you're taking 20. Imagine over a day or a week, you get smoked. You will be smoked in this example. And that is the way this is going. Speed is of the essence and it's true in coding. It's true in agentic flows. It's true in every part of the AI landscape.
I mean, let me just ask you this question. How big's the market for slow search? Really, how big is it? It's zero. How big is the market for dialup for slow internet? Right? How much would I have to pay you? Let's try turn it around and say there's a negative market here. If I gave you $1,000 a month to have slow internet in your home, right, you wouldn't take it. $1,000 a month, you wouldn't take it. That's how impossible it is to engage with an important technology slowly. Why do we believe that inference will be any different? There will be zero market for slow.
I am kind of pushing them. When you power Codex and you're able to be so much faster, if you're Claude Code, are you not like ah bugger? They are. I think you have to be. Are you able to sell to them? Also, again, please tell me to shut off.
Oh, no, no, no. Look, I think right now we are digesting one of the largest deals in the history of Silicon Valley.
You're like, for fuck's sake, Harry, give me a break. I've just signed a 20 billion deal. You want more?
You know, while we were on the road, some investors would ask, they say, 'Oh, you're heavily concentrated. You have a big portion of your business with OpenAI.' And we say I talked to you a year ago when I had a billion dollar deal with G42 and you said you're heavily concentrated. You have a billion dollar deal with I said I come back to you in a year with a 20 plus billion dollar deal. And you tell me the same thing.
But with a different customer as well.
With a different customer. And I tell everybody that the way you get good at and the way you succeed with many customers of size is first you win one. The way to catch big customers is first catch one and learn, build the muscle, change your supply chain, learn how to work with a large customer. Then you're in a position when the next one comes to have a chance to win. And what's more, chance to keep them happy once you won. And then once you have that muscle, you're in a position to go out and win the next one.
What are the biggest challenges in fulfilling it? And with the greatest of respect, do you go to sleep at night going that is quite a lot.
Look, I think what has happened and Sam said this. He said the first time people used GPT I think it was what four something they said oh this is amazing and the next day they're like how come it's not faster. The rate at which you get accustomed to something and then want better is amazing in our industry and it used to be the case that 20 megawatts was a lot and then 100 megawatts was a lot and then a gigawatt was a lot and now we're running around looking for multi-gigawatt facilities.
And that's in any other time 750 megawatts would have been a mind-boggling amount and now we're like yeah we got that. Right. It is the change in mentality over the last year or two for everybody in our industry has been sort of extraordinary.
Five years ago if you'd have said we're engaged in a multi-gigawatt buildout or you think about what Crusoe is doing or think about some of the other cool companies, what SoftBank Power is doing, what some of these groups are doing, and you say, 'That would be delusional 5 years ago.' And right now it's like, 'Oh, another one. Yeah, that makes sense. I mean, we should try and get our UAE Stargate at 5 gig.' Oh, oh yeah, yeah, no problem. That seems reasonable.
That's what's happened. It's this extraordinary extraordinary sort of change in thinking.
If we are nonchalant to multi-gigawatt build hours today, what are we in five years time?
It's difficult to imagine.
And by the way, that is exactly where I think Sam is the best in the world. Maybe Elon is where everybody else's brain shuts down, right? When you're trying to think about 100 gigawatts or 500 gigawatts, those guys, they have sort of this ability to not be constrained by sort of the way the world has always been. And that is such an extraordinary power.
When you have such scale as the multi-gigawatt you mentioned there the 500 gigawatt, does energy not just become the core crux and bottleneck that enables winners and losers? I certainly think that people like Sam and Elon and others have said that's what they believe.
That at the end we're in the business of turning electricity into intelligence. And therefore, the limiting factor is electricity. I don't know if I agree with that, but that is certainly a very reasonable view from where we are.
Why? What's the bear case against that? What's the alternative argument? It doesn't have to be yours, but that you bump into something else that what happens in fact is our models can't keep getting smarter. That you hit something that has an assumption built in that the models keep getting smarter and you keep feeding them more energy. And at the end of the day, the models are either smart enough or keep getting smarter so that it makes sense to keep feeding them energy. That might be true. I don't know.
Do you think we'll be able to build out data centers in the way that we need to and build out capacity in the way that we need to when we see that 40 out of 100 data centers are now not being built out even post approval because of local municipalities permitting disruption. AI is not popular.
Harry, I think it's hilarious. People say, 'Oh my god, the data centers are late. Oh my god, they're delays.' Like, 'Have you built a kitchen? Your contractor was late.' Right? Pick a little tiny project in your home. Right? Was it built on time and on budget? No.
Now imagine building something the size of 50 football fields and requiring interaction with local municipalities and power companies and regulated industries. These things aren't going to be delivered on time historically. People's mind explodes and they never think about their own experience in their own homes. Right. Does your contractor show up every day? No. Right. Does he do exactly what he says he's going to do? Rarely. Right. Do the materials, the tiles or whatever you selected for your home, right? Do they sometimes delayed? Yes. All that same thing happens when you build a data center, right? The generators, sometimes they're late. Sometimes they fall off a truck. Literally, they fall off a truck and damage is done to them. Are the transformers late? Sometimes the transformers. I mean that everybody suddenly sort of throws their arms up and says, 'Oh, everything's late.' or they have to deal with localities.
Anybody who's built anything big knows this is par for the course. This is what building is.
I totally get that. So, you're not concerned then about bluntly local neighborhoods seeing data centers as.
Yeah. I think our industry did a shitty job of engaging the community properly. I think Brad Smith put out a post a while ago that should have been the way we all work from the get-go, right? And it was these can be clean. They can make jobs. They can be good for communities. We can do this thoughtfully, right? These create thousands of local jobs.
Thousands. And thousands of local jobs means restaurants and lunches and hotels and a whole Right.
The way they were done was I don't know if sneaky is the right word but sort of shielded and wasn't open and they weren't good neighbors. Now there is no reason why we can't be good neighbors. There's no reason why we can't add these to communities and have the community benefit from it, right? And we have to do some thinking, right? We have all the heavy equipment out there. Build a football field for the local school. Build a school at a church or synagogue to the community, right? We can be good neighbors at very very low cost. We can pay our own way, right? Don't try and use sort of loopholes in the way power companies have historically amortized the cost of new power lines over 30 years and push that on the community. That's BS. We ought to pay our own way. We ought to look after our neighbors. And when we do that, I think that the neighborhoods that embrace this will benefit enormously. But we didn't do a great job. I mean, we didn't do a great job as an industry at all.
Feels like kind of the cartels in Colombia where it's like, you know, they built the churches and they built the schools and they had such good businesses and such high margins that they could kind of get away with it because of that. And hey, great.
I don't think that's right. I think these localities have a resource that isn't being used. They have power. And many of these land is cheap because nobody wants it, right? You're not going to parts of Metro New York. Everybody wants that chunk of land. You're going to places where the price of land is depressed. You're near power resources.
And my position is that we ought to be good neighbors and we ought to be transparent. We ought to pay our own way. Now, this seems not to be very controversial in my mind. And I think the best sort of and most concise description is what Microsoft has put forward. And we should have been doing that from the get-go. Everybody is or most communities are comfortable when their neighbors pay all their own way. And that it's only when groups tried to shift costs or not pay for the full resources they're using. Our data centers don't need to use a ton of water. They can recycle it. You can have a closed loop. We can pay our own way. We can upgrade substations. We can upgrade sort of grids and pay for it in its entirety. We shouldn't be pawning that off on local communities.
Do you worry about like AI as a brand? You see Meta lay off a huge amount of people today and it's challenging to see at 4:00 a.m. emails from Zuck and you know jobs being lost.
I do worry about it. Those are people and they have families. I think there are sort of two views, Harry. I think to date most of the layoffs were AI-washed. They were because we did boneheaded hiring during COVID.
It is actually because a great deal of productivity gains has occurred over the years that we're just now harvesting. The ability to gather information from across the organization to synthesize it and put it in one place is now changing what it means to be middle management, right? The role of information gatherers and presenters is being eliminated. The ability for us to automate roles and none of this is AI yet, right? That is really 90%, 95% of what in my view what the terminations have been about. It's easy to put them under the umbrella of AI. Now AI is starting just now to have meaningful enterprise impact. But if you are an engineering organization that can't see how to take advantage of vastly more productive engineers, I
I don't think you're long for this world. I mean, the list of things I want our engineers to do is 50 times as much as we have engineers, right? If we get more productive, we do more things, we're going to hire more engineers. We're not going to hire less engineers.
And can I ask you, we saw Benov say that he spends 300 million a year on Anthropic, which equates to about 3.8% of developer salaries on Anthropic. To make it justify the valuations that we're seeing for these companies, it needs to be 20%. Do you have any concern in that movement from 3.8% to 20%?
No, I mean, I think if you look at, and I've never done this in any detail, but if you look at what we pay hardware engineers and you look at the tools, the EDA tools they use, I bet you're much closer to 15 or 20% than 2 or 3%. What's happened is historically software engineers used very low-cost tools and hardware engineers used extremely expensive EDA tools. And so that's interesting, isn't it? I mean, I think the cost of bugs in hardware is so high that we became accustomed to using many expensive tools and in software we threw people at the problem rather than tools. And as AI becomes more productive, I certainly don't see a problem where software engineers are using 50 or 100,000 a year each in tokens. There are 47 million software engineers in the world. I mean, that's 5 trillion just in software engineering token use.
We mentioned hardware engineers, we mentioned software engineers. What role does not exist today that you think will be incredibly commonplace in three to five years?
So I've been a part of over the past 25 or 30 years several technical transformations that produced jobs in companies that didn't exist prior to the mid-90s. The role of CIO didn't exist. CIO arose as a role with Cisco with their sort of rise to dominance. Prior to the mid-90s, the amount of enterprise networking was to minimize. There was a role that was often VP of telco infrastructure. That job is gone. We don't have a phone system. In fact, we don't have phones on people's desk. Call me on my cell phone. Right? That job disappeared completely, gone, and companies that built the PBXs like Rome and all these other, that business has shrunk to nothing. Now later what happened in the 90s and 2000s, right, with the rise of Palo Alto Networks and these other security, the role of CSO never existed prior to that. All right, and what you're going to see is the rise of roles that reflect the governance of AI in companies. Some companies have chief AI officers. I don't know if that's what it is, but as these technologies sort of become important in a company's life, new jobs emerge, jobs that never existed before, new organizations exist where there were none and previous ones disappear. I think the role of HR changes fundamentally. The part of HR that just answered questions, that provided information about benefits, that disappears. AIs can answer all your questions. They can provide better answers, faster answers, more thoughtful answers. It becomes something different. The management of people becomes something different. I think there are all sorts of other parts of organizations that have fundamental changes because AI can answer the questions that they used to answer.
