You got board meetings every six weeks and all you've got to say is still can't make it. Right? Is that the board meeting? I mean, what else are we going to talk about? Still can't make it again and again. And they're like, 'Should I get in? Can I do anything to help?' That's right. They're like, 'Can we help?' It's like, 'Nope.' But we always believed that if we could make it, there would be huge demand for it.
All right. Super excited to be doing this today. Andrew, thanks a bunch for being here. So Andrew, you're the, you know, CEO and founder of Cerebras, which is obviously one of the most important chips companies. And also really happy to have my partner Eric here, who's going to be kind of half host, half guest, who, you know, has been working with Andrew since the beginning. But thank you both for making time. What I want to start with is it's 2026 now and obviously Cerebras is very important in the sort of AI landscape but you started the company in 2016 and AI was not what it is now back then so I guess what was the kind of headspace then what was the idea the insight like what were you building when you started the company and what was the thinking?
Well Jack thanks for having me. It's always fun to hang out with Eric and so appreciate you having me. I think as a computer architect when you see a new workload on the horizon you get excited. It's very difficult to win share in a mature market in compute and so when something new emerges you get excited and you lean forward and you say well can I make it faster can I build a chip that is better at this work and then you ask the other question is is there enough of this work to justify building a chip that's better? So you have two questions. Can I and should I? And what we saw with AI was an extremely computationally intensive workload. Unlike for example the rise of ARM processors for cell phones that they weren't computationally intensive, they were power intensive. Right? We saw a problem that was going to be hard on compute and we saw that what was being used was an architecture that was being remodeled for it and that we could build something better. And that was sort of the insight and then it was a long hard slog.
Before we go to the long hard slog, Eric, obviously for you, like as an investor, you know, you were not, you know, also in this AI era, but you saw something that made you invest and you were, I think, you know, previously not a chips investor and you done more like
And probably never will be again.
Probably never will be again. Like, would you do that every time again? Was there something that made it clear to you or was it like what happened for you when you made the investment?
I think one of the scariest things about venture and I think about this all the time is the more you work on companies and you see the challenges and you see how hard it is you build up scar tissue and then the thing is like well you should do it again but like will you do it again and I ask myself that all the time like there you actually need some naivete like you need to have this like oh can do attitude and this like naive bent around it to attempt it like otherwise you just don't do it again. And I've definitely built up a bunch of scar tissue. And I don't know had I actually had any idea how hard it was going to be for them, like how much science and technology had to be built, how many innovations, how many times like the company was going to come to the point.
Don't tell your VCs how hard your shit is. And the moral of the story is tell them it's no problem. Keep it inside. Push it down. Push it down. Just don't tell them hard problem. All's good, man.
So, what happened? So, you basically, you know, you start the company, you get to the tape out, and then you're done, right? It's like that.
So, how's it go? So, what happened?
I think we had, you know, this isn't our first chip company. This is my fifth. And the team had been together, the founding team had been together at the last one and we had some really clear sort of philosophies on what you should do and our view is that if you're going to attack Goliath if there's a giant standing in the market like Nvidia was even at that time that being a little bit better or a little bit cheaper is not an available strategy right they had high margins so if you come in and even if you're two-thirds the price, they can just cut cost. They can just charge less. They can bundle it. They can do a hundred other things. What that means is you have to go out with something way better, 10, 100, 500 times faster. You have to come up with a product that has a value proposition that even if the other guys give it away, all right, they can't compete. And so that's sort of one direction of our thinking. And the other building off that is we want to do hard things that produce that advantage, right? That we want all that hard stuff within the building because that's under our control. All the other stuff's not under our control. But if we can build something that is so fast nobody else can do it, you can't give away your parts to achieve what we can achieve. Then you're on to something. And the only way to do that in our opinion is radical innovation.
You can't incremental your way to vastly better.
So what has to be in your control? You have to have all aspects of the design, the implementation, you have to understand the manufacturing, you have to understand every part. And for us that meant chip, board, system, software, all the way up to the API. And that was not easy. That makes it more expensive. It makes it take longer. It means there are more ways to fail there. When you do radical innovation, there are no vendors waiting for you, right? When you build a chip the size of a dinner plate, you can't go to a catalog and find a heat sink, right? Because nobody's ever built one bigger than a postage stamp. So nobody has stuff ready for you. And so you end up investing an enormous amount of time in building things that support your thing, right? The surrounding components. But the result of that is you develop this extraordinary expertise. We didn't start world leaders in packaging. We're right now the best in the world of packaging. We earned it failure after failure year after year until we got it. And so what you want to do is you want to think about sort of how you can innovate across the various elements of a full solution and push as hard as you can.
I think like Andrew taught me this in semis and everything else just to like make this like what you need to be shooting for concrete. So if you think of whoever the incumbent is in whatever market you know they're getting say twice as good every year in a market like this is very dynamic for a new company to get to scale to start to get to scale it's going to take five years at least let's say with everyone moving as fast as it can AI tools you nail everything first tape out works first bring up works you know everything goes right five years okay to get to scale and so you're basically at two to the fifth. So that puts you at 32x and then you need at least a multiple advantage of that. So you say 3x. So you're basically at 100x. So one of the things that happens when you hear a lot about these new ideas is they're like, 'Hey, we're going to be 50% better. We're going to be two times as better.' But what they are, they're like, 'Once we have this design, it's going to be even 10 times better than what exists today.' But what exists today isn't the target. Because if you're looking at the companies that are out there right now, these big companies with a lot of resources are actually doing great work. Everyone's Nvidia is advancing the ball materially every year like very significantly.
Nvidia and Google and Trainium, they're moving the ball. And so you got to aim at 100, 500, 1000 times better if you're going to arrive.
Yeah. If you're going to intersect the market ahead of them.
And so that just like when you realize that you're like, 'Oh, wait. It can't be small changes in something. There has to be a very big underlying architectural change, a big thought like there has to be something materially different whether it's wafer scale, whatever it is like those things matter. You can't get there with a collection of modest improvements.'
Right. Your biggest competitor buys silicon for less than you. They buy manufacturing capacity for less than you, right? They probably pay less for their EDA tools. And so you can't go at them, you know, you can't run at Goliath straight on. I mean, you got to think about how you can deliver something profoundly different.
What were the like along the way after some of these like early moments? I know that the company had like, you know, some near-death experiences or whatever. Like what were the hard hurdles?
I had some near-death experiences pain in my chest when you can't build the thing you're supposed to build. Yeah.
Like what are those periods like how would you frame what those periods are where you're like we just need I don't know is it a certain amount of time and a certain amount of capital and it's just going to be a valley of death? Is it that you just don't know whether you're going to make a technical breakthrough? Like what are the things that lead to those?
I think in our space we were always confident if we could make it we'd sell it right that there are sort of two axes in our life there's can you make it and the other axis is can you sell it right and we always believed that if we could make it there would be huge demand for it. I think nobody had ever done wafer scale there was a whole sort of collection of people who were saying it could never work and we were unsure too. I mean we believed we could do it but there was no evidence and there was a period of time about 18 months and we couldn't make it and we're spending about 8 million a month. You got board meetings every six weeks and all you've got to say is still can't make it right that's the board meeting. I mean what else we going to talk about? I mean, still can't make it again and again and again and I give a huge... Right. Right. And they're like, 'Should I get in? Can I do anything to help?' That's right. They're like, 'Can we help?' Like, 'No, there's nothing that can be done. Nothing to be done.' But we believed because we had ideas still. We hadn't run out of ideas. And each time we built it and it failed, we'd go through sort of good engineering practice. We'd do a full failure analysis. We'd understand it. We would and we wouldn't fail that way again.
All new mistakes. So we sort of had that was sort of our mantra. Only new mistakes, right? Only new failures. And over time we could see progress. I mean, in the beginning, we were shattering wafers, right, in seconds, and then it took minutes, and then we had one run for an hour, and then we shattered some in minutes again, and then we built back up to the point. We had a day in sort of July of 2019 where we were running and temperature was flat and we just stood there in a tiny little office that had been converted into a lab and we drilled a hole in the wall to suck the air out and stared at a server which is about as exciting as looking at Paint Drive. Said, 'Holy crap, we've solved this problem that nobody in 75 years of compute had ever solved.' And that was one of the great minutes of my life.
That's cool. When you look forward like if you so you know there's the wafer scale and then everything around that the packaging and cooling and powering it all these things. But you know when you look at the R&D envelope that you have going forward over the next five years, right? I know there's like things you're excited about, but like where do you take this like giant thing? Like what could be the next wafer scale innovation?
I think if you simplify what we build down to its most fundamental elements, a computer is built of three things. We do calculations, we store the results, and then we move the results to where they're useful. So we build a core that does the calculation. We use memory and we move data to and from memory where we store the results and then we have IO and that's how we ship the results to somewhere where it's useful. I think if you're in the computer business like we are for AI, you better be working on all three, you better be thinking about how to make your cores faster, how to make them tuned for AI, but general enough to withstand the innovation happening in AI. You better be thinking about memory and both capacity, how much you can store and how fast you can get data on and off it. And then you got to be thinking about how you get the results off your chip and somewhere where they're useful and that's your IO. And so we're working on all three of those. And we have programs with the US government already, big programs where we're thinking about how to stack memory, right? How to put HBM onto an SRAM based wafer. And what this does is it enables HBM to behave like SRAM which is exactly what everybody wants. You get the capacity of HBM and the speed of SRAM. We're working on optical wafer stacking. So you put an optical switch onto a wafer and this would change the world. You know Jack Dongarra said and he's sort of one of the pioneers in big computing. He said, 'We've been better at making flops and moving flops,' right? And that's the IO part. And so thinking about how to solve that problem by bringing optical switching smack up against compute is something that we also have large government contracts for and we're enormously excited about. So, you know, you got to get faster. You got to find ways to store more and get to memory faster. And you got to find ways to move those results at 10, 100, a thousand times faster to where they're useful.
I want to go to a bit broader in the supply chain. And you know, I think it's kind of like well understood right now that AI is very supply constrained, but I like probably many other people, I wouldn't say I have like a perfect understanding of the supply chain. And one of my favorite things doing on this podcast is you get a, you know, successful person like you and because there's cameras on, I get to ask you a really simple question and I get to, you know, be humored with it. So could you explain sort of in a somewhat simple way how do we go from like sand to a ChatGPT answer?
Dude, it's just TSMC. That's it. There's a black box. You call that TSMC by another black box called ASML. And out the other end comes chips.
We're supply constrained. But I don't know what the... Can you teach me what the supply how it works?
Yeah. I think the first thing to think about is that a fab and especially a fab that builds at cutting edge geometries is a modern pyramid. It's one of the greatest things humans make.
It is a collection of machines that take a chunk of silicon and use a photolithographic process in which they etch transistors into that piece of silicon such that when you deliver power to it they do calculations and it is a collection of I mean this is a factory it's just a reasonable way a real factory cost 40 or 50 billion to make it has a four or five year lifetime.
When Samsung was building a factory in Texas, they began building a power plant. The power plant was used to make concrete. They ran concrete trucks, hundreds of concrete trucks, 7 by 24 for years to pour enough concrete to build the foundation on which to put the fab, right? These are unbelievably complicated things.
So, ASML makes the machines. ASML makes the machine that does the photolithography. What's that machine like?
It's the size of... each machine is what is it 50 or 60 feet long and 20 ft high.
Costs what half a billion?
Yeah, they're expensive. What's interesting is that they sell the same machines to different fabs and fabs use them in different ways and are able to do different things with TSMC able to achieve things that others can't.
And can anyone else make these machines is it really just ASML that can make these machines?
Right now, it's only ASML in the world.
Yeah. I mean there are very few things that this is a true monopoly not born of you know what De Beers did which was try and control supply. They have technology that others haven't been able to replicate.
Are we supply constrained on lithography machines?
Probably not. I think the challenge in our business is that demand moves extremely quickly. And if you've looked at sort of a great company like Nvidia's demand, it's exponential. But you can't build fabs that fast. If a fab takes 5 years to build, right, and you see demand increasing, there's no way anybody would have expected demand to shoot up the way it has for chips, they're always behind.
And they're investing huge capital blocks in the construction of these facilities and demand is racing past the ability to make these huge 5-year bets and we have the same problem right now with data centers, right? AI is moving at the speed of software and data centers are moving at the speed of real estate and that's why we're behind. That's just it.
Good articulation is good. Okay, so TSMC buys the lithography machine, right? So TSMC buys lithography machines. They have decades of experience. They wrap it together. They take an ingot which is a tube of silicon. It's sliced into discs we call wafers. The wafers are sent through the process. The process etches through a lithographic process transistors into the silicon. It moves through the process. They're cut. we call it diced or in our case they're not cut. Out the other end comes chips. And what's amazing about this is you take sort of some of the smartest people you've ever met and you put them on a project for two years and you send the results to TSMC and they run it through a factory that cost 40 or 50 billion to make and took years to make and out the other end comes a $22 chip. It blows your mind when you think about it. The each one of these took sort of the smartest people in their field.
It's the most impressive thing humans have ever done, right? It's among the most impressive things.
That's right. Right. Exactly. Right. And you go down the street here in the Mission District and you pay $17 for a burrito. That's right. Right. And so, and once you have the chip, you're done. Right. Then you're good.
No. Once you have the chip, you've solved about a fourth your problem.
Okay. But still stay TSMC.
Yes, they operate the they receive a design, they operate the machine, they give you the chip.
But that itself seems like not replicable. Like we don't have fabs like this in the US.
We have them run by TSMC and by Samsung.
So what... and Global Foundries have got one too. And how much of the bottleneck is at the TSMC level?
Okay. Why huge amount is that? Because of the speed of building fabs.
It's because the demand has outpaced the ability to build fabs. It's not like an apartment building where there are hundreds of builders you could choose to build your apartment building. The actual making of the fab is skills that only TSMC has. And so and even there they can't do 12 at once, right? The skills to make these things are so rare and in so few hands that you can't just knock them out. It's not a cookie cutter and that's why we don't have enough of them right now. And the fact in the US sort of three decades of bad policy pushed the fabs away. When the fabs left, the tool vendors, the collection of vendors who provided services to them, they all left. The next step in the process called packaging, those are companies like Amkor and ASE. They left and we just punted a strategic industry and we got to do better. That was that's not smart.
Yeah. So, what do you think should happen there? Like like
Well I think we should sit down with Global Foundries and TSMC and Samsung and have a 20-year period where we waive all local ordinances to allow them to build fabs. I think this is about building US domestic fab capacity.
I think I mean look when a big ship tried to parallel park in the US canal right we were delayed in chips and we couldn't buy washing machines.
Right. I mean if we lost our chip capacity it would be catastrophic for our industry. Not just for our industry but for the country for the economy. Eric, how did you as a board member when you were like learning this stuff as you went, you still obviously, you know, I don't think Andrew's been lying to me. I think you've managed to be very helpful. Like what?
Eric was extremely helpful.
So like how did you approach this?
I have a very small circle of competence. It is not any of the stuff that he talked about. If you asked me to explain what packaging is right now, I could not do it. I certainly could not do it in an adequate way. And Andrew explains things to me all the time. And you know so I think it's important just to be like hey this is what you can do and this is what you can't do you know and there are periods of time in any company particularly if you're doing technological innovation where there's just like you just have to let the engineers engineer and the scientists do their thing and stay out of the way and keep them financed maybe keeping him financed was very important I guess for
I think there are a couple things that made Eric a good board member And I think that are foundational in being a good member, right? You don't know about everything and share and make us better in those domains where you're a real expert and don't talk about those other domains, right? That sometimes there a lot of words and we were lucky. We had a really good board and everybody knew what they were good at and they helped us in those domains in which they had real expertise, right? Right. I mean, one of the advantages of the venture world is you can see across an industry, right? When we're deep in it, we're going deep. They can see wide, right? But they weren't efforts by the board to try and solve technical problems. They didn't have that that wasn't their expertise. Talking about how we might finance the company, thinking about all sorts of other things. They were enormously helpful and they were patient and I think they asked thoughtful questions and our board meetings made us better.
It's like what should a board do in a hardware company in general when you know you expect that there's going to be much less to contribute on the product? Is it financing and recruiting? And
I think the question when you've got a long hard project, right? And this is the opposite of SaaS, right? You raise money and then you spend two or three years to build one before you have any real idea what the customer is going to say. You go and you talk to customers and they say, 'Yeah, yeah, that sounds great.' Because who's not going to say, 'Yeah, yeah, that sounds great.' Right? And so the product management is unbelievably difficult. You survey customers and because it's no cost and it's easy to say, 'Yeah, yeah, this is great.' Nobody wants to sort of put their foot on the throat of somebody else's idea, right? They say, 'Yeah, yeah, that's great.' It takes you three years to two and a half years to get to the point where you can bring it to a customer. I think what you can do as investors is understand that trajectory. Understand that your first chip is very rarely a good one and it's a second or third one. I mean, even really strong teams like Google's TPU team, the fourth one was good. The first two were... the fourth or fifth were really good parts. It takes years. You have to know that going in. You have to know it's a long game, right? Ask questions related to the long game, right? Are you hiring the right people? Right? Are you thinking about this in the... is it have you made the right decision between specialization and flexibility, right? These are questions that can really help sharpen our thinking. But you know whether to use this one technique or this other technique in design whether you use one tool vendor or the other you got to let the team pick that.
Did it feel really different for you versus like an infrastructure role?
Yeah totally. It's just really different. I mean your time to revenue is so much longer. The revenue comes in like giant chunks. The initial customers you know we had US government customers then we had you know huge sovereign cloud and G42 you know then you have now today obviously OpenAI and so you have these like things so your customer concentration is more just like every dynamic is different the investment in go to market so much less so I think it feels really really different the things that are the same by the way are You need really good people from a whole bunch of domains coming together who stay motivated over a really long period of time. That part's actually the same and then the market is very dynamic like I mean think about it pre-transformer you started the company pre-transformer
Right at that time TensorFlow was dominant so the first...
Caffe was dominant ResNet was still a thing.
The these were very small very simple networks compared to where we are today. Yeah the actual ML the AI looked nothing like it does today.
