Andrew Feldman, CEO of Cerebras, his fifth startup. He went to Stanford, played soccer there. He met his wife over Argentinian tango. He created a 46,000 mm, 4 trillion transistors. Do you have it with you? Okay. This is a million dollars. If you want a quick, you know, cash infusion, sell this on eBay. Okay.
All right. Fastest in the world in inference. Steve Jobs insisted the Mac boot up 10 seconds faster. This is that solution for AI, claimed it could save 100 human lifetimes every year. Speed matters at Cerebras. These guys are fast. The chip industry is 70 years old.
Hold on. Wouldn't it be great if he introduced me at Thanksgiving? I mean, my brothers, the doctors, would have nothing to say at the table. That would be phenomenal.
Yeah. Yeah. Right. The chip industry is 70 years old and Cerebras is taking the hardest path. And, you know, like an Indiana Jones movie, they're winning and they're getting there. And this panel is partly about hardware and what we need to know. And it's really fitting that Raza is here. Hosted the speaker VIP reception. Who was there last night? Okay. Who has house envy? You can't buy it. He's not selling. But it is beloved in the semiconductor industry. And it's really fitting that he could be here because he knows where it's come from. He built the foundation to where it is today. And as we talk about where we're going, this is a great connection. All right. Dave is probably the cleverest, most resourceful, most tenacious entrepreneur I know. When he was at MIT, at midnight he would get computer time to do AI stuff. He had a connection with Minsky's lab. He's my business partner by day. And I asked him to be part of this because he is using AI every day to kind of figure out how to solve some of the trillion-dollar opportunities around speed. And it's just very fitting to have this panel. All right. So, we have three people on this stage who have built the chips, funded the companies, and taken on some of the biggest players in the industry. Dave just sold a company for 100 million. He wants to plow that money into the chip industry. AI conversations tend to be about models. Today, we're going to talk about going underneath them to the machine, actually running AI, and whether we're building the right one. Every conversation about AI right now is about the model. So now we are going to talk about the thing you know that is the foundation. The models get all the headlines. The hardware gets all the problems. Let's start there. Okay. Andrew, can you just stand up and tell us what your life has been like? You took a company public. You're a multi-billionaire right now, on paper. You know, what is your world right now? And why is Jensen calling you? A year and a half ago, I interviewed him at his big supernova event when he was like a guy, you know, people knew him. He had that million-dollar chip he would walk around, but you weren't taken as seriously as you totally are now. What has changed and what's your plan?
I think for you entrepreneurs out there, when a little success comes your way, you're the same bozo as before there was success, right? And that's the truth. I think we are the same team that started this 10 years ago. The things we like are the same. I think we went public in May, one of the largest semiconductor IPOs in history. All the problems that we had before the IPO are there after the IPO. Right? The IPO solves nothing. I think it gives you a little more money, but it can't help you solve a technical problem. It can't figure out a solution to droop across a printed circuit board. It can't design chips. And so, we embarked 10 years ago on a big journey. And we don't do this for money. We don't do this to sit and talk before you. We do this because we really don't know how to not build stuff, right? Even my co-founders and the thousand-plus of us now at Cerebras, when we get a little bit of free time, we go home and we build stuff at home. Sometimes it's with software, sometimes it's cabinet, sometimes it's metalwork, sometimes it's... but that's who we are and those are the things we love doing. And so when in 2015 we saw the rise of AI on the horizon, we thought there was something we could do. And as a chip architect knows, what a chip architect lives for is a new workload. Right? In static state, at equilibrium, nobody changes share in the chip world. It's when new workloads arise that huge share changes, right? When a new workload called Ethernet switching arose in the late 90s, all the incumbents died. Every one of them, Nortel Networks, 3Com, you go through this list of companies, all dead. When a new workload arose in the mid-2000s, right? All the existing players, AMD, Intel, they had fabs, they had all the architects, they got zero share of compute for phones. And so when we looked out, what we saw was, holy cow, there's a new workload coming. And we believed we could build a company that would accelerate this workload. And we believed there would be enough of this workload to make it worthwhile building a company. And in every one of our forecasts, for the first time in my career, we were wrong on... we estimated it was smaller than it was going to be. It's my fifth startup. Every single estimate we've ever had before this company, we guessed that the market was going to be bigger than it actually was. In this case, we guessed the market was going to be smaller than it is. And what's happened over the last 18 months is this explosion. Remember, we make AI with training, but we use it with inference. And as AI has gotten smart enough to be useful, what we're watching is a sea change, a wave of it being used. And when it's used, speed matters. And our premise 10 years ago, probably seven years too early, was that speed would matter, that we could build a machine that was 10, 20, 30, 50 times faster than a GPU. And so that's a very long way of saying what's the last 18 months been like? We have a 25 billion backlog, right? We will increase manufacturing by 8x this year and 8x next year. We entered the year with several hundred megawatts under management and we'll end this year with contracts for gigawatts. It is just an extraordinary time. And I tell people that the right way to get the timing right in a startup is to be wrong for a long time and not be dead. And that's what happened to us. I mean, we invented this. We solved this problem. Maybe this is longer than the answer.
We solved a problem that everybody in the industry said couldn't be solved. That, with the exception maybe of AI and Andy Bechtolsheim and maybe Lip-Bu, everybody said you're out of your mind. And in 2020 we solved it. We delivered it to market and absolutely nobody cared. We had solved a 75-year-old problem in the computer industry. We had achieved what was one of the holy grails and absolutely nobody cared because fast AI didn't make any sense when the AI was dumb. And now the AI is smart and getting faster, getting smarter at an extraordinary rate and everybody wants answers more quickly. And the final thing is, it's not easy being married, it turns out, in a great market. My wife says that, you know, when nobody wanted to buy my stuff, I was stressed at 11. And now everybody wants to buy our stuff and I'm stressed at 11. And I told her that in 33 years of marriage, maybe you should learn something about my stress level. But that's what it's like. It is an extraordinary time. I think for those of you who are from the valley, right now the level of intensity and the excitement around AI in the valley is unlike anything we've ever seen. Yesterday, my first meeting was 7 a.m. at a coffee shop in Menlo Park, and there were about 20 other people doing AI meetings at 7 a.m. on a Sunday. And that's what's going on right now. And it's really a pleasure to be a part of.
The models keep getting smarter. Is the hardware keeping up or are we running today's AI on yesterday's machine? Atik and Dave.
Well, we're definitely running on yesterday's GPUs that were designed for graphics and video games. And you know, it's funny. When you talk to the people who are now, you know, multi-billionaires, decillionaires, none of them saw it coming. They were busy building the best conceivable GPUs for video games. And it was the AI researchers in the basements that said, 'Hey, our algorithms will run better on that chip.' And I think the audience for that was pretty limited at that point in time. They're like, 'Yeah, we don't care about you.' Boy, did that invert in a hurry. But, you know, one of the byproducts of the way technology evolved, you know, I think Andrew, you said your IPO was the biggest semiconductor IPO of all time, but I think it was actually the first or second biggest IPO of all time, period.
It was the biggest before SpaceX came up.
Before. Yeah. And it was funny because it's the top of the pyramid, you know, being the coolest kid in town has a very short shelf life these days. But you at least you had it. You had the Chrysler Building.
But you know, it's funny. I was talking to Andrew about two things. First of all, the humility with that level of success, having the biggest IPO of all time. The humility in the moment is just mind-boggling to me. It's just so cool. But the other thing he said was if you look for people who are aware of AI algorithms and transistors at the same time in their head, there's like one in a million engineers that spans that top to bottom. And why did this happen? It happened because, you know, the biggest IPO before that was Meta. Meta is social media. Before that was Google. And it's all software, software, Python, Python. You know, no one's thinking about chips. No one's thinking about fab. The entire fab industry moved to Asia. We just completely lost the core and all of a sudden like it has flipped and now all of a sudden... but we just don't have the people who can think from, hey, this transformer change will require this transistor to operate or load in this way. And boy, if you develop that skill set, you are in huge demand at Cerebras and everywhere.
Well, basically that is true today because, you know, AI came on so fast. But the entire history of process technologies and semiconductors has been on the basis of analyzing the workload and trying to build machines that will execute the workload, all the way back to even Alan Turing. I mean, we stand on his shoulders. I mean, my entire career, my life, association even with Andrew, all comes from that person starting and thinking about how to build a machine that would execute the problem that he was facing. To go from there to processes in the early days. In a way I see analogies even at that time that the problem always remained on exactly how the processing has to be done very fast on the workload that is available and how you feed the processor from memory. How do you enable that continuum of not having a bottleneck either at the memory stage or at the processing stage that came through multiple generations. And you know, when I was being introduced, you talked about being there early. Yes, I have seen Shockley sitting in his office. Yes, I've competed with the, you know, the entire Gordon Moore, you know, the guy who has a law associated with his name. I've competed and beaten his company when they were a juggernaut that was so powerful. None of you may be old enough to remember when Intel was so powerful that even Microsoft would not cross the line with them. That was the time with a handful of people. We started off addressing the same problem. We introduced techniques of working on the workload that was available at that time and we introduced things like out-of-order execution, branch prediction, prefetching. And then later on we went on to see that there was again a bottleneck being created as you had to get more memory fetches and then hardware assistance in scheduling of instructions. All that happened and that brings us to this time when suddenly the workload has started transforming at an extremely rapid pace. So I don't take everything away from Jensen that he basically was building graphics chips. I remember I was on the advisory council of Stanford School of Engineering and Jensen already had become famous because of his graphics chips and I met him and I said, Jensen, what are you thinking about right now? And he said, I'm trying to get GPUs into data centers to be part of the compute workload execution. And I said, but your GPUs are unusable. I've been trying to get my company to use them. At that time there was a software company I was... But they are so hard to use. And that's when he said, I will address that problem. I will. And he developed CUDA, the ability to distribute the workload onto his GPUs. He gave his machines to Stanford for them to use in the mathematical engineering labs that they had over there. Fast forward to what Andrew was talking about. Now I have a long association with Andrew because if you work in this field you know each other, you know who's doing what. And this is with Cerebras. When I first basically was described Cerebras, and you know, some of his key people have worked for me, and when he described it to me, at that time AI was not on the horizon. It was high-performance computing that this was built for. But the problem that he was solving was the right one. The problem he was solving was taking care of basically the flow of the workload from memory, from processor to processor, into the ability to then process. So he was creating a platform which had architectural innovation and physical innovation where the memory was very close to the processor and the other processors were also very close to the processor. What he showed as a single chip basically is translating an architecture onto a platform. As we go forward, the problem is going to get worse and worse and there will be a time then he will have to do something more than what he is doing. But every new innovation that is coming is dealing with the problem of a bottleneck occurring now with AI, particularly as the AI workload itself is transforming. It used to be training, then it moves to consumer-dominated inference. Now it is going to enterprise inference and then within enterprise inference the transformation is occurring again. And the way it is occurring is that AI in the meantime has opened up the context window. Opening up the context window is basically an invitation for unstructured, huge amount of everything to be sucked into it. When that happens, your inference is not a single-stage inference like you have with a consumer or an initial enterprise environment. It is a multi-stage inference and it has fan-outs. And now it is carrying forward any ambiguity in the assignment of weights in the tokens is going to also backfire on you, even the slightest change. So now what happens is that with this enormous transformation in the workload, every step has to retain the reference to the context of the answer of that step as is going to be used by the next step. This is going to be incredibly memory-intensive. We haven't seen the end of it and we will have to have new innovations that address this, even a step, but in the same direction as what Cerebras was trying to do. And if you take a look at some of the new companies that are coming on the horizon, they're including things like in-memory compute. In-memory compute was originally only a dream to be used at the very, very edge. Now they're talking about using it for data centers, but the idea is the same. How do you basically enable the interaction between memory and compute to be as low a latency as possible, as efficiently translatable as possible. It can't be done alone by hardware. It will need scheduling also of the workload itself.
All right. Thank you. I have a bunch of questions for you, Andrew. We want you to answer them real quick and then Atik and Dave, if you could weigh in, is he right, is he wrong, or do you know something that Andrew doesn't know? Okay, so what is the real bottleneck? If you could remove only one constraint on AI tomorrow, compute, memory, networking, or electricity, which one would you remove? And will your answer be the same three years from now?
I think right now all of us are constrained by data center availability. The entire industry, I think we're trying to move at this enormous speed, sort of the speed of software, and data centers move like real estate. And that's holding us all back.
Right. Three years from now, are you evading that question?
I'm not evading. I'm ducking and covering. I think in three years it will be fab capacity.
Yeah, okay. I'm going to go to a hundred billion dollar question, but before I do, is he right? Is he wrong? Or do you know something he doesn't know that could be helpful?
I think he knows a lot. I don't know.
He's absolutely right. But the main thing is there are things that we can fix, people sitting on this table, or we can influence or be involved in fixing, and those that we can't. Energy and building data centers physically, we can't fix. That has to be fixed elsewhere. What goes inside those data centers, that we can influence and fix.
Yeah. Okay, watch this. Dave's going to say something profound and shocking.
Well, what we do right now at AI inference is utterly idiotic. The weights do not need to move. And yet, we use massive amounts of energy in HBM, the most expensive chips in the world, to move the weights out of HBM memory into a GPU, use them for a picosecond, and then dump them. It's utterly idiotic. I know Andrew knows this because we've met and talked at his office about it. There are solutions that are a thousand to a million times more efficient. The only reason those don't exist already is because the supply chain doesn't move quickly enough and the software people are innovating at such an incredible rate that nobody wants to hardwire something and find out it was wrong. But when we get the supply chain figured out, which will be easily within three years, we'll be running most likely on the upper end of a thousand to a million times more efficiently by not moving the weights.
All right, I have a lot of questions. Next question. The hundred billion dollar question. We're committing hundreds of billions of dollars to AI infrastructure. Are we building the foundation of the next economy? Are we overbuilding the biggest technology bubble in history? What is this, pets.com or whatever? That's a 2000 reference for those millennials or Gen Z. What would have to be true for all this capacity to earn an economic return?
You know, no, he's... I'm going to him. You get to say if he's right or wrong. I just want to make a comment. You know, I am old enough to remember there was a company called Digital Equipment.
General Duryea put 70,000 into it. It was head-to-head with IBM.
Why? Because the answer that Ken Olsen gave is, I cannot imagine so many people requiring little computers.
Yeah. Ken Olsen worked on Project Whirlwind, which was at MIT that spun out Silicon Valley. You don't get Silicon Valley without Project Whirlwind. And his co-founder is an MIT guy, by the way. All right. So, what's the answer to that hundred billion dollar question?
I don't think there's any doubt we're building the foundation for a fundamentally different economy. I think there will be bumps along the way. I think if you take a 10 or 15-year view of pets.com, it turns out that pet food online is really good business.
Chewy. And a huge portion of what Amazon does and a large portion of the total pet food business is now sold online. I think there will be bumps along the way, right? There will be bad ideas that we adopt for a little bit. There'll be sort of cul-de-sacs in innovation, cul-de-sacs that we'll have to back out of and keep going. But I think the following, and I said this at Atik's house. I think there's a chance that someone born in 20 years never knows anyone who dies of cancer and never knows anybody who's hit by a drunk driver. Okay? And the number one killer of healthy adults is car accidents. The number one killer of everybody is cancer. And so I think we will solve big and interesting problems in the next 20, 25 years and that will change what it is to grow up and to live in our society.
So my next set of questions is about AI when it's 10 times faster. But Dave, did you want to weigh in? Right, wrong, or you know something that he doesn't?
Well, so it drives me nuts when people compare what's going on right now to pets.com because the internet, you know, I was really fortunate to graduate college right when the internet was exploding and taking off and it worked out incredibly lucky for me. But everyone's like, 'Yeah, but then it overheated and collapsed.' Yeah. But then Google was born right in the middle of the collapse and Google just... and look how... look where Google is now. Look where we are now. Yeah. I mean, it's like, was the internet a bubble? No. The internet is the foundation of basically everything we do now. It was never a bubble. It's just people panicked and bailed out and gave up on it. That always happens in America. It will inevitably happen in waves of AI. AI will still, 20 years from today, be the biggest thing that's happened in human history by far. I mean, Andrew's characterization is exactly right. And so, yeah, America always overheats. Yeah, the capital always floods into the wrong little nooks and crannies and it collapses and people say, 'See, I was right all along.' No, AI is here to stay. It's going to be massive.
All right. Speed versus intelligence. Have we spent too much time asking how intelligent a model is and not enough time asking how fast it can think, as someone who is at the forefront of speed?
So, I'm... now that I'm a public market CEO, what I see around me is everybody selling their own book all the time. Everybody telling me what I should do because it's in their interest. As somebody who builds fast hardware, we're not delusional that dumb fast is not helpful to anybody, right? But what's happened is the models have gotten so smart and are on a trajectory to continue that speed has jumped to the fore. The open source models are really smart, the closed source models are really smart. The small models are really smart. And in that environment, what you want is an answer quickly, right? You know what I like to say is if your 12-year-old has been naughty, don't take away their phone. Set it to dial-up rate, right? That's a punishment, right? The inability to get to where you want to go quickly is a punishment. And that shows you how important speed is. And so that's my answer.
I think it's one of the most important questions of this next coming year because, you know, we're seeing with these smaller Chinese models like GLM equal or very close to equal frontier benchmarks, but the parameter count is down significantly, a factor of four or more. If you can get the parameter count down on a model and have it be just as good at a task and then you can fit it all onto a single Cerebras or a set of 10 Cerebras wafers, you pick up a factor of 10 to 100 in throughput. So the question is, if that works, you got to keep in mind that what we use right now for AI surprised everybody and it was trained on every word ever written by humanity, including every Reddit post, every Trump tweet, every Taylor Swift concert. It's all in there. And when you ask it a question, it's like, yeah, I know all about Taylor Swift's eras. Like, well, I don't need that knowledge to do this task where I'm trying to write code. I'm trying to design a chip. I'm trying to design a rocket ship. If I can get the parameter count down yet be just as smart within a domain, then it all fits in much smaller memory and it's blazing fast. So the possibility of that is incredible. Sorry, I...
I want to address what Dave said because this is an extremely important area as AI transitions from just data centers to the edge and to edge devices like robotics and even to basically handheld devices which can operate without making cloud calls because here the scale that you provide has to be much less. The accuracy may have to be more. The weight ambiguity may have to go down significantly for it to match the underlying hardware's ability to execute.
So all right, my next question is, you've said speed is productivity. Give us an example... 150,000 people who are dying may not die because of that. But let's get specific. Give us an example of what becomes possible at 10x today's inference speed and what simply isn't possible now. You know, maybe you're too busy on the speed and not thinking about this, but people must be saying, oh, with what you've created, I could do this. Imagine the same intelligence but responses are 10 times faster and 10 times cheaper. What new applications suddenly become possible? I mean, this is when we have a symbiotic relationship with technology that, you know, we just haven't thought of. Black and white to bring us into color.
Okay. So, let's think about what speed does across the landscape of technology, right? Netflix began doing what? Shipping DVDs via mail in envelopes.
Right. And then the internet got fast and they didn't become better at shipping DVDs in envelopes. They became a movie studio, right? They were able to bypass traditional theaters and for very little money they were able to get right into millions and millions of homes and they became something fundamentally different. And if you had to wait four hours for your movie to download, that would never happen. That time and time again in our industry, speed has unlocked new ideas and new markets. Right? Coding agents aren't interesting if they're slow. They're also not interesting if they're dumb, but once they're good enough, they're not interesting if they're slow. Personal agents, some of the fastest growing applications right now are Instinct and Muse and this collection of personalized agents. If you have to wait eight minutes for it to order a burrito for you, right? One, you could almost drive there and get it. Or two, you can go to the computer, your phone, and you can order it yourself, right? The ability to execute and do things with AI is fundamental. And if you think of something like the guys who are doing law or accounting, right, their whole business model is to bill every six minutes, right? Maybe it would be less offensive if they got 30 times as much done in that six minutes, right? And so when we look out into the landscape...
Calling lawyers offensive?
I think it's... Yeah. I think a billing model of every time you call me is offensive, or I call you. Yeah, I think that's not good.
Well, sometimes they're really, really valuable, but not in six-minute blocks. I think that...
The making of the stuff we do, and even if you look out into your daily life at work, you know how much time is spent in an enterprise taking things out of Workday and organizing it so it can go into ADP, right? Just stupid manipulation of stuff from one to another, or going from Salesforce to your accounting tools. All this goes away quickly, and it changes what our organizations can be and what they are, and how we're going to be organized. And all of those are only made possible if you don't have to wait forever to get it done.
What do you guys think? You know, he's in the eye of the storm, you know, building this, but you know, you guys are futurists and following trends.
I think speed is directly translated into less human effort to get a far better result. And a really simple example of that is the vision transformer. Like if you wrote code six months ago and you're trying to put a GUI on it so you know what the hell is going on, it just flat out gave you garbage. And the solution turned out to show the model what it built visually and have it learn to understand what you like. And the solution there is called a vision transformer. You just take an image and you cut it into 16 by 16 pixel little chunks and you pretend each chunk is a word and you throw it right back into the same LLM that read every Trump tweet and you say, 'Pretend this is language and just freaking tell me what's going on in this image.' Nobody thought that would work, but with enough speed, it just flat out works. So now, if you're building an interface on top of whatever you're building, it actually looks at its own interface and says, 'Yeah, I built garbage. Let me work on it again.' Okay, now this is something you're going to like. And then when you're talking to the AI, you're like, 'Those buttons are messed up.' You literally prompt it messed up. And it says, 'You're right. I'll fix them.' Could not do that six months ago. That unlock is pure speed.
But there are consequences, and I'm not a fear-monger, but when basically these capabilities are enabling Navier-Stokes equations to be solved where Terence Tao, Field Medalist, basically is worried about what he's going to be doing if these tools are going to be solving problems. It is not unfounded that we should take a pause and think about the socioeconomic impact that this is going to have.
So if we were designing the AI computer from scratch today, and I don't, maybe this is futile because you know there's so much we're building on, not inheriting decades of computer architecture, would it look anything like what we're building now? And if we were doing that, what disappears first?
I mean, if you're building it from scratch for the future, all of the modules would be photonic for sure, and then all of the processing between the modules would be normal transistor math.
