Okie dokie. We're in the remote studio celebrating a big big raise with Andrew Feldman of Cerebras. Welcome.
Well, thank you for having me. It's such an honor to finally have you here. We've been sort of poking around this sort of Cerebras ecosystem for a while. I run the AI engineer conference. I run lat space. I'm friends with your entire marketing team of course and also more recently I joined Cognition which is a big customer and friend of Cerebras. Here, I mean, let's get the big news right at the top. What are you announcing today for your fund raise?
So, we announced a $1.1 billion fund raise that we had completed. It was done at an $8.1 billion post money valuation and it was led by Fidelity and a treaties management. So, we're announcing that we raised the most money on the highest valuation with the best investors in the category. And so, it's a good day. I think when you raise money, a lot of excitement, and then you sort of put that down and move ahead to keep building cool things and executing. So, that's today. We're doing our high fives, and tomorrow we go back to work and keep building cool things.
Yeah. I like I love this the spirit of the thing. So, help me contextualize, you know, like billions of dollars is such a impossibly huge number for people to understand. How does a round like this come together? How do you contextualize this in the scale of what Cerebras has done?
Sure. I think we were founded nine and a half years ago and I think you know in early 2016 I don't think we could have imagined the AI market behaving like it is today. And I think we were in 2016 we met with Sam Altman and Ilya Sutskever at OpenAI and they were an idea and we were PowerPoint right that's amazing and what AI was doing was identifying cats in pictures and I think they ended up investing in us both of them and many of their colleagues in our early rounds but I think what happens is that you begin building things your vision is how can you make your customers life better? How can you delight them with their experience or delight their customers, right? How can your infrastructure play a role? And we saw early on that AI would be a monstrous market. We probably guessed wrong that it we probably didn't think it would be as big as it is, but we thought that if you did extraordinary engineering, if you tried to do something that nobody's ever done before, you could do something really special. And from there, you know, we shipped our first product in 2020. We delivered the next generation a couple years later. We delivered the third generation a little less than a year ago. Customers around the world have been deploying and benefiting from our performance. Really focused on performance both for training and for inference. You think 20 times faster than Nvidia B200 GPUs. And it's been an amazing run. And so the way round comes together is you decide you're going to go out, you engage at this stage with investors who do both public and private investing. Like any other sales process, you go and you visit them and you tell them your story. We were massively oversubscribed and we were able to choose sort of the best of the best to be in our consortium.
Yeah. So I'm going to flash up the announcement on screen. Mostly we talk to researchers and engineers more technical podcast on our side. I mean, while we're at this, while we're talking about a fund raise, let's contextualize a bit. Fidelity is a big name. A trades I don't I know less. What is a good investor to you? What makes a good investor versus a not so good investor?
I think it depends. There's no single rule. It depends on the stage. I think in your early stages, you need people who understand the risks and challenges of being an early stage company. And in our series A we had Benchmark who was perhaps the best in the business along with Foundation Capital and Eclipse extraordinary companies. I think in later stages as you get close to IPO you're looking for a very different type of investor. You're looking for an investor who primarily does public markets. You're looking for an investor who is extraordinarily thoughtful, is a leader in their field. And I think that's what we achieved with Fidelity at Treaties, Tiger, Global, Valor. Each of these are leaders. They're looked to as extremely knowledgeable in the category. And you know people take notice when the smartest of the smart money invests behind your idea and your company.
Yeah, that's amazing and congrats on all those. Let's dive in more sort of the technical side obviously because we speak more to the engineering crowd. We talked about 20 times faster. That's obviously not like a strict number, but you see this is I've seen this chart evolve over time like you started out with the llama number and then you sort of branched out into all the others.
That's right. So obviously the growth of open source and the growth of these like really large open source models mostly as well now has really benefited you talked with artificial analysis I'm familiar with them too I also had them for the AI engineer conference was it a surprise that you had like such like higher performance numbers than the competition
So inference performance comes from memory bandwidth and the memory bandwidth is the limiting factor in inference performance. Remember in order to generate a token to generate a word all the weights have to move from memory to compute. If you're constrained right there, your inference is slower. And we have 2,625 times more memory bandwidth than the GPU does. We use a different type of memory. We use SRAM whereas they use a flavor of DRAM called HPM. And so we were built to solve this problem. We were built every ounce of our design and our architecture was designed to provide huge amounts of memory bandwidth. Maybe the way to think about memory bandwidth, imagine you've got your cup of tea there, but imagine a cup that holds Coke. The cup is your memory capacity. And the Coke you pour into the Coca-Cola you pour into the cup is your data. Now, the rate at which you can get Coke into your mouth is a function of the straw, right? It's not a function of the size of the cup. It's a function of the size of the straw. And GPUs have a very narrow straw between compute and memory. We chose a fundamental different architecture. We got rid of the straw altogether. We built a chip the size of a dinner plate so we could put all the memory onto the chip. We could use SRAM. And this said, all right, we've got our Coke in our cup. Get rid of the straw and just put it right on our mouth. And that's the way you get huge amounts of data.
That's exactly right. That's exactly right. Moving. Were you just pretending to drink tea? Did you pour diet coke into a teacup? Is that what you're doing?
No, I didn't. I happen to have a Coke can next to me. But I'm actually sick. So, this is why I'm drinking tea to stay hydrated.
To stay hydrated.
So, we weren't surprised. Every ounce of our design, of our architecture was built to solve this particular problem and enable the type of speed that your guys at Cognition have benefited from, the guys at Mistral in the closed source world benefit from. We serve lechat from Mistral Meta has benefited from IBM benefits from AlphaSense and hundreds of other customers around the world benefited from this extraordinary speed.
Yeah. Fun fact, I used to work at Centio which was an AlphaSense competitor and they merged into AlphaSense. So I'm also a shareholder.
You've been at our customers your whole career. This is awesome. This is awesome. So, I mean I think like you obviously made the right bet on the large chip and you know there's a number of sort of people in the large chip sort of industry. You know, you're not the only ones like what has made you stand out so far? You know, I think like what has been hard for a non-hardware person like myself to understand is, you know, like a lot of people who are sort of competing with Nvidia and bidding on like more on on-chip memory and all that like it's hard to distinguish between what the strategies are and I think over time as well your different generations you are developing more and more of an understanding of what the market really wants. So I just want to get a sense of like how you see this market the large chip market.
Sure. I think that the way to think about this is that there are really two types of memory. There is DRAM and HBM which they can hold a lot but they're slow and that's what Nvidia chose and they chose that primarily because originally that was the right choice for graphics. Lots of capacity but slow. On the other hand, there's SRAM. SRAM is the exact opposite. It can't hold very much but it's blazing fast. And so what we saw was that if we built the largest chip in the 75 year history of the compute industry, if we built a chip that was 56 times larger than the largest GPU, we could stuff it to the gills with this fast memory. All right? We could fill it with the fastest memory and thereby overcome its limitation of not having a lot of capacity and just benefit from its blazing speed. Now if you build SRAM based solutions based on little tiny chips you need to use hundreds or thousands to do work. So if you build a normal sized chip say 800 square millimeters right the size of a post stamp and you want to support a trillion parameter model or 600 billion parameter model you might need 4,000 chips. All right? That's a lot of complexity. That's a lot of wires. That's a mess.
Yeah, you need interconnects as well.
You need interconnects. It limits the things you can do. It makes all sorts of cool AI techniques like speculative decode extremely difficult. Whereas if you have a giant chip, you might only need a handful. And so I think that's one of the ways to differentiate. I think also just go up and look at artificial analysis. Wherever we are, we're the fastest. Not by a little bit, but by a lot. And the reason we like them as a benchmarker is they go to your cloud anonymously every day do a little work publish the results right there's no gaming there's no months of preparation for the test that advantages big companies like MLPerf every day they go and this is exactly what your customers get there's no gap between the benchmark and what your customers actually see.
Yeah, it's like you know MLPerf is kind of like the Olympics like you really prepare for it and it's like a once in a lifetime you know and then like the artificial analysis is more like Apple Maps like estimating traffic for you you know
Right you know it doesn't really matter how fast you are at the dragstrip if every day you go to the grocery store right you can so that's the way we think
So and then little follow up on sort of the inference versus training split. I'm getting a sense that like most of these are just trained on non-Cerebras silicon and then inference on silicon on Cerebras silicon. Do you see that persisting going forward? Are you specializing in inference? Do you want to would it be a goal for you to have more training workloads or do you not care?
We support both training and inference and we deliver solutions on premise and in the cloud. So we are perfectly happy for our customers to use us to train or do inference and we do a great deal of work with large enterprises in training with Mayo Clinic with GlaxoSmithKline with the US military with the department of energy with our customers in the Middle East. We've trained leading models language models in Arabic in Catalan in Hindi. What's happening in addition to the giant sucking sound for fast training is that these models are being used, right? I mean, we make AI with training and we use it with inference and the amount of AI being used right now is extraordinary. It's just off the charts and I think there is certainly an unquenchable demand right now for fast inference.
Yeah, I think that the numbers I had was something internally for Google with the TPU workload, it's something like a 2 to 3 ratio of training to inference. And I wonder I feel like for Cerebras it might be like I don't know 1 to 5 in favor of inference.
I think that's not an unreasonable guess.
Yeah. Okay, cool. Amazing. And then what do researchers tell you they want? Like you know I'm looking at these charts of Quen 3, Llama, Llama 4, OSGPT, and obviously there's more. The GLM just came out today and I know there's a lot of work that's going on between our companies on the open models side. What trends are you seeing and what do you need to support on the silicon side for those trends to happen?
I think while we design silicon, we also build the system and the software layer and many of our customers including Cognition benefit from us via our cloud offering and so it's a sort of a soup to nut solution. The trends are pretty clear I think for sort of AI companies like Cognition like all your competitors like AlphaSense like dozens of others they are trying to replace closed source models with very very fast open source models and they're trying to drive the open-source accuracy dramatically up and they're doing that with fine-tuning they're doing that with all sorts of interesting ML techniques I think the closed source community has come to realize that being slow is not ideal and that there's a very sort of interesting tweet by Paul Graham and he said something like I'd use Google half as much if ChatGPT weren't so slow right and that's very interesting that if you make your customers wait they leave you right they go and use something else and so
They don't necessarily tell you about it too it's just that they're voting with their feet yeah
Yeah they're voting with their feet and like it's slow like maybe like 20% of my work goes over. It's not the full thing. I'm not rage quitting. I'm just choosing more one over the other.
That's exactly right. That's exactly right.
We see at the large enterprise level particularly those who have large data assets a desire to train their own models and to go a little smaller say in the 10 to 30 billion parameter category. I think especially the very large companies enterprises have concerns about some of the data that was used in training some of the open source models and so they want to take the architecture of the open-source model randomize the weights and begin again and do that with data sets that their own legal teams have approved. So we see that in the large enterprise category. I think areas that are of interest are sort of securing agentic work, right? I think when you spin off dozens of agents, the sort of attack surface of the solution expands geometrically and so there's a lot of work being done to think about how one might secure an agentic flow like that. I think still the bulk of the models are in the transformer category for language. We continuing to see innovation in diffusion for images. I think many of the new models are tweaks to some of those core foundation architectures.
Yeah, it's weird how like you know the transformer paradigm has stood strong for quite a long time. I do think like there are some discussions you know I just recorded a podcast with AI21 yesterday where they you know they've been really exploring hybrid models but I feel like at least the paradigm of like deeper models or like you know like more I think that's not really going to go away that they might have some innovation in some part of the attention variant that you might be using but like nothing that you have to particularly take care of at the hardware level.
Yeah, I think you know we designed this architecture before transformers were invented and one of the hardest things in computer architecture and one of the first things you do is you decide what you're not going to be good at. I'm going to build a chip. What am I not going to be good at? We're not going to be good at general purpose compute. We're not going to be good at graphics. We are going to be good at AI work. Second really hard question in computer architecture is what should I be general at and what should I be specialized at? And in 2016 17 as we were setting this architecture we could have said what we want to do is embed 3x3 convolutions into the circuits and that would have made right 3x3
