We just did a first close of a billion dollar fund raise at an 11 billion valuation. I've been in this industry for 32 years, building high performance chips for a long time. I've never seen the interest in semiconductors higher. Now what you're seeing at scale with Anthropic and with OpenAI and with Gemini, we got millions and millions of people using it every day. We released SN40 a couple years ago. It became incredibly popular because instead of a 130, 140 kilowatt rack of Nvidia GPU, we were outperforming them with a 10 kilowatt SM40 rack. You could take a trillion parameter model and run it in a single rack where it would take dozens of racks of other people's equipment to run the same model. We're a $2.5 billion raise in the history of the company and there aren't really that many companies that have raised into the multiple billions. It's all about scaling. It's all about who can get to scale faster.
Rodrigo, welcome to Sourcery.
Well, we're here for context. We're in Paris right now for the Raise Summit. And right now, we're sitting right in front of where the conference is. I don't really actually know what this park is called, but it's next to the Louvre.
I don't know if you know.
You've been here a bunch, right? I've been here, but not sure if I know exactly the name of the property. Right, right in front of the Louvre.
Well, you have some big news. I think this will come out about a week after the news drops, but we'll still make a clip on that. So, what is the big news?
Well, we're super excited. We just did the first close of a billion dollar fund raise at an 11 billion valuation. This is a great show of momentum for the company and a great show of support. The round was led by General Atlantic with a number of incredible investors that came in: Seligman Ventures, Troll Prize, Capital Group. These are all significant American investors that are coming in. That shows that the company's got momentum. We're driving towards the scale and a significant amount of capital infusion to help us do that.
All the energy right now is going into semiconductors. I'm sure this was a very hyped up round in some way or another. Maybe it's been faster than others. What was the process like for you?
I've been in this industry for 32 years, building high performance chips for a long time. I've never seen the interest in semiconductors higher, right? And I think it's a realization that chips are at the center of this transformation. If you look at what AI is doing in the world and the buildouts of the data center, you can't do it without chips that run and run efficiently. And so with SNOVA, we're coming in and providing technology that is able to take it to scale, take it to a level of inference scaling that's just really not that practical to achieve just with traditional GPUs. And so I think the world sees that and the excitement is coming in from some of the top investors in the world.
So where we are at today is inference. Inference has really taken the stage and it's been the next evolution of computing and where everything's going with AI. So for people that don't know SambaNova, can you walk through the products and how you've evolved them for inference?
Yeah, I mean, look, with AI, you've got a sophisticated audience, so they know with AI there was always the training and the inference. There's no point of training a model if you aren't going to inference it, if you're not going to use it, right? And so the example I use with people is, you know, you don't go and invent a search algorithm if you're never going to do search, right? And so saying we're not going to train a model if you aren't going to use it. And now we're in the phase of using these models. We've always used them, we've always inferenced them, but it was still research to train models better and better. And so when we started the company in 2017, we were very focused on how do we actually lower the cost of training, right? And back at the time we were training models for image recognition. Can we tell the difference between dogs and cats? And you know, can we recognize voices? Can we make voices? You know, we're doing all of that research, but really in the end, inference wasn't really a problem yet because the number of people using it were very small, it was people trying to test the model that they trained. Now what you're seeing at scale with Anthropic and with OpenAI and with Gemini, and you're at scale, you got millions and millions of people using it every day. And so now you have the problem that SambaNova was originally focused on, which is around efficiency. How do you actually deploy at scale so the whole planet can use it without burning up the planet, without kind of running out of data center space, without, you know, blowing up your infrastructure cost? Because at scale, the number of chips deployed for inferencing will be orders of magnitude greater than whatever you're doing for training.
And so walk through some of your chips.
Yeah, so we started a company, you know, so we've taped out six chips in the last seven years and we'll tape our seventh one next year. We're really excited about generation five that's shipping later this year. When we started with SN10 and SN20, these are the early chips. They were really focused on training, you know, and they were focused on can we train those models faster with fewer chips? As you know, some of the largest AI labs are deploying thousands of chips just to train one model and they were trained for months and months and months, right? And so we were focused on training. As the world moved along, it became very clear that the bigger challenge, the bigger problem was inferencing, right? That once you were at scale, how do you actually deploy those same models you train on all of these different, you know, data centers for people to use? And now you can say, you know, I need a gigawatt data center somewhere in West Texas. That's one way to do it. But how do you serve all of the countries, all the planets, all the different users that are worldwide? And so you need to think about power, you need to think about data center, you need to think about latency, right? And these are all things that we started taking on. And so by the time we released SN40 a couple years ago, it became incredibly popular because instead of a 130, 140 kilowatt rack of Nvidia GPU, we were outperforming them with a 10 kilowatt SM40 rack, right? In a 10 kilowatt SM40 rack now suddenly, and it was air cooled, you didn't need liquid cooling upgrades. Suddenly, every data center that's around the world that you're using traditional CPU, traditional storage for, you could just roll in a SambaNova rack, air cooled, and you got state-of-the-art inferencing faster than on an Nvidia GPU, right? And so that became a really, really popular way to deploy, especially if you don't want to put hundreds of racks of Nvidia GPUs. We would collapse the footprint because we could take a trillion parameter model and run it in a single rack where it would take dozens of racks of other people's equipment to run the same model. And so that's kind of what SNOVA is really focused and known for, is really driving premium inference, the largest models that are very low cost, very low power, and delivering ultra high performance, which becomes really, really valuable if you're actually starting to deploy these incredible models right at scale.
How has rack composition changed? If you look at kind of for training, one of the challenges that you have is you have to aggregate all of these racks because you need thousands of GPUs together and they have to work in sync, right? As you're training each loop that you're training, you're working in sync. And the problem there was that if any one fails, the whole cluster fails, right? And so you have to put these checkpoints, you have to kind of stop, you know, every so often to make sure that you store the progress you made up to that point in case the next cycle fails, right? And so there's all of this work and then you need really, and you talk to tech folks all the time, you really need really, really high performance networking to connect all those things through. And so your networking equipment becomes really, really expensive. You need a lot of memory, you need a lot of software coordination and things like that. As you go into inference, the beauty of inference is it's scale out. And so basically you're adding racks as your users grow, right? And so with SambaNova, that minimum quantum is down to one rack, right? Where if you have other service providers, you just to run say DeepSeek model, which is now one and a half trillion parameters, just to run that, the minimum for some of the other providers might be 10 to 20 racks. And so you're starting to think about, okay, well, if I want to be an inference provider and I want to run these large models, my minimum cluster just to start serving is 20 racks and the cost structure's high, the power needs are high. Some of we can actually reduce the minimum down to a single rack. And so now you just grow as your user base grows, significantly more efficient to deploy, significantly more flexible. If you're going into environments where you don't have gigawatt, you can go into your data here in Paris, downtown Paris, you can find an existing data center and deploy five racks, 10 racks there for ultra low latency where the users are, right? And so this is kind of what's changed in the racks is that, you know, we're very focused on bringing it to broadbase deployment, standard everything, standard 19-inch rack, standard air cooling, no complicated liquid cooling retrofit in the data center. We're using standard Kubernetes, standard Red Hat Linux, standard Ethernet at the top for networking. We don't have to use all this kind of really expensive networking equipment that you need to gang these things together. And so that allows people to go into existing data centers, roll this thing in, pull out the old gear, and you're up and running with new services which otherwise might take you nine months to a year, maybe as long as 18 months to build a gigawatt data center, to bring liquid cooling in, to bring all this new power in. Sometimes you have to figure out, you know, how to secure new power, new energy and nuclear power plants. And I mean, all those things that people are talking about, it just takes a lot of money, a lot of time, but you don't have to do it if you use SambaNova equipment. That's pretty good.
Yeah. I mean, it's operating at the velocity of inference, which is people want to stand these things up quickly, right? It doesn't take months to inference because these models have been already trained, right? Whether that's an OSS model from OpenAI or Mistral model in France, we talk about Mistral, or you know, you've got the DeepSeek and MiniMax models, amazing models that are out there, or the frontier models or the closed source models that people are offering, it doesn't matter. You deploy this technology, you can bring those models in immediately and you're up and running with the latest and greatest AI models in the world and then you can upgrade them as you go along. So, do you buy the headlines that new data centers should be 50 or 100 billion dollars?
Well, I think you're going to have some data centers like that because I think there's still going to be large scale deployments, large scale access that people want. And I think you're going to find that the world is going to be heterogeneous, that there's going to be those large data centers that people go and secure a lot of capacity for some of the things that they want to do. And I think you're going to see this new wave of companies that are doing distributed data centers, right? So these data centers that are midsize, right? They're midsize and it's going to be even more important as you go into this agentic world because you know in the world of agents, you're not dealing with a single model and a single prompt, right? If I go to ChatGPT, you were here in France and you say, what should I do if I have an extra day in France? Right? You can talk to ChatGPT, it will generate an itinerary for you. That's between me and the model and it produces the result. In the world of agents, these agents are orchestrating within themselves without us. My problem starts in the beginning, 10 agents are all intercommunicating and each of them taking some amount of time. And so if you actually have a lead time or response time that's say two seconds, which for one user to one model is not very long, right? For our eyes, it takes us longer than two seconds to read the output, right? And so that's okay. In the world of agents where you have say 20 agents orchestrating with each other, if each of them takes two seconds to respond, now I got 40 seconds. The end user did an initial prompt here like, please move some money from my bank account, my Bank of America bank account over to PayPal and then give me a report of all my expenses over the last, you know, six months. Okay, that's my prompt. Then the orchestration of security and balances and all of that has to happen and the output, if each of those 20 agents took two seconds, that's 40 seconds, you've already given up on that prompt, right? So the response time by the end, the user's expectation is say one to two seconds, divide that by 20, it's less than a second per, it's 0.1 seconds per. And so your response time is going to be really, really important and that's why latency matters. And so you're now seeing us coming in and saying, look, we're going to deploy the hardware where the users are in large metropolitan cities, right? Because that's where business is being run and so latency is really important. So we're going to deploy that. Well, unfortunately in those large metropolitan cities, you don't have those gigawatt data centers. There's no space for Manhattan.
In Paris, right? Where are we going to find space to drop a, what did you say? How much money did you just say?
50 to 100 billion. I mean, where are you going to find even the space to build that? You're going to find some space in some place to build that, but in terms of ultra low latency in the big cities, you're going to have to find smaller quantums, right? Smaller spaces that allow you to deploy what you need for that, you know, for the users there that require that really, really low latency. And thinking banking, healthcare, like there are many use cases where your latency is really, really important and you're just not going to want to wait.
On the topic of inference, what is premium inference?
The way we define premium is ultimately the highest value use cases and there's two dimensions. Basically on one dimension is the size of the model because it's about accuracy.
Okay. And so years ago we used to talk about hallucinations, right? You talk to ChatGPT and come, come this... That was a fun time. I think, wait, like you should probably look back, like hallucinations was a fun time.
There's less of it now. There's less of it, right? These models are getting pretty good and there's less of it and yet accuracy is still incredibly important, right? And so why are these models going bigger and bigger? It's not that people want to spend the hundreds of millions of dollars to train, right? I mean, they're fighting for that bit of accuracy because you look at a model like Claude from Anthropic, right? You look at that model, why did it become the most popular code generation model? When software developers go and type, it generates really good code. And you take a different model, sometimes not as much. And this is where also the open source, the MiniMax model that you know that's out there as an open source model became very popular. Incredibly accurate when generating code, right? And so these models still are being valued significantly for the output that they generate because if you can trust it to produce good output, you don't have to invest as much human energy to go double check it, right? If you have to go double check it, then certainly you're starting to invest more time. And so on one dimension is premium is how accurate the model is and today that's proportional to the size of the model, right? And so you have models that are very near Llama 8B, for example, is an 8 billion parameter model, very small. By 70B, pretty small. 70B used to be the big, it's tiny today, right? When you had ChatGPT and GPT-5 at 5 trillion, right? The new models are heading towards 10 trillion. Even the open source models are already one to two trillion parameter models. And so now you're starting to see these models getting very big because people are looking for accuracy, right? And they want a model that handles a broad range of things but handles it correctly. And so we're very focused on making sure that we handle the largest models well. And then the second dimension of premium is burn it fast, right? For the reasons I just described about agents, that you know you don't want to take a long time, or we live in an impatient world. Right? You and I, I mean, a few seconds, we're starting to tap the phone, something happened, right? And so if the service behind it is agentic, you need to actually run really, really fast. There's not time for you to actually wait. And so that combination of running really big models, as you know, they don't run fast, right? Or you look at services like Groq that run fast, you can only run the small models, right? So how do you find the ones that run really fast on the big models? And that's where some of the strength is, you know, that we take the biggest models in the world and we run them in the original precision. We don't, you know, we don't quantize. You know, quantizing is, you know, you chop half the weights off. So we don't chop the model down, we just run original precision, full precision, we're faster than anybody else.
This episode is brought to you by Brex, my favorite. You become what you spend on, and I refuse to spend my time on work that shouldn't exist. Expense reports, receipt chasing, and manual closes. The companies building what's next from Vercel, OpenAI, Anthropic, Granola, and Deepgram all made the same call. They all run on Brex. Brex is the intelligent finance platform that combines cards, expenses, and banking into a single stack with agentic finance built in. AI agents that handle expenses automatically, enforce policy before spend happens, and close your books in minutes. That's why Sourcery runs on Brex so I can spend time on building and not busy work. It's time to get Brex AF. Learn more at brex.com/sourcery. That's brex.com/s-o-u-r-c-e-r-y. Bye.
At what point do you think speed is going to bifurcate the market much more into different pricing units? I know that token costs and everything is a really big topic right now, but for consumers, we do expect speed, but like, should we be getting enterprise quality speed?
Well, I think you're going to find that people pay for it. And it's always been true. If you look at the internet, you know, people paid initially, they would pay a premium for the upper end of the packages for faster internet, you know, and then some people will still remain on the basic, right? And same cell phone. But here, I'll say this. Look, when 5G showed up, nobody's signing up for 2G, right? If your phone, if your cell phone is not transferring very fast, you're getting very frustrated, right? So I think on the curve is over time. As fast inference is broadly available, who's going to want slow, right? And so it's going to not only be the premium today where kind of the people who need it, code generation people, real time banking, real time healthcare, you know, there's a number of industries where speed matters because it's their livelihood. They're going to pay up for it. That's kind of the premium service that we're offering. But over time, what you're going to find is that all of us are going to want fast. Like I don't see a use case where the average population, either consumer or enterprise, is going to say, actually, I prefer this slow. That wasn't the case on the internet. That's not the case on cellular, you know, on mobile data, right? It's never been the case. Let me pay more for this slow, right? Or even let me pay a little bit less for this slow, right? Most people over time are going to say, no, I want the fastest, right? And so as the cost of delivering fast goes down, you're going to see most people switch over to the fast. This is what I feel like, you know, the premium inference, which is large models, which equals the most accurate, the most accurate models and fast, ultimately steady state is what everybody's going to want, right? Is whether today we can offer a low enough cost that everybody can afford it, right? And so until then, you're going to see people very quickly moving over if they have a need for that fast inference and have a need for that accuracy, which is I think a large part of the enterprise.
There's also a couple other macro themes that are happening that are just going to explode data and usage across the world. One of them is Starlink. Like, I don't know if you've been flying around on planes that have Starlink, but even having access to that in remote areas, it just increases the amount of work you can do and work for, you know, like edge cases. Another part of that is edge computing. I think, do you guys deploy in like, um, like edge remote areas as well?
Well, what happens here is, again, this is tying into your question, is a $50 billion data center ubiquitous? It's going to be hard to say that when you have economies that can't afford a $50 billion data center, right? And you're going to see across the world. If you believe that AI is going to be a technology as pervasive as internet, which I do, right? That this is something that everybody on the planet should have access to, right? And so mobile service, internet, AI, everybody on this planet should have access to it. So if that's the case, then it's not going to be equally deployed because in some countries you can't afford to put a hundred billion dollar gigawatt data center somewhere and then let tens of hundreds of millions of people come in and use it, or you might be in a place where you know you don't have that and you need something significantly smaller. Today because of SambaNova technology being as little as 10 kilowatts per rack, we can put them inside shipping containers, right? So we build out these data centers, clusters of 10, 20 racks inside the shipping containers that you see on these ships, right? And then you deploy them in these edge data center use cases in a much, much more cost-efficient, power-efficient footprint than having to build out this liquid-cooled gigawatt data center with its own nuclear power plant, right? And then you can put, you know, a Starlink connection to it, you know, you can bring the internet in, you can actually have solar farms next to it, you know, and then you can actually power that. So there are lots of different ways that people are creating these data center clusters that allow their communities, right, whether that's in regions or in industries or, you know, in sectors, to be able to get access to the best models.
Yeah. One exactly like one particular company I was thinking of is Armada. I don't know if you know Armada, but Armada, they deploy modular data centers on the edge. And so that, I mean, those are, it's not just in like remote communities, but it's for critical industries, critical industries that don't have that real-time data that they've had before. So it's like oil and gas, it's mining, it's, you know, oil rigs out in the water, all that kind of stuff. And even for like military and defense and that kind of thing.
