The Compute 100: Erik Bernhardsson, CEO of Modal

When you're building infrastructure, you spend basically like two years in a cave, because like if you're building a database, like you know, you can't like sell a database until you know you figured out, you know, storage, transactions, durability, like consistency, like all these like hard things.

It's great to be here.

So Modal, I think the easiest way to put it is we're building AI infrastructure, though it's a little bit more broader than that. We started five years ago. We had this vision of building a fairly general purpose platform for all kinds of applications, mostly in AI, machine learning, data. Spent the first couple years building a very sort of deep compute platform and to some extent also storage. We basically built our own file system and our own container runtime, and two years in, we didn't really know what's the killer application, what are we doing here? But then Gen AI came and it just turned out to be exactly what we're looking for. So over the last three years, four years or so, we've seen a tremendous growth with mostly Gen AI inference. More recently, our sandbox product also has kind of taken off, and we've seen a lot of use cases around everything from background agents to reinforcement learning. We also have people using us for training. We have people using us for fine-tuning and all kinds of stuff. So it's starting to emerge like this like platform thing, and really the goal for us is to help people that are building AI applications not have to think about infrastructure. We handle the infrastructure below.

We compete a little bit with them. I think it's like kind of borderline. I think our vision is much bigger. I think, I mean, my view is like the biggest competitor is like homegrown Kubernetes wrappers or clouds maybe to some extent, although we use the cloud, so I think it's like more like building a layer above. So I mean it depends on the customer, but I don't know, I think I like to think we're building something new, maybe more like, you know, something that hasn't quite existed in the past. But certainly for certain types of applications, LLM inference for instance is one of the things we do, we do end up having, you know, Fireworks and Baseten competitors.

I mean, some of it comes down to just like, you know, we're better along dimensions like cost, performance, latency, things like that. We have a super crack team like doing a lot of LLM inference optimizations like in the LLM inference space. But like what people also like Modal for is it's much more broad. Like we have a sandbox product like I mentioned. We do have competitors in the sandbox space. We tend to win really big on scale in the sandbox space. And then we have a bunch of other stuff. We have, you know, batch jobs. I think we're, you know, much there's a much bigger delta there versus I don't even know who's the competitor. I think that would be like more like people just going direct to the clouds, which is very painful. We have a notebook product, which there's obviously some notebook companies. But yeah, like the goal for Modal is always to build like something very general purpose, and I think that's maybe something that is quite unique to us, except for maybe like, I don't know, Databricks, but you know, we're taking a little bit of a different approach architecturally.

Yeah, I think a lot of what led me to build Modal or to think about the ideas that led to Modal was I spent seven years at Spotify, and in particular, most of that time I built the music recommendation system, which is a very complex system and arguably AI, right? Like, but not Gen AI necessarily, a lot of matrix factorization and complex stuff. Basically ended up spending way more time building the infrastructure to support the system than to actually build the system. Ended up open sourcing, for instance, a vector database, a workflow, and then I don't know, fast forward like five years later, I quit my then job, which is a different job, started playing around with ideas.

That was Better, correct. Yeah, so that was another kind of chapter of my life. I was the CTO there, but yeah, started playing around with different ideas. I got very excited about just like there's kind of a lack of good tools for building like data-intensive, compute-intensive products. And that's something I'd experienced firsthand at Spotify, but also elsewhere. There's a lot of different just, you know, compute-intensive stuff that is not well supported. So started to think about like how would that look like? And pretty quickly I got super pumped about this idea of building something like Modal. And yeah, like again, like I mentioned, spent like a year or two having a very strong confidence that like we're just going to build this compute platform and then it's going to be useful for something. I didn't know what exactly it will be useful for, but then Stable Diffusion came, I think it was like September 2022 or something like that, and we realized like we have GPU support, we have this serverless, this is like a great application. So we put out a demo very quickly, and then it just kind of started exploding. LLM inference was something that we started doing a little bit later, but...

I mean, there's a little bit of luck, a little bit of like serendipity. I do actually think like even without Gen AI, I do think we would have found many other applications. There's many applications in biotech and other places where I do think something like Modal could have been very valuable. We would have maybe had a very different angle. Yeah, like you know, because we built this compute platform, right? And then we got pulled very hard in the direction of inference for...

Something like that.

It's like next month, like...

Yeah, I mean, I wish I could say I had this like, you know, insight and I was like, let's bet the company. Like, it's always more complex than that. I think we did it when it did come out, we were very fast at like putting out a demo of how to run it on Modal, and just organically people started picking that up and like, yo, this looks like super cool, like can we like run our whole like, you know, company on this because we do Stable Diffusion? And so it took another couple of months, but then we ended up, we started seeing like a bunch of like larger users, I mean large for that time, we had like basically zero revenue, picking that up. So it was a little bit organic. So it wasn't until like maybe five, six months later where I felt like, okay, I think we have product market fit. There's a market is pulling us very hard right now.

Just hyperscalers.

Well, we're not hosting models. I think a different thing we do at Modal is like we never really care about like being an endpoint provider. We let people run their code, right? Like it's a very general purpose platform. It's almost like, you know, Kubernetes or AWS Lambda or EC2 or something like that in the sense that like we take users' code and it could be anything. So it's a very general purpose platform in that sense. It is very easy to run Stable Diffusion on Modal, and we realize like that is a golden opportunity for us.

I mean, I do think the hyperscalers have like a great business model. They're in a great spot. I think it's, you know, primarily Jensen likes to kind of diversify the buyers. So if it wasn't for that, I think actually like I don't think we would have seen as many neoclouds. So it's late March 2026.

So I don't know. Like I find it so hard to speculate on these like macro things. Like I don't know how long this is going to last. Like last time we were in it for like, you know, six months, nine months, twelve months or something like that. Like we're probably going to be in it for a while as well again. But yeah, like the last three months has been, there's been a complete change in the sort of the market, the supply demand balance.

Yeah. I mean, I would also argue like a different way to look at it is like three years ago was like training driven, and training to me is like, sure, yeah, you obviously need to train models, but now it's inference driven, and inference driven is like way more correlated with actual demand and actual user value. So in that sense, I think there's like sort of more of a, it's more sound in a way, like there's more of a sustainability in the supply shortage today than it was maybe three years ago where it felt a little bit more speculative, a lot of people just hoarding GPUs because that was like a meme, that was a thing you did as an AI startup.

Five months ago or three months ago or something like that, we were primarily running on the hyperscalers plus like one or two neoclouds. Today we have like 20 neocloud integrations, something like that, just to be able to get deals done with like any possible neocloud. The capacity is out there. The prices have gone up. The terms are not as compelling as they were before. You know, you need to do longer-term deals. You need to pay more money up front. But fundamentally, there are deals to be made. There is capacity out there. There's a lot of stuff getting installed. I just came back from GTC. I mean, I spent like two days running around like it felt like an open-air drug market, just like begging for GPUs, you know, do you have any, like just, you know, trying to find, you know, supply wherever I could. But, you know, so definitely I feel like as a CEO, I've gotten way more pulled into like, you know, figuring out like where do we get the GPUs. I think that's the value prop also as Modal. Like, you know, you want a lot of GPUs, like you just go to Modal and spin them up. Like that's always been part of the value prop and what we sold. So it's got a little bit harder for us, but you know, that's also like part of the value we create. So...

I mean, right now we're running 10, 15,000, 20,000 GPUs depending on the day. I think by the end of the year we might run 50, 60 plus thousand GPUs. So I mean, we're going to have to go out and buy, you know, a few thousand GPUs every month through this year. So, you know, it's a lot of money.

Yeah. I mean, it is like a different skill, you know, like you're like WhatsApping with people in random countries and just like, do you have any, like, you know. But I don't know. I kind of enjoy it. It's like actually, you know, it's like sales too. It's like a skill I've had to pick up. Like, you know, I didn't do B2B sales before. Like, you know, I kind of enjoy it in a way. Like GPU stuff is like very quantitative too. Like we do a lot of very advanced stuff on our side with the hyperscalers. Like we monitor their pricing dynamically. We have a whole, it's like a linear programming. It's like a complex optimization problem that we run every minute to basically like match supply and demand and scale up and down dynamically based on where we see demand and solve that in a very quantitative way. So it's actually a very fascinating problem to me, that sort of dynamic like supply demand matching. So I kind of enjoy it.

Yes. Correct.

Yeah. So the value we create, I mean, we create a few different values, but I would say like there's like you can distill it down into like maybe two major things. The first thing is like we manage all the capacity. So you don't think about capacity. Everything is serverless. Everything is usage-based. When you run stuff on Modal, you can scale up and down. Like we just charge you for the time the GPUs are actually running, and we're a multi-tenant platform. So because we have a lot of users, we can sort of aggregate that capacity and manage that at scale very efficiently. So if you need 100 GPUs, like we can get you that within seconds typically. If you need a thousand, we can get you that within minutes, but you know, sometimes even faster. The other thing we offer as a major value proposition, this is the developer experience. In the past, let's say you're training your own proprietary model, we work with, you know, Suno for instance, they have their own proprietary AI model for music generation. Like we have a lot of customers like that. If you had to deploy that yourself to the neocloud, you'd have to think about the containerization, load balancing, routing, you know, dealing with, you know, writing a bunch of YAML in Kubernetes, you know, and there's a lot of issues with that sort of besides the fact that, you know, just the scaling and in particular like the developer experience is awful. So one of the big things we enable for a lot of these AI companies is we just factor that out. Like we basically say you don't need to think about infrastructure. Like just let your machine learning researchers just deploy to production. Like we handle all of that for you. All the routing, all the load balancing, all the containerization. You can iterate super fast, try things out. Like we just take care of that. So that ability to move very fast is the other big thing that people really find compelling with Modal.

I think in a way like Databricks and Modal probably sees the world in a very similar way. I think we are both companies, Swedish CEOs actually kind of randomly, that both see a kind of world where there's a lot of sort of AI products out there that all like building a platform. Like we're trying to build a platform with a whole toolbox, a lot of different tools in the toolbox that all help people build AI applications. I think Databricks is like building it in a very different architectural way. Like they're primarily single tenant. It's not necessarily in my opinion AI native or container native. I also think to a very large extent they're focusing on a very different user base. Like we are focusing on the cutting edge like AI startups. Like I don't think we've been in a deal, hasn't been a single competitive deal so far where we've seen Databricks. So I think while we think about the world in very similar ways, I respect them a lot. I don't consider them a competitor.

Yeah, there's a few different things. I think inference, you can slice it in a few different axes. I think obviously maybe looking at modality first, like within LLM inference, like there's like new breakthroughs every day. Like this morning there was a new KV cache thing coming out of Google. There's like stuff like that, and the world moves just super fast. And so one of the things we have is we have our own team, mostly based in San Francisco, super cracked, like kernel CUDA wizards. Like they're all just like, you know, contributing to FlashAttention 4 and, you know, vLLM and SGLang and just like making sure that like we can offer the best, fastest, cheapest inference experience for LLMs on Modal. Then you look at other modalities, you know, audio, video, those are often like more driven by like very proprietary companies or like models companies like building their own models basically. In that case we focus on more like the autoscaling, our ability to just like manage the infrastructure. Another thing that comes up in all of these things is low latency. So we're basically like rethinking a lot of how we think about inference, like decentralizing everything and running things all over the world. The idea being like we can serve you from like wherever we are, like near you. We'll have GPUs, we already have GPUs in 80 or 100 different regions or something like that, but making that sort of very dynamic. We've already spent a lot of time on the cold start problem and like managing containers and scaling that up. So with Modal you have this like very dynamic experience where you can, and I think that's something that's very, very important for inference, but somewhat, you know, idiosyncratic. Like you don't need it for training. It's actually fine to work with like this rectangular block of like GPUs, but for inference you have very volatile demand. So I think the last five years of, you know, investing in our own file system, investing in technologies like GPU snapshotting, CPU snapshotting, the ability to like scale up like a thousand containers in a few, you know, seconds or at least minutes and, you know, pre-initialized with all the models in GPU and stuff like that. Those are things you're going to need at very large scale with all the inference demand growing. So I think we're quite well positioned to serve a lot of that.

For us, it's actually been pretty evenly split across modalities, which I'm excited about because I think the other thing with LLM inference is like it is very large and is very growing, but it's actually dominated by a very small set of models. So it creates a little bit of a different dynamic where I think there's like thinner margins in that space. And then of course there's like the sort of threat of like custom chips. So I'm glad, I mean, there's a lot of revenue opportunity in that space. We're growing very quickly. But I'm also like very happy that we have a lot of users in other modalities. So I would say like it's all, you know, text versus audio versus video. Those are all like fairly evenly distributed. We also do a lot of other stuff. Like we actually have a lot of customers in computational biotech. We have a lot of customers doing, you know, molecular simulations, protein folding, things like that with Modal, like very compute-intensive stuff. We have other stuff, we have some people using it for weather forecasting, for instance. So, you know, and then of course like, you know, the sandbox product, etc. So it's not just modalities, also, you know, industries and use cases. We actually feel like we have a pretty diversified set of use cases, which is exciting.

It's growing very fast. Very fast.

I mean, like we, I think it's like basically public that we crossed 100 million end of last year, and I mean like we've grown a lot since then. So...

For LLM, it's almost everything, right? You know, for the other modalities, like we work directly with model builders in many cases. I don't know, I don't have like a super strong point of view. Like I think it's been remarkable to what extent, you know, these companies, especially out of China, have been able to, you know, compete with the frontier models. And then of course, like if you work with specific customers and they have their own data set that you can fine-tune on and do reinforcement learning, you can beat the frontier models, which I think is very exciting. So we work with a, you know, there's a small handful of customers, they do the, you know, reinforcement learning or fine-tuning a model, and they, you know, see better results, you know, starting with Qwen or one of those, and using that rather than Anthropic, which I think is remarkable. And that's not throwing shade on Anthropic. I think they're a remarkable business too. Fantastic business. But I think it's been sort of fascinating. Yeah, I wouldn't have expected a few years ago.

I mean, I think there's like definitely pockets of like exuberance. You know, you see some of these like big rounds like being raised. Like, does the world need, you know, 35 frontier labs? Like, probably not. Like, I don't know, it seems like there's like a little bit of like an exuberance in that space. Um, you know, long term, like what does inference demand look like? I don't know. Like, but like on the other hand, like there's so much like customer demand for this stuff. Like, I, you know, so I find it pretty hard to believe like we're in a bubble. Like, I think, you know, there are things that look bubbly, but I actually don't worry that much. I think there's just so much stuff to be built and it's happening so fast and there's so much customer demand, so much willingness to pay, and people are clearly seeing value in these things. So I'm pretty pumped.

Yeah, I mean, biotech to start with. I mean, drug research, right? Like, you know, and it's obviously, you know, drugs take a long time to find, and, you know, so we'll see it probably in 10 years, like sort of downstream of all these things. But like, of course, if you can cure cancer, like that's a pretty good ROI. Um, or, you know, or whatever, like obesity, I don't know, whatever people suffer from, many things. Um, then you look at things like coding, of course, you know, everyone, you know, in the space knows this, but like, you know, AI coding today or coding as a whole is obviously very different than it was just a few years ago. I don't write like a whole lot of code these days, but when I do, I mean, I usually use like AI tools, and they're very good. Like, they're getting better. Like, I don't know if it makes me a 10x, but, you know, like 1.5x to 2.5x, like, I don't know. Like, so clearly a lot of value. Um, what else? I mean, we have a lot of customers. There's a lot of like use cases that are just like fun, you know, like using, you know, AI to generate music. I spent, you know, seven years at Spotify. I love music, and I think music is always like a place where you see new technology, you know, going back to like Edison, it's always been, you know, gramophones or whatever. Like you always see like the first wave of any technology sort of in music, in my opinion. Um, what else? Uh, I was thinking about something else. Drawing a blank now. So much stuff.

Vibe coding, I was going to say. Oh, there's like vibe coding tools. Like who could have predicted a year ago, like you got Lovable, which is a customer, you know, Replit, you know, v0, there's so many of them. Like I don't understand, but like, you know, I tried them, they're really good. Like it's crazy to like, I would have never been able to predict like a year or two ago. I actually know Anton at Lovable, and when he started the company, I was like, that's not going to work. And then like, you know, a few months, and he asked me to invest, I was like, uh, I don't know. And then like a few months later, like he just started like posting, he's like, revenue's like that. And like, so I was like, oh my god, Anton, like you got to let me invest. And he was, you know, very, he let me. So I'm very happy about that. But, you know, and now they're doing whatever, like 400 million or something. It's insane. You know, people are paying for this.

Anton wasn't, he was at Depict, which is another company I invested in. Um, there's like a few, I think the Lovable, the Legora people came out of Depict too. Yeah. No, Depict. What are it called? Depict. That's what it is.

Well, Spotify, that was like completely serendipitous. That was like me in university, like not knowing, you know, I graduated university. I mean, this is a long time. It was like 2009, um, 2008. And I did a lot of competitive programming growing up, and there was like a kind of a small group of people at my school that was pretty good, a couple older, couple years older than me. And, you know, so I looked up to them. They were...

Very good, and basically all of them joined Spotify. Yeah, and so I was like, what is this? What is Spotify? Like this music streaming? I don't think that's going to work, but like clearly I have a lot of smart people, and you know, so I don't know, I'll just go work there and see what happens. Like I want to work with these people. So I was actually kind of skeptical about the old business model when I joined Spotify, but you know, it worked, you know, and it's great. Like I mean, I love the product, I use it so much, it's an amazing product. So I had seven very interesting, awesome years at Spotify, learned so much.