High Performance Compute: Building the Cloud for AI Teams | Erik Bernhardsson | RAISE Summit 2026

Hello.

I took some time off after my previous job during the pandemic and I was sitting at home and honestly just started hacking on stuff. Maybe to go back a little bit. I spent 7 years at Spotify and I built a lot of infrastructure and I built the music recommendation system at Spotify. And one of the experiences is building the music recommendation system at Spotify is basically ended up spending 90% of the time on the infrastructure and 10% of the time on the actual machine learning. And so 5 years later during the pandemic, I was kind of toying around with different ideas like building some prototypes and I got really excited about like why is there such a big gap in infrastructure? Like no one's making it easier to do this. What would the cloud look like if you started over and built it for all these like AI, data, machine learning applications and yeah, I got pretty excited, started building what's now Modal. Took a couple years to find product market fit, but that's the origin.

I think at the end of the day there's like the most important thing is obviously it's like hiring people who are just like incredibly smart. But I think at a startup you want people who are like very high agency, very commercial. Almost everyone I think of out of the first 50 people were like friends or friends of friends. People we trusted, people we know have good taste, people we know are really good engineers. And I think you need to work with people you know and trust. People who are very commercial. You don't want these people who are just building technology for technology's sake. They need to understand that the point of a startup is to make money.

I've spent the last 20 years interviewing and hiring people. If there's anything I've learned is like the more interviewing I do, the harder I realize it is to predict how good someone's going to be based on the limited information you get throughout the interview process. So but I do think, you know, having structured interviews, you know, filtering for like excitement about the product, the commercial aspect, being like kind of a strong communicator and having sort of a you know, just like crystal clear ability to talk about what you're building. Like all these things are I think very important at startups.

Yeah, Modal is a little bit hard to explain, but roughly we're building AI infrastructure. And so the idea is that there are thousands of companies out there building AI applications, you know, incredibly cool things like Suno or Lovable or Cognition or Runway or Decagon or I'm mentioning a bunch of Modal customers. And all these application companies and many more hopefully in the future, they are in the business of, you know, building great applications and they shouldn't have to deal with infrastructure. So, we manage all that infrastructure for them. And it roughly boils down to two things. One is we offer a much better developer experience to getting things up and running in the cloud and scale managing all of the productional complexity of it. The other stuff is we manage all the infrastructure, too. So, we have, you know, 20,000 GPUs, you know, probably 50,000 by the end of the year. When you deploy things to Modal, like we manage that for you. We end up basically becoming a cloud provider for you. So a lot of you there's a lot of different use cases. Inference is our biggest use case. That's about 2/3 of our revenue. But we also have a sandbox product. So for inference, it's a lot of LLM inference, a lot of GenAI, like image, audio, video stuff. For sandbox, it's all these like, you know, agentic stuff, a lot of reinforcement learning, background tasks, there's our background agents, vibe coding platforms, we power a number of vibe coding platforms. Working on a training product, working on a batch product. The limit of Modal is basically like we want to build like a whole toolbox of like everything you need to build an AI application. Like in the limit, it's like don't even go to like AWS or NeoCloud. Just come to us and we offer a much better way to like get these applications to production. You know, fully usage-based pricing, you can just iterate, you can just pay for whatever you're using. You don't have to think about reservations. If you need a thousand GPUs, we can get you that typically within seconds. Sometimes minutes, it depends, but like very fast. So, you don't have to think about that sort of complexity of capacity, routing, load balancing, containerization, all this stuff. We take care of all of that and have a bunch of different tools on top of that.

It's super hard. I think one of the challenges but also the opportunities of Modal is like there are so many directions we can go in but that also means it's very daunting like to kind of figure out how to prioritize all of that stuff. We ended up spending the first 2 years building a lot of low-level infrastructure which is people thought we were kind of nuts. Like we built our own file system, we built our own container runtime and scheduler. Like all this stuff. We don't use Kubernetes, we don't use Docker. We built all of that ourselves. And those were hard challenges that we had to overcome like trying to figure out how to do these things and had to put a lot of people on these things. But we felt like it was needed because like existing tools didn't actually give the developers we weren't able to deliver on the developer experience we wanted with existing tools. So we decided to build all of that ourselves. Now the question is like which direction do we go in? Should we focus on training? Should we do LLM inference? Like I mean we do LLM inference but should we build a bunch more features? Should we go into some other verticals? It's super hard. I think at the end of the day like we have 100 engineers even if we had a thousand engineers it'd probably be quite hard. I bet like Andy Jassy or whoever or Matt Garman who runs AWS he has like 50,000 engineers. I think at the end of the day he still has to make some hard decisions about what to bet on. So it never goes that way. But I think you have to have a little bit of like sort of your own experience. I'm leaning a lot on my experience building AI and machine learning applications. A little bit listening to customers. A little bit kind of just speculating about where the world is going.

Yeah, and just for context and not to give ourselves too much credit, but like one example like 3 years ago we released a sandbox product and it came out of a conversation I had with my co-founder Akshat. We were talking, we realized a lot of people seem to be using LLMs to generate code. And we figured they need a place to run that code in a way that's like kind of isolated and safe and programmatic. So we released a sandbox product. Barely anyone used it for 2 years. But then it exploded last summer and now we have, you know, tons of revenue from that. So it's a little bit like infrastructure is always like that. You kind of just have to like argue from sort of, you know, what you see on sort of a principled basis and just kind of build for that. I will say like just to answer more questions about like the future, a couple things like right now I'm excited about I'm actually very excited about CI, continuous integration. I think with all these agents the CI has always been like kind of like a space full of like terrible products in my opinion. Like GitHub Actions is the biggest one and it just sucks. I think there's always been an opportunity to build something better, but I think especially with agents iterating on code so fast that means there's even more of a need to build like faster basically. I think the ability to run tasks and verify that code works is like a big bottleneck right now in delivering software. So that's a space I'm pretty excited about. Another space I'm pretty excited about is audio and video. I think we've seen some dramatic progress with LLMs. And LLMs are obviously incredibly, you know, useful for many different tasks. Audio models are still like especially like speech to speech are still kind of bad. I don't know if you have like audio assistants like Google Home or like Alexa whatever. They're just like so bad still. Siri is so bad. That's a space that I feel like we're just like one or two breakthroughs away from making that work. So one of what we're doing at Modal is we're investing a lot in low latency infrastructure. Basically the ability to like run GPUs all over the world, kind of build like a CDN for inference, and just like have like very fast, low latency, streaming, bidirectional inference. So, yeah, CI and real-time speech-to-speech would be two things that I think, you know, give it a year, give it maybe 2 years, but I think they're going to be huge.