Hey guys, we are back here at AWS re:Invent, day four, and I'm here with Erik from Modal.
Yeah, it's great to be here. Thanks for stopping by our booth.
So, Modal — we're building AI infrastructure is usually how I explain it. We basically, our thesis is basically a lot of traditional infrastructure does not work in the age of AI, and all these people building AI applications, they need better abstractions. They need to work with GPU capacity all over the world. They need to scale up and down. They need to work with large models. We basically decided traditional infrastructure — we're going to rewrite all of that. So we built a whole new stack. We support a range of AI use cases like training, inference, notebooks, code execution, powering a lot of different use cases like computational biotech sometimes, but also a lot of GenAI of course, vibe coding, and many other things.
How did you come to start working on this problem?
I mean, I love infrastructure work. It's so fun to do the foundational stuff, and there's clearly this like enormous transition that's happened around AI products and what they need.
But how did you get this going?
Yeah, it was somewhat serendipitous. I mean, I've been building machine learning and AI products for a big part of my career. I was at Spotify for many years, built a music recommendation system there. And I saw a big gap in infrastructure for these applications, like data-intensive, compute-intensive applications. Started hacking on it during the pandemic, that was just before the sort of GenAI thing took off, and so we were there like quite well positioned to power a lot of these particular — like Stable Diffusion was our big, you know, first killer use case. But over time we see a lot of things around even like protein folding, but also like LLM inference of course, training and fine-tuning, and now vibe coding has been a big, super fast-growing vertical for us in terms of building infrastructure providing this.
Where kind of are you in the stack? Are you like buying chips from Jensen and racking machines? We're just above the metal — like how does that work?
We're a big customer of AWS. We spend a lot of money on AWS. We run a bunch of other clouds too. We don't want to be in the business of running data centers. We want to do everything above. So we get the bare metal, we run code on them, and we built our own container layer. And then the idea is like basically we do everything up till the actual application. So we work with AI application developers, like machine learning engineers basically, and they write code, they deploy it on Modal, and we manage all the sort of capacity and scaling up and down. It's fully user space too, kind of nice with GPUs because they're expensive. So we only charge for the time that GPUs are actually running. So we kind of sit in that layer between. Our view is that GPUs, they're obviously like, you know, amazing computational machines, but you also need a software layer above to make development easier to work with these GPUs.
Could you give an analogy to maybe like traditional infrastructure in terms of how that translation works? Is it sort of like how people think about Lambda or running VMs, or is that like a hypervisor type of thing? And how is it different from the GPU workload?
Yeah, I mean, like in a way we're like function-as-a-service, like the cornerstone of what we do, right? So you can deploy code and it basically acts as an autoscaling function. Unlike Lambda, we focus on very compute-intensive things and GPU support and running all over the world, and actually good developer experience. You know, I mean Lambda is a great product, but it's kind of hard to work with. And so what we do is we use gVisor under the hood for isolation of the workloads. I think another way to put it is that we replace Kubernetes in many cases, and Docker also. So we're like in a way like kind of a managed system that replaces all the underlying orchestration and the scheduling, but we're also, you know, in a sense like we almost become like a cloud provider because people deploy code to us and then we kind of abstract away the underlying clouds. So from their perspective it's like they're running it on Modal, we charge them for the time it's running, in the sense like we then arguably become a cloud provider. I would say like another analogy would be like Vercel.
I was just gonna say it sounds a lot like Vercel, which also is a big AWS customer.
Yeah, where like, you know, and Vercel is like maybe, you know, they're more in the frontend, you know, focus, but like you can think of us as like Vercel but like more like the AI — I mean Vercel is like also aspirational doing more AI, but yeah, we're like very much focused on like heavy-duty like machine learning applications.
Yeah. You're actually doing the AI.
