CEO & Founder, Lambda
Serverless Framework v4 with Austen Collins
Ep #94 of the Real-World Serverless podcast
In this episode, I spoke with Austen Collins, founder and CEO of Serverless Inc. about the upcoming release of Serverless Framework v4.
We talked about the origin of the Serverless Framework and the challenges it faces. We discussed the rationale behind the upcoming changes in v4. Including the ability to easily switch between containers and Lambda functions as the deployment target, and the revenue share model for Extensions.
Links from the episode:
Serverless v4 announcement post https://www.serverless.com/blog/serve...
HashiCorp's license chan...
About Austen Collins
Austen Collins, founder and CEO of Serverless Inc., discussed the upcoming Serverless Framework v4 in a December 2023 podcast. He stated that the new version would introduce a revenue-based pricing model, charging companies with over $2 million in revenue for future versions, while also adding the ability to switch between containers and Lambda functions as deployment targets. Collins described the change as a shift from giving away the open-source framework and monetizing elsewhere, and he characterized serverless as "still the best option in town by far" for getting to market at low cost. He also commented on the broader infrastructure landscape, noting challenges such as anti-competitive licenses and a "Russian doll effect" where projects reuse each other's components. In earlier appearances, Collins has consistently described serverless computing as the natural evolution of the cloud and emphasized the goal of reducing deployment speed to around three seconds. He has spoken about the importance of developing directly on cloud infrastructure rather than relying on local emulation, and has advocated for separating infrastructure resources from frequently deployed code to avoid deployment issues. Collins has also discussed his involvement in the NFT market, describing it as a community-driven space and stating that "play-to-earn NFTs are going to change everyone's life," while cautioning that research is necessary to avoid projects that are "rug pulls" or derivative works.
Transcript (78 segments)
I
Interviewer0:11
Serverless Inc, and you may know him as the guy that created the Serverless Framework. Hey Austin, good to finally have you on the show.
A
Austen Collins0:17
Hey, and great to be here. It's an honor to be here. You know, few have been in the serverless movement and producing quality content as long as you have, so super excited to be on the show and just to have an interesting talk with you about all things serverless.
I
Interviewer0:34
Yeah, I've been using the Serverless Framework for so long. It's still my preferred option when it comes to doing any work with AWS and serverless. So I was a big thanks to you in terms of all the work I've been doing the last couple of years with serverless stuff. And I guess it's amazing to hear you say that too. You know, when did you get started?
A
Austen Collins1:12
API Gateway integration was announced at re:Invent 2015. I think that was a big turning point in terms of now you can build entire systems using serverless stuff. I think when Lambda first launched in 2014, which is the S3 trigger, and then Kinesis, you could do some small bits of stuff. So you upload a file to S3, you can do some automatic processing with Lambda. But once they introduced API Gateway, that became a game changer. You can actually do more general purpose compute stuff with Lambda and build real systems. So yeah, I started working with Lambda just before API Gateway integrations were available. Back then, I actually tried...
I
Interviewer2:12
I think 2016 when I started doing work. Well, I left one job with an e-commerce company, went to work for a social network in 2016, and that's when I started my first really big fully serverless project. And I think at that point you guys had the 1.0 or maybe no, it wasn't 1.0, it was a 0.5 or something version that was the current version of the Serverless Framework. At least at that point it was fairly stable, it was usable. Everything I needed to do was able to do with the Serverless Framework. So basically it saved me from having to create a lot of my custom build tools.
A
Austen Collins3:10
Where no one can debug anything, to running everything in Lambda and trying to figure out a lot of the patterns and what it actually looks like to run a serverless application in production. How to do debugging, how to do logging, how to do testing, all these things that people haven't quite worked out. So we spent a lot of time just figuring a lot of those things out. And the Serverless thing was a big part of that. So that was my guess when I got into serverless, almost eight years now I guess.
I
Interviewer3:38
Yeah, you were there right at the beginning. That was when everything really started to come together. And I forgot you were a JAWS user. And maybe I could back up and explain some of the context for listeners.
A
Austen Collins4:10
Based on the success of the Serverless Framework, I'm an AWS Community Hero. I think I was one of the first few serverless heroes. And what else is relevant? Also created a specification adopted by the CNCF called CloudEvents for standardizing metadata on event payloads so that they're easier to transmit across environments and just be more portable generally. And today, still running Serverless Inc with a bunch of awesome teammates, and we live and breathe serverless. But we got started, I got started, again back in early 2015. Actually, late 2014 when Lambda came out, and for me it was...
And thinking, this has got to be the natural evolution of the cloud. You know, let's have APIs for every use case that are autoscaling, pay per use, never pay per idle. And a lot of what development could be is just composing these things together to make broader applications and larger use cases without having to need a ton of money or a huge team to do that. And so I was such a huge fan of that general theme of just empowering the few over existing power hierarchies. It's just what it's all about for me when it comes to dev tools. And so then Lambda came out and Lambda solved this missing piece of, well, here's how you could run custom code, business logic really easily in a way...
Architecture on the horizon. Like, there's a new architecture that's possible. Let's see if we can build entire use cases and applications on all these managed services and just putting our custom logic in Lambda basically to deliver software again faster than ever, scales effortlessly, and only charges you whenever it runs. That just felt like this has got to be where the cloud's heading, right? This has got to be the natural level of efficiency that's coming to the cloud. And so, but at that time there was no story for doing that. Lambda came out, it was really weird. There were a few limited use cases, like you were talking about. It was so early. And I kept kind of waiting around for a couple months. I was like, come on, someone's got to be doing this.
Hacking away, and definitely a side project, right? I'm like, I'm going to spend this much time on this project every single Sunday morning. And it was called JAWS in the beginning, and it had this cool shark mascot. And the tagline was 'The Monstrously Scalable Serverless Framework' with this really cool shark. And I put it out there. I almost didn't post it anywhere, but I was going off to lunch and I kind of put it on Hacker News before I went off to get a sandwich or something. And I came back and it just blew up. It's just one of those incredible things. I hope everybody has an opportunity to experience something like that with something that they've made, where you just see people gravitate.
It's hard to explain, but developers loved it. And that was the beginning of the Serverless Framework. We had some challenges around the trademark with JAWS and the shark icon. And when I say 'we', it was really just me because I'm working on this myself, not making any money off of it or anything. And all of a sudden we had some copyright infringement warning letters, and we knew we had to do a name change. And it really broke my heart to do that because I really loved that initial branding. But it wasn't clear that serverless was the category or would even be a category. It was just something that was...
Broader narratives and place the camera to get people to lean in and get engaged in the story, empathize with characters, and build something that's emotionally resonant. And I've always taken those lessons to the products I build because I think you have to design them for emotional impact as well. And so when I was out there in the early days talking to people about this, it was the serverless buzzword. I would say that to developers and I just watch them light up. And I couldn't explain it. I think they just loved hearing that word because serverless was kind of this general description for everything that they didn't really want to deal with. They just wanted to build. Typically they don't like to maintain a lot of things. And I was just noticing that I'm always looking for when people are leaning in or they start...
To see if we can get the domain. And again, so early, a lot of people were saying this is a silly name, this is a weird name. Why would you call it serverless? And I'm like, well, it's about as technically accurate as 'the cloud', right? In terms of an adjective. But the emotional response when you tell that to a developer is so real. So we changed everything once it was clear I could get the domain. Changed everything to Serverless Framework and created serverless.com, and then we were off to the races. And the vision in those early days was, let's build this framework for this new serverless kind of managed service architecture. And you know what happened...
Adopted as, 'This has got to be top three strategic priorities over the next 10 years for us because we think we could really lead cloud in general through the serverless category.' And so in the early days, the goal for me was always we want to support all managed services, be open to all vendors. Developers shouldn't need to switch things around, but they need to have freedom of choice. They need to be able to take the best of breed products, and there should be one universal experience for doing that, composing these things from any vendor into an architecture that's just kind of wrapped in great developer experience. But AWS came on very strong and invested so aggressively in the serverless category and continued to...
The AWS architecture. And it was together, I think, with us showing that there was a real architecture here and making it accessible to developers, and with the strength of AWS's marketing budget and their investment and belief in this space too. And I have to give them all the credit in the world for coming up with Lambda and whatnot. But I think that's kind of the story in general of the Serverless Framework and in many ways the serverless category.
I
Interviewer12:39
Yeah, the stuff that AWS has been doing the last 10 years has been pretty amazing in terms of the amount of services and features that they continue to pump out even now. I think there have been quite a few pretty big...
For server-side rendering and a few other things as well. And I guess by the time people listening to this is already after re:Invent, so I'm sure lots of other things probably got announced during re:Invent as well. And I'm still surprised by the fact that Google is still very much focusing on the container story. I know Google Cloud Run has got quite interesting in terms of being a serverless container workload and being able to run containers fairly with low effort. And AWS has got Fargate for that as well. But Microsoft has also been doing a lot of work towards their Functions as a Service offering.
Well, I guess just going back to the Serverless Framework, obviously you guys announced that version 4 is going to come out sometime in 2024. And there have been some really big changes that have been announced already in terms of there's going to be a pricing structure. So for companies that are making more than $2 million a year in revenue, they may have to start paying for some features. Also, there's going to be a new extensions system, which is the next iteration of plugins if I understand correctly. And I think also in the launch blog post there was a mention of some...
For folks who are wondering about what's going to happen with Serverless V4. And I know a few customers have asked me, 'Okay, what are your thoughts on the coming changes to the Serverless Framework because of the pricing change?' So I guess I want to get your take on, well, get your side of the story. You know, why the changes? What are we going to be getting with Serverless V4? And tell us about why you guys are making this move and what does it mean to customers?
A
Austen Collins15:45
Yeah, absolutely. Well, we talked a bit about the origin in our background and whatnot, which you were certainly experiencing firsthand as...
In tech years, I'm sure you can relate. It's a long time. And the framework has been this incredible success story. We've had millions of developers building millions of serverless applications. And the framework continues to grow year over year at a phenomenal rate. And the best part of it for me personally is that now there have been dozens of companies that have been founded with a big focus on just using serverless architectures and the Serverless Framework who have achieved unicorn valuations and have even exited or IPO'd with smaller teams largely just using the Serverless Framework.
There's no better proof in the framework and especially the movement and the underlying infrastructure as a service that this is so powerful than hearing these stories of people starting with nothing, very few resources, achieving these astronomic valuations and exiting successfully. Like, yeah, this is the stuff that gets me out of bed in the morning. After all this time, you can really see the impact. Plus, in the majority of Fortune 500 companies, you'll see the Serverless Framework in there somewhere, no doubt. So the framework has been this phenomenal story. And frankly, in many ways, I can't explain it. From that first day of posting all the way to today, the growth continues. And we haven't spent...
This thing in many ways, I can't fully explain. And so, you know, excited, feeling happy about that. And then but then there's the serverless movement today. And on that side, it's like, I don't know about you, but I'm not feeling like we've had the success that we were hoping for. And when I go back to that original vision of this world where you have all these managed services and an easy, universal developer experience to compose these together, harness them, and build applications with low overhead, faster and more easily than ever, I don't know. I don't see it. I haven't...
On the movement, it's like, well, first off, on the upside, I think there's a ton of new vendors in the serverless space, which is amazing. This is exactly what I was hoping. Serverless is what we expect from new infrastructure as a service. So there needs to be a new class of infrastructure as a service offerings that come to the market that are serverless first. And we're seeing that. Databases, caches, all types of stuff. When people go to market these days, their initial landing page is going to say 'serverless' on it somewhere if they're offering infrastructure as a service. And that's so cool. That's what it's all about. More of these are the greatest building blocks of all time. Let's get as much of this as...
These services, and you have to figure out how to compose them together, develop them. What does that experience look like locally, on the cloud? How do you deploy them? How do you test them? How do you scale the project? How do you organize the project so they can scale? How do you observe them? How do you debug them? It's a new type of architecture. And looking at it now, I think that the workflow is still fairly problematic, which is a barrier to entry for a lot of folks. And I don't feel like we've done a good enough job there. The DX is still good, but now there's just a lot of serverless services. Lambda itself is pretty complex. There are a lot of knobs and levers alone just on Lambda. And you have a lot of different serverless services that I didn't expect would come out that would be part of...
New serverless services. And some of these new vendors want to bake in the workflow into their products. They want to own the whole workflow too because it's very compelling if you could offer workflow and the infrastructure as a service together. But in many ways, that results in a more fragmented development story. And I see a lot of fragmentation right now, more than ever. I see it in the serverless space. I see it in the developer tooling space in general. Just a ton of complexity, a ton of fragmentation. And then we still have some challenges. We still have some valid complaints around lock-in, which I feel like we could improve some things there.
Today, we've got these cost concerns. And a lot of reasons for those, but I'll say the Serverless Framework, we get a lot of requests from users about reducing costs at this point. And is that because they just have so many serverless services and they're in this kind of layered scenario of costs that they didn't really anticipate in the early days? A lot of these people have really mature workloads that have super high volume, consistent high volumes. And they're like, okay, well, it feels like we could run this more efficiently now potentially. And then it's a new economy, higher interest rates. We were talking about that earlier before we started. But that changes everything. Suddenly, controlling cost is more important than ever. And there's just been a lot of...
Ways to go. Some people in the serverless community are saying it's a win. Serverless has just become cloud. This is how it's always meant to be. But I don't know. I still feel like we have a long ways to go here. And frankly, if a new developer is starting, I'm not even sure the best place to point them towards in general to kind of get going in this new ecosystem. So a lot of the changes that we're going to talk about with the Serverless Framework are really motivated by us looking around and thinking, is this as far as we can go with this? Is there more that we could do here? Because I feel that there is. And I've been in this from the beginning because I just felt like there was...
I
Interviewer24:10
Friction in terms of someone trying to do serverless stuff, having no previous experience with using AWS or using serverless components. There's a really big mindset change they have to go through. A lot of the practices that they used to don't really quite work when you're doing serverless development. And probably even more than that is the fact that you really have to design your applications differently. And so taking a lot of the existing workloads to serverless usually means quite a bit of re-architecting, making certain decisions, but also adapting your practices as well. Things like over-reliance on running...
What I've been doing is developing all of these practices that kind of merge local development with remote testing. And so they're coming out with a set of practices and processes that makes it easy for you to develop, but at the same time taking advantage of the fact that with Lambda and all of these serverless services, you can deploy lots of them and not pay anything because you only pay for them when they run. So you can have a temporary environment. And that's one of the probably most impactful practices you can adopt when doing serverless development. The ability to just bring up a new environment when you're working on something, make changes there, test against that environment, and then tear it down when you are done.
When you said serverless is the cloud, so to use the cloud you have to learn five, maybe ten services just so that you know what to do for API, what to do for pop-ups, what to do for building event-driven architectures. So all of these things means you have to learn a service for a particular kind of architecture. Which means certainly, looking at a lot of customers, they have to take on a lot of new things in one go. Well, with a lot of enterprises, it's not a fact that they have to learn how to use Lambda, but it's a fact that they have to understand how AWS works, how cloud security and the shared responsibility model works. They need to understand how...
By running all of the server components that were self-hosting all of this stuff themselves. So on the one hand, yes, once you understand how all of that works, it's a really powerful thing. I mean, nowadays I can take on client work, I can do everything myself in a fraction of the time that used to take an entire team of people to do. So in terms of being able to get a lot more done with fewer people and fewer resources, it's absolutely amazing what you can do. But the premise is that you have to have people who know what they're doing, who know how serverless stuff works and how to design and build serverless architectures. And getting there is not easy. And also, you know...
That's going to cost you an arm and a leg compared to using ALB and not having the right caching in place also means that your Lambda function may just be running a lot, and that can be a problem. And the same goes to pretty much everything. Once you have to think about scale and cost efficiency, there are a lot of things where you just think, okay, that's a lot of places where people can shoot themselves in the foot. So a lot of what I've been doing the last couple of years has just been focusing on teaching people about those caveats. And it's not easy to talk about because there's no one answer. You can never give just one answer. It's always 'if this, then that', and then you have to factor in all of these different contexts people have. And that's kind of what...
So it's a really fast-changing space, and it's fun, but also at the same time, it can be exhausting. And if it's difficult for me, who basically does this full-time, it's just going to be really difficult and impossible for people who have a full-time job doing something else and building their products as well.
A
Austen Collins29:35
Oh yeah, yeah. Complexity. We're facing growing complexity in this movement in general. And I think it's an opportunity though for developer tools. That's an opportunity for us. And what's motivating these changes is, you know, we've got new challenges. There's new challenges in the...
Brought us into making some big changes with V4 effectively. So if I were to kind of bucket those changes, it's basically a new model. But it's a new model for us as a company, but it's also a new model for infrastructure as code. We've learned a lot about building infrastructure as code projects over the years, and so we're going to explore some new models with that as well. But starting with us, transparently, we're actually a small company. We haven't raised incredible amounts of cash like a lot of other companies in the...
Of profitable company. And one of the things that's always been really hard for us with the Serverless Framework is where the incentives are. The incentives previously have been in, well, we give away this open source framework and we try and monetize elsewhere. So basically, you kind of find product-market fit on something and then you have to go find product-market fit on something else and commercialize that. And I think that's just been a big challenge for kind of open source companies in general. It's not exclusive to us. But I think it just goes back to incentives. At the end of the day, if your incentives aren't around the big...
Building out infrastructure as code products in particular requires an ungodly amount of energy and resources. It is an incredible amount of work. You have to burn a ton of calories to keep that thing going. And talk about that, but for us, it's pretty clear we just have to fix the incentives. What are we doing? What do we really believe in? And how do we just make sure that we're focusing on that the whole time? And the answer is the Serverless Framework. That's what we started with. That's what we want to continue on. And it's what we want to use to approach a lot of these new challenges. So we are doing some changes in the upcoming version of the Serverless Framework. These will...
Startup. If you haven't hit that mark, you don't have to pay anything. It's very similar to what Docker has done in a lot of ways. And that we're charging based on instances. Basically, for the Serverless Framework, that's kind of synonymous with CloudFormation stacks and a kind of grouping of infrastructure. If you look at how a lot of the infrastructure as code companies are pricing these days, they might be selling a dashboard, they might be selling via users, but a lot of them increasingly are selling around resources managed by the infrastructure as code project. And we looked at that and it's kind of confusing. It's like, it's resources managed by the project per hour. So you have to figure out, okay, how many resources is this project managing?
Number one, these projects take a lot of work and effort to maintain reliability. And for infrastructure as code in particular, you really want that thing to allow you to continue to deploy over time and improve quality all across the board, but then innovate like never before. And these changes are definitely different for us, but I'll tell you already we have more people on the Serverless Framework than we've had in years. We've got more massive roadmap items that we've had in years. We've got a whole bunch of quality fixes, including just a nice support offering that comes built in and much better reliability across the board. And it just feels like everything frankly...
About IAC models. And again, how much resources it takes to keep these things going. And it's not just about getting started because it's very easy to say in the beginning, 'Oh, we figured it all out. We've got a self-sustaining open source community and this is the way it's going to work forever.' It really comes down to over time, how do you sustain a project like that over time when there's new vendors, there's new services, and the configuration is expanding and changing across all those? It's a massive surface area. In fact, I think it seems like it's been really hard to do infrastructure as code without corporate backing. I can't think of any big infrastructure as code...
Just really hard, especially if you want to sustain that over time. And even with the corporate-backed projects, there's a lot of challenges there. Looking around the landscape today, there's sort of this Russian doll effect with infrastructure as code projects where it seems like Terraform has some Ansible components, and Pulumi has parts of Terraform in it. And then a bunch of startups are being built on CDK, which is built on CloudFormation. So we're kind of all of a sudden reusing each other's stuff a lot. And then all of a sudden you throw in this new trend of anti-competitive licenses into that, and that feels like, oh, this all of a sudden is really fragile. And then you have...
Since we started with CloudFormation in 2015, there have been so many things that we've always wanted it to do that we've never seen. And I get it, it's hard to do CloudFormation at its scale and innovate on top of that. And then of course, it's only AWS focused, right? So some challenges there. Okay, great, CloudFormation is reliable, but you have to be mostly all in on AWS. And that was never my personal hope. As a developer, I feel strongly that developers have to be able to choose the best of breed services to build the best products. AWS is very competent, but still you're going to want to use Stripe or the most recent example, I mean, we'll see. There's a lot of drama around this company as we...
Product, I think, is really important. And then I've also noticed that AWS has actually spread itself really thin. So they're not just doing one IAC, they've got a few takes on it. So it's interesting. It's like, well, they have a big company that has infinite resources, but it changes when they do a lot of the different variations of the same thing. Because if you do a lot of variations of infrastructure code in particular, it again takes ungodly amounts of energy and resources to keep IAC projects going. And then if you spread yourself thin across CloudFormation, SAM, CDK, and CDK across multiple languages and whatnot, there's a lot there to maintain over time. So it's been hard to see how to do IAC self-sustaining.
Ecosystem. We have over a thousand plugins that come with the Serverless Framework that allow you to do all types of new use cases or add new workflow capabilities to the framework. But those plugins aren't always maintained. And so it's hard for people to figure out what's interesting or what's working still today. And again, for a lot of us, it was like we had a core incentive problem, which we're switching here. But also I think that there's probably enough evidence here to say like we should maybe explore new ways to do infrastructure as code and ways to sustain this. And that's also a big...
Is, you know, we'll see. It's new, it's ambitious, but if it works and it takes off, I think it'll have a broad effect on the infrastructure as code space potentially.
I
Interviewer40:24
Yeah, I guess one of the things you mentioned is just the fact that AWS has got so many different infrastructure as code tools. And one of the things I think Ben Kehoe keeps talking about is that over the last couple of years, well many years, AWS has been very focused on fixing problems on the client side, using CDK or SAM, as opposed to addressing some of the root issues at the CloudFormation level.
That's something that people have been waiting for for years now. Are you talking about the recent change like this week? Yeah, this week. Yeah, the new feature to make it easier for you to import resources into a stack without having to create some manual JSON manifest. So that's been a really welcome change and honestly probably one of the only meaningful updates in CloudFormation besides support for new services. And I think they are getting better now to release new services with CloudFormation support after people like Ben Kehoe just been shaming them for years. He's good at that.
Yeah, so I definitely see that they've actually switched focus back into SAM and have got some new people into the team, and they've been releasing quite a few new features recently. And we also have, like I said, this layers of tools that build on each other. At the base level, you've got CloudFormation and Terraform, and then you've got tools that build on top of that. And you've got things like SST which is built on top of CDK, which is built on top of CloudFormation. So you've got all these different tools that build on top of each other. But at the very bottom layer, we really need to see more things being done to CloudFormation to improve it so that we have core features...
Team. It's great that you guys are building some of these toolings, but then it's only going to benefit people that are using SAM. It's not going to benefit anyone who's using the Serverless Framework or using CDK or some other tool. And even with things like CDK and SAM, I still think that for people that are focused more on the backend side of things or DevOps engineers, they're probably good enough. But there's a whole world of frontend developers who are not that familiar with AWS. They don't want to have to learn all of these nitty-gritty stuff. They just want to have something that's easy that gives them the benefit of using serverless technologies without having to learn all the nitty-gritty of how Lambda works and all of that. And that's where things have a sell. And then, you know...
Interesting to see that whole space. And I guess that's where I'm interested to see what Jeremy is going to do with Ampt and what they're going to do in terms of abstracting away the underlying complexities so that more people can get into this space and get the benefit of serverless technologies without having to understand all the minute details that you have to learn and understand before, otherwise you're going to shoot yourself in the foot. But yeah, so I'm really interested to hear about these new extensions and whether and how it's different from plugins. I understand the revenue share model and I really do hope that addresses in some way the...
Relying on people to put in free labor to maintain projects that we use for commercial reasons. I really do think that the whole model needs to change. You can't just rely on people to volunteer and put in the labor for free when you are benefiting commercially and financially. So yeah, tell us about this extensions. How is it different from plugins? And also, how does the revenue share model work? How does it count towards, you know, which part of how much of the revenue from someone paying for using the Serverless Framework goes...
A
Austen Collins46:10
Open source is broken or anything. I mean, who am I? That doesn't make any sense. Open source is obviously one of the greatest, most virtuous things that humanity has done. Coordinating people at such a massive scale on a voluntary basis. What are the analogies for that? This is totally unprecedented and incredible. And if there's going to be a highlight reel of humanity at the very end of things, open source is going to be there on that somewhere. It's an incredible model. But the burnout, the fatigue is real. And it's especially real when...
Importantly, after the first year, I'd say 95% — this might be a rule for open source, I don't have any data on this, but it's all observational — but it seems like 95% after the first year, the innovation is kind of done. There's just fatigue. It's like this cake is baked. We'll just do bug fixes and security patches and stuff. And it's tough because you're potentially picking up a lot of broken stuff. We've had this in the Serverless Framework with the plugin model for a long time. So we're trying to change that, figure out a new model here. But the difference here is extensions are designed around adding new use cases, new capabilities to the...
Isn't support already. And they could be anything, frankly. We'd like to think that based on our community and what they like, it'll be built on serverless services. But anyone could build one of these. And because we're charging for the framework, which is a tough transition, it does unlock new possibilities. And so we thought, hey, what if we explored this creator economy model and we shared 80% of the revenue of people using those extensions with the author? And so who would this be for? Well, indie hackers looking for passive income. When I go on Twitter, especially my Twitter feed, probably your Twitter feed, it's a bunch of these super scrappy, cool AWS hackers like...
Just this huge community around people who, developers who build these micro developer experiences. And I think if there was a way that people could maybe get some passive income from that without having to do things like raise awareness, all the hard parts of getting a new product to market, raising awareness, building a community around it, and all the marketing that has to go into it. So the Serverless Framework has incredible distribution. If you, it's...
To go to market with this stuff and access the Serverless Framework's distribution. So we're probably the biggest base of AWS Lambda users and serverless users outside of AWS itself. It's a very qualified group of people who spend real money on projects. And so to be able to tap into them easily, and we'd love to promote this stuff. I mean, this model is only going to work if we're doing this together with the creators and whatnot. So to have instant access to the Serverless Framework community, plug it in, and just focus on building the product. Let us help with the promotion. We'll make discovery really easy, and you just focus on building the best possible micro-DX for whatever it is, provisioning some type of use case typically. So for them, I think it's...
Around that, which has been an amazing and awesome journey. But I think there should be a second option for folks, especially in this new world, this new economy. Like, hey, just go to market really easily with your DX. And also for independent software vendors, if they have a new serverless infrastructure offering, it would be so fantastic just to tap into the Serverless Framework's community and just offer an easy deployment, infrastructure code scenario within that community. I think it's just great for them. And consulting firms who are building these repeatable use cases over and over again for their clients, for them to be able to offer that to clients and then kind of open it up to the broader community to use and maybe make some passive income from that, I think it's really interesting. Now, this is early and it's highly...
Good. And super interesting there. So it's a new take, hopefully better quality, more sustainability over time, and it really helps us build community in a way we haven't done before. And I will say we got inspired by this a lot from Terraform and the recent license changes. We were talking a bit about the anti-competitive license changes in the industry right now. And I knew when Terraform came out with that change that we were going to make this transition to having fees with the new version of the Serverless Framework. And I was looking at what Terraform is...
That we could actually have a strategy of being more inclusive. Like, what could we do? Yes, we have to charge, but what opportunities does this unlock? And that Terraform experience really got me thinking around, hey, let's try and design something where we're not shutting anyone out. We're trying to get everyone to participate. And let's see if we could build something bigger than ever with this approach that we could all kind of share in the success of. And so that's kind of the high-level goals of extensions. At the low level though, they're just containers actually. So you build an extension in a container, which is different from a lot of the approaches we've taken in the past because we've actually tried to approach new ways of building...
Developer experiences. We've been at this for a long time. We had a lot of opinions. This is the first time though that we've had no opinion, frankly. And that comes in the form of containers. So it does not matter what vendor you want your extension to deploy to. You could use any vendor. You could use any language. This could be written in anything. And you could also use any existing framework. If you just want to build something that's already used as CloudFormation or CDK or something else, you could just bring that into your container. At the end of the day, extensions will be things that people add to their serverless.yml and they'll be able to use them programmatically as well. And we abstract spinning up these containers and calling them and passing data into them...
Into a container pretty easily as well. And so it's a very — this is what I was hoping to show off today, but hopefully we get a chance to dig into the demo in just a couple weeks. I think we're a couple weeks away from bringing on the author. But it's a very interesting take on all this that offers total flexibility, plus the benefits of isolation with containers, better consistency, a better security model than I think we've had before. And ultimately gets us back to that vision of a universal experience for deploying managed services, serverless services across vendors in a uniform way. Any use case, anything is possible. Let's be as inclusive as we can regardless of language and vendor and all that stuff.
Today, we've got well over 100 extensions. We haven't barely talked about them. We kind of put a little snippet in that blog post, but we have over 100 extensions registered already and they look awesome. I look at the list and I think if just two of these got built, it would be such a game changer. So many of these things that people want, our users want, that we can't get to. But if just two of those on the list got built, it would be such a dramatic improvement for people using the Serverless Framework.
I
Interviewer56:39
Is the list of extensions already publicly available so that anyone can go and have a look and see what extensions are going to be there?
A
Austen Collins56:51
Not yet. Not yet. Okay, working on it. Our first goal is...
We're using containers. We want to abstract that as much as possible. People shouldn't have to think about the container as much. And so we've really built this nice experience over that. And so we're wrapping that up and we're sending that out. So we owe people to register extensions. So in the blog post for V4, we said you could register in our initial registry right now and get a namespace. And so we have a list privately in our database of people who signed up to build things. So that's what we've been looking at. But we'll open that up. I think we're going to send out the authoring kit to folks, make that public in a couple weeks, and then we'll make the...
I
Interviewer58:10
Are similar to the existing plugin system? There's going to be a lifecycle in terms of when the framework is going to invoke your extension. And your extension can do whatever you want it to do in your code. But can it modify the CloudFormation template that's been generated so you can do some of the things that the current plugin system can do? Would that be the case?
A
Austen Collins58:33
We're working on how to modify the existing CloudFormation template. Right now, that's not going to be offered just yet. It is, you know, creating another CloudFormation stack within your extension, for example, would be a simple way to kind of get around that. But a whole bunch of data, ARNs, things like that will be easily passed in from the framework so that you can reference a lot of things on your account easily and use...
I
Interviewer59:11
One of the common use cases for plugins is to kind of act almost like a macro. But then the lighter way, you don't have to deploy to CloudFormation to do that. You can just have your own plugin that modifies your CloudFormation template locally before you send it off to CloudFormation. And there are some really clever things you can do. I've used so many plugins in the past that do things that make life so much easier. The one that I use pretty much always is the serverless-layers plugin, which packages my deployment, looks at my dependencies, sees if they change. If not, then if they change, then they create a new layer version and update all my functions in the CloudFormation template.
A
Austen Collins1:10:11
The way you write your code. We're not going to be able to solve all those problems, but we're going to chip away at a lot of them, I think. And it's partly enabled by changes in the underlying infrastructure as a service. These things are evolving, and it looks like this is now more possible than ever as a result. And I think if we solve that, we'll have done something hopefully pretty meaningful for the next chapter of serverless.
I
Interviewer1:10:38
So I guess to kind of wrap up on the V4 changes overall, yeah, we're changing a lot of things. But the themes, we believe strongly that there are new challenges that we need to solve. It's going to take a lot of work to do that, and we've got some pretty killer solutions on the table.
A
Austen Collins1:11:11
This exploration of what does the serverless architecture version 2 look like. How do we enable this world where you could start, never pay for idle on Lambda and whatnot, and get to market really quickly, but then have the flexibility built in to optimize for cost performance or whatnot later down the road? And then lastly, let's revisit infrastructure as code overall and let's explore something new here. Let's see if we could build a more sustainable community, a better end product for end users over time with actually sharing some of this revenue with creators. So lots of exciting stuff upcoming, and you'll see us probably ship a beta in December of V4. It'll be early and...
I
Interviewer1:12:11
What a great opportunity to be able to work on stuff like this and give people these great tools and the empowerment that comes with it. It's very cool. And then you throw AI into the mix. And sometimes when you're growing up, it kind of feels like there are so many people in the world doing so many things. How do I ever stand a chance in making a dent in the universe and doing something meaningful? It's like, well, I think the answer is in serverless services and these incredible building blocks. And then you add in AI on top of that. And yeah, the world is filled with interesting people doing a lot of different things, but I think the power that individuals will have to actually...
Scrappy guys on Twitter talking about how to build all these wonderful things just on their own. That kind of thing is unthinkable maybe going back 5 or 10 years, but nowadays with the tools and technologies available to us, anyone can do it.
A
Austen Collins1:13:29
Really, that's what it's all about. That's what gets me out of bed in the morning after all this time. It could be quite a grind, as you know.
I
Interviewer1:13:37
Yeah, yeah. So we are just over time already, but Austin, thank you so much for taking the time to speak to us today. And yeah, looking forward to the beta launch. And hopefully by the time you listen to this or watch this episode, it's already available so you can go and sign up and try out...
A
Austen Collins1:14:10
Your support. Take care. Goodbye everyone.