AI Era Playbook for Developers (Guillermo Rauch in Klemen Selakovič) — RE:moat podkast 42

Well, first let me back up the statement, because I think one thing that happens with Silicon Valley CEOs is that you can say anything and everyone gets a free pass because AI. And I was recently looking at our own data of the evolution of Vercel. For those that don't know, I started Vercel in November of 2015. So last year we celebrated the 10-year anniversary, and almost at the perfect time that we celebrated the 10-year anniversary, we calculated that we've done a billion deployments. A billion is a huge number of deployments, especially considering that Vercel makes everything that you deploy routable. We host, we build, we route, you can time travel. If you guys have used the Vercel platform, you know these benefits. But then since November last year till now, in about 10 months, we added another about 1.4 billion deployments.

And even this morning, I woke up really, really early and I was asking my agent, because the way that I get this data in itself is really interesting. We have an internal agent called V that can answer anything about the company, and I was thinking about this fireside today or this panel, and I started asking some questions. For example, the agent was telling me that there's no signs of this acceleration stopping or slowing down, if anything it's continuing, or that about 90% of traffic to Vercel is originating or basically targeting deployments made by agents. So basically the agents have really taken over and also have a life of their own. And as you guys have seen, right, like we just design, they can do certain things at this point, they can do almost every task that a human engineer could do. And so if this pace of acceleration continues, I think we're going to see more software being created next year than possibly in all human history.

Now, is it a good thing for developers? I think absolutely, right? One of the other things that I do looking at Vercel and all of our agentic usage is I try to understand how much of the work that we do day-to-day has been fully replaced by agents. Meaning, do we have humans steering this ship or do we have agents reaching full autonomy? And the answer is we have humans steering these agents. In fact, the word that you guys do see in harnesses these days is steering. We're always steering these agents. And we're still putting a lot of human effort into making sure that these products are amazing, that the infrastructure is sound. So for as long as that's true, I think it's going to be a panacea for engineers. Also, I caught a little bit of the previous session and we were talking about skills. Clearly, different people get more out of agents than others. And so, it's just like all software engineering, some of us produce less or more code than others. Now, with the addition of skills and how you leverage the harnesses, some engineers will get more out of agents and those engineers will be at an advantage.

Yes, I can. I think there's quite a few things, and I think this is a great question because it's almost like the new curriculum, right? If Astra was to grade me on how good are you at agentic engineering, we all have to put our effort into thinking what are all of the different dimensions that make you really good at agentic engineering. So one that immediately comes to mind that I've mentioned a few times is what is your ability to parallelize work. I do think that back in the day there was a certain class of engineering mentality that it was like you go really deep into a problem. I can almost equate it to when you see the agent reasoning, right, and the agent is hyperfocused and going back and forth. Well, no, the evidence is this and I have this gate and blah blah blah. Like all of that we used to do as our own brain monologue, but we could only do it on one task. You know, no matter what you think about AI, the reality is it can scale way better than us. It can parallelize and multi-thread way better than us. And so I think what makes you really good as an engineer has definitely shifted. I think can you multitask more? Can you context switch? Can you have a bunch of agent threads, whether it's investigations or debugging or trying out different hypotheses or working on different features across git worktrees? There's definitely can you manage multiple agents at once. And you know, some people have used the cheesy metaphor of like you're now an engineering manager, you're not an engineer, but I do think there is something to that, right? The other one is how well, and I truly believe this is, how well do you understand the model? Are you getting the most that you can out of this incredible wonder of, you know, it's part database and it's part reasoning engine and it's also someone that can critique your work. And so I do think there's still a lot of value to the way you prompt it. So a good example here would be, you know, sometimes you might infuse into the prompt things that are just wrong or you might be too opinionated and perhaps a certain model can actually offer you a bunch of alternatives. And so I think whether you spend the right amount of your time planning with the model and hearing the model out will, I think, make a difference between the best developers and the ones that are not getting so much out of the model. And the other one I think that's really exciting right now is, okay, assuming that models are amazing at like 90% of the task fully autonomous, are you able, and by you I mean your company or you as individual developers, are you able to actually leverage that autonomy? A really good example of this, if you go and act, is Sahil, the creator of Gumroad. He's created this little agent that does a ton of his work, responds to every support ticket, and the support tickets that actually are pointing out product defects, because like anyone that's shipped a product at scale knows that you have to give support for your product and you get tickets and you can answer those tickets with AI, but some of those tickets are actually pointing out that your product is broken or that there's a design flaw in your product or that you have to improve the documentation. And so have you been able to deploy what we can call this like a software factory where a signal from the universe comes in and you're able to turn that pretty autonomously into product improvement? I think the engineers that can do that today, you know, Elon calls this the factory is the product, because in order to produce great Tesla cars, it was not about producing one. It was about scaling up the production line of these vehicles, right? And so are you able to create that factory for yourself and be really productive and obviously ship high quality products?

Yes. I'll tell you about a category of things. So I actually tweeted about this a couple days ago. So like I think the intensity and focus in the pursuit of an objective always astonishes me. So a couple days ago the example was I pointed out to this agent that the, and this I think a lot of you guys will resonate with this, where there's just a long tail of bugs and you're like, I could solve, I have the intellectual capacity to solve it. I just don't have the energy. And so an example here was my website was rendering incorrectly only in in-app browsers inside a mobile screen and just the effort to reproduce and launch the iOS simulator and do all of the measurements, all of the class measurements, like you know that the iOS viewport is dynamic. So as you scroll it gets larger and so when you position something you have to be aware and now there's selectors that make this easier like dvh viewport units whatever but like it's still a nightmare. And so that entire class of software engineering problem which is like we can all kind of intellectually solve it but the energy required is just astonishing and also when you do fix it the product is absolutely higher quality. I love throwing agents at those problems. Like we can call it sort of like quality assurance and testing and whatever. Models have been getting so much better at vision. Sometimes I think vision can be slow. So I'll give you a good example of a good agentic engineer sometimes knows how to prompt and create verification loops that are highly efficient. And so it's not just whether a model can do something sometimes but it's how fast can you make the model do something. And so as an aside, answering the previous question, the other one, so we've been spending a lot of time looking into cybersecurity and spending tokens and money and research into cybersecurity and the same principle applies. When Kimi K3 came out, you know, we started evaluating it against our cybersecurity benchmarks and what we saw was that Kimi is just absolutely relentless in trying to exploit an objective that you give it. And it kind of scared us to be honest because you know the modern models and harnesses from the big labs in the US kind of hide a lot of their thinking tokens and reasoning tokens and they refuse a lot of prompts. So to see the dedication, so there's this tale of two cities. You can have a model that's like ultra dedicated in fixing these trivial rendering bugs, but the same model can be ultra dedicated in hacking you. Now, for every sort of bad premonition about AI, there's also upsides like you can deploy those models to protect your codebases. And so these models will use their immense resourcefulness in trying to find bugs in your codebase and they're amazing at it. So at Vercel, we have an internal agent as part of our factory called Hacker. Hacker is doing this but defensively all day and all night trying to find people sneaking in backdoors into our open source software supply chain or trying to analyze whether the code has any edge cases that the engineer or the engineer's agent didn't think about. So everything around cybersecurity has impressed me so much. I mean so much so that if you guys haven't tried it yet, I highly encourage you to run DeepSeek, which is our cybersecurity harness. Run it on your codebases and it's very likely that it's going to find a lot of stuff. You know, there was this meme that went viral on X about like coding is solved, bugs are not, which is kind of a silly statement. Coding is solved, bugs are not. There's a truth to this in that if you ask some of these modern models to find bugs, they will find them and then you'll fix them and then you ask again and they will find more and then they'll find more. So pay attention to the cybersecurity category of bugs because I think it's an open liability for the world and it's something that, you know, it's a good use of your tokens.

Yeah, I have a channel on Slack. Sometimes people ask me how I run the company. So I'll give you a little bit of inside baseball. So I have a channel on Slack called product area updates and every week I get a report from every Monday from every product area. This is what allows me to scale myself horizontally by the way, like we have a lot of product areas, right? We have AI and we have labs and we have CDN and we have compute. So I get these updates, right? And yesterday I was looking at our data infrastructure team. So this is the team that maintains our data lake. It incidentally maintains the agent that I just told you about, the one where I can ask any business intelligence question like what percentage of traffic to the Vercel global CDN goes to deployments created by agents. So that team was giving me an update. They're canceling five SaaS subscriptions. I learned about this yesterday. Five. And what I look for as well is I'm not naive, right? Like I actually tell you guys like we all love to create new zero to one things with AI. And there is a downside to that. AI is so good for like that initial, it's a mix of like an IKEA effect. If you've heard about the IKEA effect, people feel better about assembling something themselves than the easier route. So this is something an effect to correct for, like it's not always good to create things yourselves. And there's also that no software engineering project is just zero to one. The hardest part of engineering has always been one to 100, keeping it fast, keeping it secure, keeping it scaled. So what I did is like I started poking holes and seeing, okay, is it right that we're converting these four SaaS into things that we've rolled ourselves on top of ourselves? And the answer actually was overwhelmingly yes. For example, two of these tools were doing the same thing but for different personas. So one of them was like what you would call a no-code data transformation pipeline. And so that kind of software, no-code as the name implies, emerged for a time when code was scarce and difficult to produce and difficult to understand. We're past that time, right? Code is basically free now. And so we've deleted a lot of these no-code things. By the way, we've also seen that with our customers, right? There were a lot of no-code things for building websites and applications and we've seen tremendous migrations from those solutions to Vercel. A lot of website builders, a lot of app builders, people are just using coding agents, Next.js, some framework, whatever for Vercel. So obviously you're also asking someone that has a lot of skin in the game. I love that this is happening because you know I was literally telling 20 CIOs the other day, Vercel is the last platform you guys need to buy because if you buy Vercel and you govern it and make it secure then you don't need to procure more software and by the way from a security and governance standpoint that's it, it really is freaking awesome because what happens, what happened to, and the reason that the CIO, just chief information officer or chief information security officer, is that enterprises were buying and buying and buying and buying software and they needed to figure out 10, 20, 30 different ways of managing permissions, directory sync and all this stuff and so actually companies are loving the fact that they're strengthening their security posture as they generate more software. Now you know there's probably classes of software that you shouldn't generate and what I hear from engineers a lot is like, okay, does that mean that no one will ever buy software ever again? I really think that if your software is more on the infrastructure side or is providing a platform that works really well with these agents, meaning that you expose an MCP or an API and that agents love to use, then your software, your infrastructure will continue to be prolific within these companies.

Yeah. You know, I posted a thing that went viral this weekend about like 80% of our token traffic is now open models. There were a lot of responses to that that were somewhat like cope or defensive of the AI labs. By the way, the AI labs are doing great. Like all of AI is doing great. Astra, OpenAI, Astra, Anthropic, Vercel, like everyone that's in the token economy, I think, is doing well. Like I think we can't, no one can really complain. If you're creating value and you're disseminating intelligence and you're making things that before took lots of teams of people and like now you're making them easier, you're probably doing great. But I'll say that people were saying, well, yeah, because open models use more tokens or open models do this or do that or but the spend. Well, no. The reality guys is when we started the AI Gateway, it was all OpenAI and then a lot of it was Gemini and then a lot of it was Anthropic and those three have consistently been losing ground to open models. And so that doesn't mean that I'm bearish on them by the way. Like I would actually like if I were to buy stocks and this is not financial advice, it's like anyone that's in the AI world at the different layers, like I would have them in a basket of like AI goods. I love AI. AI, you know, is again is doing amazing. So, but I'll tell you open models and faster models seem to be the new trend. The excitement about Jev points out that people need high quality, fast decision making and they weren't getting that from the big models. And so, a lot of people are saying, okay, get it. I'm using AI but I'm optimizing. Now another thing that AI Gateway is offering companies is just the feedback loop of like do you know how much of your, how much your engineers are spending every day, like how much you're personally spending, do you have budgets in place when you issue an API key, are you putting a budget on it? And so AI Gateway is doing this like cost governance task that I think is really important because it's really easy to just set your coding harness to Fable 5.1 extra high fast and forget about it. And so what the data is showing us is that people are moving past that sort of token maxing, use AI at all cost mentality and they're starting to optimize into, you know, combinations of models. The other thing that the data shows us is that when a team adopts AI they always end up adopting multiple models. I think the average enterprise is like over five. And enterprises are now getting really savvy about evals. This is also inside baseball about Vercel is we're almost like an eval first company now. When we develop Next.js we start with evals which is kind of nuts, like you used to start developing a framework by I don't know, like writing tests, writing software, maybe writing documentation, like different approach. No, it's all eval first now because we care about how the different models and agents use our products, right? When we design APIs, we're eval first. And so I think the world has moved into this eval first like sort of or is moving towards that and that's going to help make more intelligent decisions about models. And right now you kind of have to navigate this fog of war of like vibes. A lot of us are still deciding on models based on vibes. Oh, I saw on Twitter that like Astra is better. It's like is it, like some of our Astra came out and they moved back to Soul for certain things. So I think the TLDR is that we're seeing tremendous model diversity.