AI Without Borders Ep. 18 | The Self-Driving Agentic Infrastructure with Vercel CEO Guillermo Rauch

AI support agent now handles 82% of tickets. When Karpathy tweeted that, my main advice to Vercel is the tools that you thought were the right tools for AI even just a couple weeks ago might not be the ones. There is new bets that the company can afford to work on. You have to say I have this hypothesis for the future and once upon a time v0 was one of those bets. Not every bet is going to pan out. For a thousand bets that fail, I just need one winner. It's a bitter lesson for agents. If you're constraining your agent too much, you might think that you're too smart, like, well, I'm going to put this agent into this tiny little box and I'm going to create a whole universe around it where I decide other rules and whatnot. My cautionary tale is that there's only two business models left in the world of software.

Thanks for having me.

Yeah, this is a great question because I've always been quite outspoken that companies trust Vercel as the most important infrastructure of their business because every business, even 100-year-old, 200-year-old businesses or even governments, are becoming more and more so online entities. Just yesterday the US government announced the new food pyramid and it was delivered as a website built with Next.js which I'm really proud of and this is happening across the board. Every company, whether new or old, is becoming an internet company and increasingly these companies are becoming AI companies which is very, very exciting for us and the goal with Vercel and our self-driving infrastructure is to serve them. But because this is critical infrastructure, I've always been, as you know, super AI-pilled. I'm obsessed with AI. I use it all the time. But when it comes down to dealing with security, privacy, availability, durability, I've always been more analytical. Would I give an agent full access to all of my secrets and all of the customer data and all of that? And so I'll tell you what we've learned so far. One of the things that Vercel does and why we like to call it self-driving is that anytime something breaks in production, you start getting errors, even anomalies. And what I mean by an anomaly is suddenly you have more traffic, suddenly you have more load. Perhaps you weren't expecting it. Vercel is automatically detecting that. If you compare that to the world before Vercel, engineers would first write their software, host it somewhere and then if they were really disciplined, which is as you know developers, we are not always super buttoned up in everything, you had to remember to configure alerting and you had to handpick the alerting threshold because the reality is that software and perhaps the people that don't write production systems for a living don't know this. There's always some amount of noise in the system, a crash here and there that to you, the engineer, you decide, well, that's okay. That's acceptable. Maybe to give you an example that's super relatable. People all over the internet install the weirdest browser extensions and Chrome extensions. And sometimes those Chrome extensions break the software and the engineer gets notified, hey, this random extension to capture online coupons broke my banking website.com. And so the engineer makes a judgment call there and says that alert is not interesting to me. We do not support the coupon Chrome extension breaking our website. And so what you want is a very adaptive, intelligent and dynamic system that surfaces to you the really important insights from production. And so what Vercel agent does is for example, if you're using Slack or other systems that we can notify you through, it'll say, 'Hey, these are some errors or some anomalous activity that you should be paying attention to.' But it doesn't stop there with just alerting. It does an entire AI investigation. It correlates what just happened with a lot of other data points. And what's fascinating is this is already how we're running vercel.com. So one of the principles at Vercel is that we use our own product first. So you're asking me how did you gain conviction that AI was at a point where it could be trusted with such an important thing as monitoring our production systems. And the reason that we gained that trust is because we created AI evaluations which is sort of I call it is the exams that you give the AI in order to determine that it's good for a task. If you compare it to the domain of people whenever you hire someone you give them training and once they're trained maybe you certify them and then once they're certified you can say okay you can be on the factory floor performing this really mission critical duty. So AI companies we are doing that same thing for the agents that we bring to production and then Vercel itself is sort of the first customer of this capability and by the way this is just one of the ways in which Vercel is self-driving and there are many others but I wanted to highlight this example because as engineers we typically enjoy the act of building the thing but running the thing in production is where it gets not so fun. So for example, AWS had a massive outage recently in US East-1. And half the world of engineering had to be woken up in the middle of the night to tend to that. And so one of our perspectives is that it's the AI and it's the agents that should hold the pager, not humans.

One thing that I've seen recently that I'm really happy about is that AI is getting much better at knowing what it doesn't know. So the default behavior of AI in older models was hallucination. When it gets an error that has complete gibberish, it follows on because it's doing this next token prediction thing. So it goes on, well that must be the truth. Whereas recent models have becoming a lot more quote-unquote introspective. They can refuse things. And this is something that we work into the evaluations as well is when you don't know something, you should say you don't know. By the way, this is advice that I always give entrepreneurs, people, my children. Like the best people that you can work with are the people that say I don't know. Honestly, I don't know. I have to check. You don't want that co-worker that is like super confident all the time and just half of the time it's wrong. It's the worst. And so this is something that we pay a lot of attention to. The other thing that's really important is we're operating sub-agent architectures. So I think it's misguided to assume that there's going to be an agent that you can toss any problem to. At least within the constraints and limitations of the technology today. So the best way to think about the Vercel agent, which is sort of the brain of this self-driving infrastructure, is that it's a group of agents, experts in different things. So we're very carefully tailoring the behavior of the agent for when it's taking on the role of monitoring the production system, of parsing the logs, of understanding the telemetry and the instrumentation. The other thing that's really important to mention is we still support all of the quote-unquote traditional mechanisms by which you monitor the systems. We have Vercel drains which allows you to get the data and put it into the traditional monitoring systems. But increasingly we're seeing that AI outperforms the traditional monitoring systems because it requires so much configuration. So this is also what I always encourage people to think about is what is the theory of a system and what is the reality of a system. You always have to compare that what AI can do for you today versus what the more human-centric system is doing for you today. And so there is expert drivers that would never crash. But sadly there are a lot of drivers that drive under the influence. And so when we compare is Waymo or Tesla self-driving better than a human driver, we have to look at the actual results and the actual data. And so this is one of the things that has given me a ton of conviction in this sort of transition into a self-driving infrastructure world. The other benefit that Vercel has is that you know I started the company before AI was as big of a thing as it is today frankly and we already had a vision. We couldn't quite articulate it in the words self-driving infrastructure but I wanted the cloud to be automated away. My main frustration that led me to start Vercel was all of the configuration that I had. It was more so in the deployment phase. It was also like hey I need to launch my new idea to the cloud that also required insane amounts of configuration. So Vercel has always come from this perspective that the infrastructure has to be automatic. It has to be self-healing. It has to scale basically when more traffic it scales more, less traffic it scales less, those kinds of things.

Well, I'll tell you what our perspective is. So when these alerts fire, you get the AI investigation and our perspective, this is why we call them investigations, our perspective is that they can propose policy changes to a human operator. So this is also how we think you know we will disrupt the world of security because what we notice is web application firewalls are a cohesion or a collection of human written rules for what looks like malicious traffic. The entire security industry was built up on this idea of a firewall with an enumeration of rules. And it kind of reminds me of Hilbert's program versus the incompleteness theorem. I would argue that it's impossible to produce a collection of rules of all of bad traffic in the world. What's good traffic to you might be bad traffic to me. And this is once again where AI will vastly outperform human operators because what these anomalies are saying, I read one yesterday. I as a CEO enjoy reading these investigations because I'm bewildered by what AI can do. And it was this analysis of you're getting all of this traffic from a Tencent network in China. The agent is really good at analyzing the traffic logs for security reasons. And it was analyzing that the ASN which is sort of the origination band of the IPs that are accessing your website. Who issued these IPs? Well, those IPs came from Tencent in China. We don't want traffic on that particular endpoint, huge amounts of traffic coming from that particular network. It made no sense. And so I'm able to apply policy suggested by the AI in real time. There is a world where let's say that the human is not available. Going back to AI models becoming so good at this sort of like judgment call situations. There might be a world where look if I'm not getting a judgment call made by a human it might be so important that I make a decision for that traffic that further automation is deployed. Right now we're very much in the world of an investigation produces a policy proposal or a pull request, right? Which is very important as well. So when something is defective, when something is wrong in the code, we can open a pull request and say we believe that this you're missing this check. For the engineers that are listening, something that happens a lot is you get a 500 error, which is, you know, a bad error, like it makes your stomach sink a bit, but sometimes they're harmless. It's because you didn't handle a certain kind of input and someone created a malformed request. And what I mean by this is that imagine you create a form that says first name and last name. And in first name, some people, teenagers do this, but also bots do this. You try to game it and you paste 5,000 emojis into the first name and so your code didn't expect that and it crashes or it throws an exception and whatever. AIs are becoming really good at saying that 500 error you just got that is not as concerning that is just some malformed input is someone trying to mess with the system I can open a pull request and fix that for you dog you just accept it. So that's kind of where we see the world evolving where it's self-healing because the AI has really good judgment it has a lot of really good context that Vercel is giving it but it keeps you firmly in the loop in making these important decisions to your systems.

As everything is changing, there's infrastructure that you've built where I feel like we've gotten lucky or maybe it's the skill, who knows but for example when we started Vercel I was obsessed with making it the easiest to use command line interface on the planet for deploying software if you have been with us for the long journey you might remember we initially called our CLI Now because I'm a very impatient person so I would write some code and I just want to press a button called Now and put it in the cloud. Turns out coding agents are computer use agents. Claude Code, Codex, OpenCode, the agents that people are using to write software also know how to run programs. Turns out this is one of the situations being in the right time in the right place. The Vercel command line interface is awesome for agents because they, so I mentioned on Twitter the other day during Christmas week and New Year's week we had record usage of the platform and the reason for that is v0 but it's also that every other coding agent in the world when you write your holiday hack and everyone's been vibe coding when you tell it to deploy they like to deploy to Vercel it's very easy to use for the agent and so we're going into a world where agent ergonomics and agent accessibility is a priority for your business. If you're a developer tools and infrastructure company, agent ergonomics means better command line interface. It means more accessible documentation in the places where AI expects to find it. It means that you design the APIs in a certain way. It makes you mindful about error messages not just for humans but error messages for agents and things like this on the infrastructure side. I also think that change is magnificent. One of the things that has been becoming also obvious as of the last couple weeks and months is the rise of the sandbox. So agents can make a lot of mistakes. They can delete files. They could be hacked. They could be coerced. It could be you know they can get prompt injection attacks etc. So we needed to develop a capability so that we could sandbox them. To sandbox them means to make them give them the illusion that they can do anything but help the developer basically create a security boundary. And so within the sandbox the agent feels it can do anything. Sky's the limit. But then the developer from the outside world can communicate with that sandbox and extract value from it. And so this is one of the new primitives that Vercel has been investing very heavily into, Vercel Sandbox. It works with any agent and it allows companies that are creating agentic experiences to give the LLMs this ability to use computers because what we've learned, you were asking things that I would have done differently when we designed v0 originally we didn't give it the full power of a computer. We gave them the power of Next.js with some constraints running within the browser knowing where the, if I had had the perfect crystal ball or the perfect magic ball, agents were headed in a direction where they were getting really good at using all of these operating system tools, these command line interfaces they can test, build, QA, search the web etc so you have to be kind of simultaneously fearless and fearful. Fearless in the sense that the more you give your agent the better results you're going to get. It's a bitter lesson for agents. If you're constraining your agent too much, you may think that you're too smart, like, well, I'm going to put this agent into this tiny little box and I'm going to create a whole universe around it where I decide other rules and whatnot. You might be underutilized. You might not be getting as much bang for your buck from the model because you're not giving the agent enough tools to work with. It's imagine you hired a new co-worker and by the way this funny enough in management I've learned this to be a huge frustration that employees can get when employees don't get enough information when they feel like they're getting siloed when they feel like they don't get access to the tools that they need to use and they have to like ask for permission and access etc. It makes for a very frustrating experience. You feel disenfranchised and disempowered as an employee. Agents can also be similarly disenfranchised. Agents, you only get to emit text like old AI chatbots. And by old, I mean literally like ChatGPT did three years ago. If you give them a universe, if you give them an operating system, then agents can be way more effective, but they can still have a lot of security controls. And so that's the idea of the sandbox. We believe that this is going to be the future of compute. Why? Because the old world of computer was you, Daniel and I would write a program. We would ask AWS for a computer. We would transfer the program into that computer and it would run and that process would take us a week. Then Vercel came and we made that process instead of taking a week take literally like less than a minute. But it's still all human mediated is you and I got together, we wrote some code, we pressed enter. The next and even today is agents are writing the code, agents are testing the code and agents are deploying the code. And all of this is happening in cloud computers which we call sandboxes. And I believe that if you think about how many Daniels and Guileremos existed and many times they had the ability to say okay the software is written let's ship it that was not that frequent like there was only like tens of millions of us and we needed to write all the code by hand and then we would decide to deploy it. I think there's going to be, you know, billions of agents, all of them using computers, all of them creating new computers, all of them creating new software. I don't think the world has realized yet just how much more demand for computer there will be. And I think this is partly because so much of the attention has gone into the GPUs. But I would make the case that the GPU is actually not that interesting in a sense because the GPU is streaming all of these tokens. And this is why, you know, fluid compute by Vercel made it very, very cost-efficient and easy for LLMs to stream tokens into your compute. But now these agents are going to have to use a lot of computers and tools and these are going to be the CPUs and the more traditional computers and there's going to be a demand for it the likes of which we've never seen before.

Mostly really excited. So when you mentioned that example when we were getting ready for this podcast I immediately thought of a co-worker. We created this Slack channel called World Cup. I'm from Argentina. We all want to see as many matches as we can because it's happening here in the US. And one of my colleagues v0'd a real-time dashboard that monitors prices. Just let that sink in. Like he has a busy job. He was just able to make a couple prompts to make himself more intelligent to share with his colleagues to add value to the world. This is the era of personal software. That software program, and this is good to contextualize what I was just saying earlier, that software program that he created could have never been a standalone business, a SaaS company. We're the price monitoring company of the World Cup. So, there's a lot of commentary that I'm not going to get into too deeply of like what happens to SaaS in the future. What happens to software as a service? Well, you know, I think it'll still have a place, but think about it. Someone could have come up with the idea of I'm going to create a price monitoring platform for tickets or something like this and sell it for like $5 a month or whatever. Now, we live in a world where my colleague Diego just prompted that software into existence. It's personal. It's the way that he envisioned it. Maybe I would have done it differently. Maybe not. I actually liked what he did. It's agentic because this thing is going out into the ticketing platform and he's querying for the price and actually I saw the curve of prices is fascinating. So meaning that he's not only orchestrating compute when he built the software. He had the agent get itself a sandbox. He wrote this, the agent wrote the software. The agent then deployed the software because now it has a URL. It deployed it onto Vercel. But then it's running more software to periodically go and get prices. And this is why I think that the demand for compute and you could imagine like the demands on energy and data centers are just going to skyrocket. Because this didn't exist before. This human potential of autonomy when creating software didn't exist before. And so these are the second and third order effects is we yet have not seen what an entire world of people doing this at scale looks like. I think this is still like the pioneers and the people that are in the know. Obviously v0 is democratizing software to an extent where like anyone with an idea can build it but it's the early days.