One of my biggest challenges today in adopting new software, both personally and as Vercel, is understanding the security debt that I'm taking on. Just the surface of attack has become so massive as of recently. We probably need to start writing code in different programming languages. The challenge that I think a lot of organizations have today is that the amount of latent vulnerabilities of all of this human-written code is almost immeasurable. We could eradicate supply chain attacks if we moved more and more of this development life cycle to sandboxes.
All right. So, Guillermo, what will matter in the post-AGI era?
I think our ability to stay in control, okay, as humans. And by that I mean creative control, control over the risks on cybersecurity.
Control over how we make the agents work for us ultimately. You know, I've been speaking a lot about the leverage is shifting one layer of abstraction higher into sort of the hardware engineering, the software factory, the fleets of agents that you orchestrate. I'm a huge believer in our taste and our creativity as human beings, because we're usually appealing to other human beings and we love to buy the stories and products and services from other human beings. You know, we like to think that we make perfectly rational decisions as humans, but a lot of the time we're actually buying into the stories and the vibes and the brands and the flavors and the tastes. Ultimately, agents are going to be hopefully an enabler of greater and greater storytelling and greater and greater human ambition.
I like what you said about the higher-level work because people always are afraid of like this job going away, this task going away. And you know, farming was something that like 90% of the population did 200,000 years ago, 200 years ago, and now nobody works in farming, less than 1% of people. So thinking of this as like filling out a form just feels like manual labor, you know. So what will that look like? Will it be like managing fleets of agents, like deciding which models, deciding like where they run in terms of the hardware?
It's so funny because one of my most, one of my best performing tweets recently was I do this for fun. You know, I build software to relax.
I build software for work and then I go home and I build more software. I build software with my kids. I build software with my wife. Like I'm always building software. So that's where I think the analogy breaks down with farming. And by the way, like there are some people that legitimately love farming and maybe they, you know, over hundreds of years, millennia, they'll still do it.
Yeah. And they keep their little like mini farm in their balcony and that kind of stuff. But I think a lot of us do this because this is our medium for creativity and there's no taking that away. There's no putting that genie back in the bottle as far as I'm concerned. Um, but it does mean that, I mean, you kind of see this with models these days being benchmarked over their ability to create new worlds. There was that Karpathy discontinuity tweet of, you know, first he introduced vibe coding, then he introduced, oh, by the way, the models are getting so good that they can create entire worlds and they can rerender The Hobbit or Lord of the Rings.
And so I do think that we're getting into the next level of how much free range of creativity is enabled by AI, creating new worlds, creating games. I've spoken of how the web itself, the web that we will consume as humans, is becoming like ultra fancy. Like the experiences that stand out to you are the ones that just like really go out of their way in terms of wowing you. 3D and animations and particle effects and things like that. And so it's amazing that like literally we can do anything now. We can build anything. And people are flexing that muscle. Like I see a lot of people that are building stuff that they're just building it because they couldn't before and they're deriving a lot of joy from building, which I think is really good.
I've been coding with AI for nearly four years. In 2023 I was the first to start the build anything trend and since then a lot has changed. We have different models, different tools, different agents. But despite all of that change, every single project I've built has one thing in common. I always use Supabase as my backend. I'm not even thinking about what DB do I use. I just use Supabase. It's simple, it's scalable, and it's agent-first. In fact, it has never been easier to get started. Now, the AI agents can literally build the entire backend for you because Supabase now works inside of ChatGPT and Claude. We all know that with AI, you can write code faster than ever. But where people get stuck is the backend, the database, the login, the storage, all the stuff you need to get right. And Supabase handles all of that for you. With their official connector, you can just ask Codex or Claude Code to use it. Just say, 'Create a new invite table' or 'Deploy this function' or 'Review my app for security issues' and the agent will do all of this for you right inside of your Supabase project. I'm using Supabase for all of my backends and so should you. And the best part is they have a really generous free plan. Just go to supabase.com and try it for free. It's the first link below the video.
So, given the fact that anybody can build anything, what gives someone an advantage? You know, people watching this, they want to know like how do I get ahead? How do I build a business? Is it the taste? Is it the creativity? Is it being more technical?
Yeah, there's so many things. You know, I was actually going to post recently. So, every time a model comes out, you probably like the first thing you do, at least both of us, like we try it out.
And one good exercise I would recommend people to do is there's a series of things that are just impossible for models to do today. Now that list is probably shrinking and changing over time. But you know, 12 months ago that list was bigger and there were a bunch of things that it couldn't do at all. And what you want to do is like you almost want to keep like a list either in your head or like write it down of like what are the wildest, most ambitious prompts that a model could do. And I think having the timing, having the anticipation really pays off because when a model drops, it's almost like people are exploring its latent space and discovering what's possible and it's an opportunity for you to like, it might be that you can build a business like, oh, you know, Opus 5.5 came out and this particular thing that models used to fail at has now become possible. So there's that whole category of things. I think people can use the creative power that they get from the models to, you know, broadcast a better message. So you see this with companies taking advantage of like, hey, models can do this, we can portray our brand or our product in terms of what this model enables. So a good example for us was that when GPT came out, there were all of these really cool things that it enabled for our products. And so GPT comes out, we can now sort of like ride that wave and like teach people. There's such an appetite by the world to learn about AI, to make sense of AI. You know, a lot of people are out there scared.
Yeah. So for example, for us, every time a model comes out, we run it through the DeepSeek benchmarks to help the world understand, okay, this model is good at cybersecurity, it costs this much. There's a lot of alpha right now in which is super ironic because models are so easy to use, like you just prompt them in English and yet there's a mastery to them.
There's understanding what model is best for what job. And so I think there's a whole space of like evaluation, whether it's that you can become a voice that helps other people through public evaluation make sense of what models are best for what. There's also the personal evals that I think has a lot of alpha. You know, another irony of AI is that we do go off vibes a lot. Like I hear people say like, 'Oh, I read this thing on Twitter, then I started prompting my model differently.' I heard someone the other day say like, 'Oh, I read this prompting guide and said that Astra does better with like less this and that and like now I'm doing that.' And I asked him like, 'But did you actually ascertain that is giving you better results?' It's like, 'No, just I'm just going off of like what other people are saying.' And so I think there's a lot of, there's a unique advantage now to are you actually deriving truth from what the models can do and do you have a, you know, a set of like personal evals or do you have a set of company evals that allow you navigate this world that is moving so fast like every single week there is sort of like a new model to evaluate or as we saw with GPT, there's even new categories of AI products coming out every week.
But isn't the vibes like part of what makes us human, you know, because the models, they sometimes like the just the practical sense, you know, it's like, okay, we need to ship this feature. If you're a human engineer, you know, like there's pressure on this and the release is tomorrow and you cannot be like overly perfectionist, the model will happily spend 30 minutes on something useless, something small that a human engineer will just skip given the broader circumstance. So do you think like the creativity, the vibes that will still remain or better models will have this?
I think creativity forever because again unless we go on like a social network made by AIs and we decide to follow AIs and how they're going to influence our future. The reality is that we're still in charge of our brands, our storytelling, our values. Even I think, you know, there's a certain design aesthetic that humans can push forward, right? Like when you look at Vercel, Vercel looks a certain way and we invest in making it so and creating an emotional connection with our audience. And so what you want to do is you want to use AI to, I mentioned sort of amplify your story. How can using AI, whether it's the creative tools, image and video models, can it help you tell a better story? Can it help you ship? I mean, what's very clear, right, is that you're going to ship more frequently and you're going to ship probably better products.
There's still a delta between like are you shipping at the true speed of AI? Are you behind? I think for a while I think we're going to have a range there where there are going to be individuals and companies that are just ahead of how much they can harness this superpower. I think there's an analogy there with, you know, when cloud computing became, you know, I guess fashionable or increased in popularity. There was a relative range of like, you know, there are companies that grokked it right away and used it very efficiently. There are companies that are still like trying to migrate from on-prem to cloud. And I think incidentally, I think AI is pushing a lot of people now like the ones that were sort of procrastinated on it. And you can sort of imagine right that there are a lot of people that are kind of procrastinating on like how agentic they're becoming and they're probably going to get lapped by this organizations that are moving more autonomously and with more conviction.
So let's go deeper on that. What separates like the company that really squeezes AI to the fullest? Is it like building the autonomous software factories because this changes every quarter?
I really, I mean, I really think like we have such a, we have a once-in-a-lifetime opportunity to do what customers want, which is sort of ironic to say, but like any big company at any given time is getting customer feedback.
You know, let's say you're Comcast, right? Like people are saying things on Reddit. They're calling your call center. They're, you know, sending you emails and their internet broke down and they want to understand why. And you know, what's your ability today to like, you know, turn those detractors into net promoters? When they're reaching out, are you responding with a high-quality agentic response? When they reach out, report a problem, are you kicking off an incident investigation? When they're reporting with a very concrete fixable thing, are you fixing it autonomously? I think this is actually a pretty decent way to grade like a rubric for how agentic are you from customer signal that is decisive about something in your product being broken, defective, or in need of improvement. Are you actually shipping that? Are you shipping the fix for that signal? The thing that addresses that signal and what is your latency? Like you could almost make a case that you can measure this sort of mathematically, right? Like, okay, signal came in. Eventually, you're going to get to fixing it. Maybe to give you sort of a lay of the land, there's the company that periodically scrubs that maybe through taste, maybe with product managers, maybe by, you know, getting enough escalations, they're like, 'Ah, I'm going to fix it.' And there's the company that is sort of like becoming truly more autonomous in a good way, right? Not in the sense of like we let the agents go crazy and ship slop. And you know, speaking of GPT and why people are so excited about it, you know, this idea of like really fast decision making can be super critical in shaping these software factories because I'm mentioning that there's an element of judgment and taste also in, you know, famously folks like Steve Jobs, DHH and many others have said, you know, like you're not just a machine that turns whatever customers want into product. There is an element of decision making there but there's also an objectivity to like, you know, your product might be erroring.
Yeah. And so, you know, throwing up a bunch of exceptions in prod or someone comes in with a legitimate very straightforward improvement. You know, we like to do this a lot on X like when people like give us feedback on X sometimes they like attach a screenshot or a video, it's like that's spot on, like there's no human creativity involved here. Like they're pointing out something that they experienced and it was a problem that we should fix it right away. And so that's kind of like an opportunity to like test out this software factory hypothesis.
Interestingly, yesterday we got a guy saying that he was getting confused by our navigation and he gave me some really good feedback but nothing that was like immediately actionable. Like it's not like I could take his thing and turn it into a prompt. And so what we did is the leader of our sort of like dashboard product area reached out to this gentleman and they started going back and forth. He gave us more videos. He gave us more qualitative feedback and even then requires another layer of human judgment to turn it into a decision for what we're going to do with the product. Now one could project very far out in the future which in AI could be six months, 12 months. There is a world where some of that UI and some of that expectation that this customer have that would become their own generative interface. But I think we're a ways away from that. So, you know, this is still our role to sort of like balance out feedback that we get from customers and then sort of eventually find its way into a prompt and eventually find its way into a shipped product.
So when it comes to like the obvious ones like button isn't working, payments are failing, do you think the number of like, like you said the latency towards a fix, resolution, is that like one of the main numbers of predicting of a business success because think like if you go from first principles business is a product that gets better over time.
And if you have a, unless you have a super small addressable market, it's like that business that can just fix everything in sub 30 minutes is going to destroy a business that takes two days.
100%. And that's been the history of technology, right? That's why I gave the example of like the companies that were not afraid to adopt personal computers that said like, 'Oh, we're going to do this stuff by hand.' The companies that were not afraid to adopt the internet early on. The companies that also made it their priority. So, there's something really interesting to learn from the history of the internet, which is a lot of companies like the internet, but it's like, okay, I'm going to hire a company that does the internet for me.
There's so many companies in AI want to do this.
It's still happening. Interestingly, right, like you have, it's so funny because I talked to a lot of enterprises and like it's so common that they tell me that they don't own their own website. And so there's a parallel here, which is that there's going to be the companies that say, 'I love AI. Can we like buy a bunch of AI licenses and throw them to our engineers?' And there's going to be the ones that say, 'No, no, no. This is a disruption that creates a new competitive landscape.' If we become really good at mastering this thing, it's a little bit like redstone. If we become really good at like manipulating this new block, this new raw material that's going to put us ahead. That's 100% what I believe. And yeah, to your point, there's an element of measuring. I don't want to get only, you know, tactical like if you improve that metric, that's the end all be all of your business. We like to say that there's iteration speed and there's iteration velocity. Iteration speed is, you know, like how quickly you're turning out stuff. And that's why, you know, when I hear about how many tokens you spend, token maxing, and when I hear about how many PRs you land, good for you, good metrics. But velocity implies speed in a direction. You could be pring, pring, pring, pring into oblivion. I'll give you a very concrete example. AI is helping people realize that they can build a lot of things. And what I don't know that it's fully helped people realize yet is that there's certain things you should build and there's certain things you should buy. There's services you can use for certain things and there's things that you should build yourself. I talked to a startup founder recently. She came to my office and she was like, 'Yeah, I'm building this really cool idea for agents' and I thought the idea was really compelling. And then she proceeded to tell me that she was also building her own email service and her own payment gateway, her own like... And some of those things might make sense, some of those might not, but like she was taking on a lot. That was sort of my sense. Like sure there's the very optimistic take which is that you can have so many agents working in parallel for you that yeah you can take on a lot more projects but I'm still a skeptic because what you really want to do is you want to pick a few set of problems where you think you can have disproportionate alpha and go really deep and spend as many tokens as you can on those problems and move as fast as you can on those problems and that would imply that vector, that direction that the word velocity implies. So it's not just PR landing, but like where are you headed and actually are you actually making progress towards your goal?
Yeah, because if you're doing 20 things at once, you're going to be like mediocre in all of them. And to be successful, you need to build one thing.
Great. By the way, this is kind of becoming a thing already. Like you realize that a lot of people ship really fast and then you come to their website and it's like a pile of slop. Like you can sense that like there's no value add over what the model did. By the way, this was already the case with the web. Like people would not respect the vanilla template that looked like you just purchased some kind of starter kit and threw it over the internet. I think people are really not like it's almost like you're tuning out a bit when you see something that looks like it was just regurgitated by the LLM because okay what is the value add like if you could one shot it, I can also one shot it and so there's an element there of like again the sweating the details that help you communicate your intent and your values and what you're trying to pursue much better.
By the way, if you have multiple $200 subscriptions a month, pay attention because I want to be working closely with a couple of you to build Cloud Room. This is an open source platform for running agents in the cloud. It's the world's first open source solution for cloud agents, right? You have Devin, you have Cursor, you have Codex, you have Claude. All of these are closed. They want you to use their own subscription, their own harness, their own GUI. But you're already paying for inference. You're already paying for tokens. You have probably multiple subscriptions. You should be able to use that inference. You should be able to use your favorite models, your favorite harness. This is Cloud Room. It's fully open source and it's free. However, right now it's not fully open to the public. I'm looking for the real power users, the people who can give me feedback daily, people with high taste when it comes to coding agents and running agents in the cloud. So, if that's you, if you're paying for a $200 plan on Codex, Claude Code, and a bunch of other stuff, go to cloudroom.dev and join the wait list. As I mentioned, every single day I'll be selecting a handful of people, giving them access and working with them closely to build this platform. So yeah, if that's you, go to cloudroom.dev. It's also going to be linked below the video.
Let's talk about your current agent engineering setup. Like how does it look like? What do you use?
Yeah. So broadly speaking for Vercel, we've encouraged people to, you know, try everything. We're not prescriptive about what coding agent to use, you know, what frameworks to use. Like our stance has always been like open. In fact, our fastest growing product at this point, I think by revenue, adoption, numbers, etc. is our AI gateway. And our AI gateway, like the thesis is, you know, I mentioned if you're a company, you should probably be investing in your agentic engineering fabric. You should also not be using just one model.
You also showed me yours and I thought it was really cool that you basically built your own client or your own IDE and you can choose between all these harnesses etc. So it's kind of stands. In fact, we have this app over here in the menu bar where we give you a sense of like what your token spend is. So every engineer at Vercel gets this app and like they can understand okay like what is my, how much did I spend on the OpenAI model versus the Kim model versus the Opus model etc. So like one of the founding philosophies is you should try many things especially models. Personally you know AI has been amazing for me because I've always been a huge believer in the simplicity of the CLI. So when I started Vercel, if you go on the wayback machine, I started Vercel by shipping a CLI. So you would go to zeit.co and you say install now. Now was our deployment CLI and as the name implies basically all you needed to write was like now in any directory and you get back your URL of what you've shipped and you know we ended up renaming the product. It's now called Vercel. But you know 11 years after the founding of the company, the thing that continues to sort of grow exponentially and continues to give us all our gravitas is this foundational concept. There's a CLI that enables you but now increasingly agents to take anything they've built and put it in the cloud and make it instantly accessible. So my personal engineering stack has always been like can I do stuff in the CLI and I've always been a Neovim user. So my next obvious logical step was Claude Code. I got really excited when Claude Code came out because everyone else was trying out sort of like building very sophisticated apps.
Yeah. And they became successful by basically going towards the kernel of what makes AI different which is well at the end of the day it's just packaging intelligence and giving you a prompt.
So today I'm using FX which is the coding CLI of Vercel Labs. One of the design principles of this CLI is that it almost feels like a shell. So when I open a new tab on Ghostty like or I open a split like it boots into ZSH but one of the things that we obsessed about is like can this boot as fast as a shell itself. So, it's a very tiny binary. Installed is 11 megabytes. If you download it today, which I'm going to... it's actually 6 megabytes. It's 11 megabytes on disk. And it's incredibly fast to start up. So, like the idea was like if I started a shell within a shell, like can I make it like just as fast?
So, what inspired you to do this? Because like most agents are very bloated, you know, with endless pre-shift skills, plugins. What comes to mind is Pi, that's probably the closest minimal agent. Were you inspired by Pi or like what was the idea here?
So interestingly enough you know Vercel for its entire history has been associated with TypeScript. We have one of the leading open source projects in the TypeScript ecosystem which is actually the leading and most popular open source project which is the AI SDK. One of the things that we wanted to experiment with with FX is two things. One, what would it look like to start investing more in native tooling? So, one of the frustrations that I always get with CLI is maybe I'll not to roast them, but like every time I would launch Claude, you can sort of see it there like there's the boot up time and then there's the like disclaimers or whatever like the time that it would take me to start up and like I just want to get something out of my head as fast as possible was kind of annoying. But the other thing is that I started realizing that the Unix philosophy is about composing small programs. And a bash, the bash shell or ZSH and whatever, they're just that, right? Like they're programs that call other programs and so there is an element of like let's experiment with making a program that is actually going to be fairly immutable over time there shouldn't be more UI added to this over time.
Yeah, that's the problem with a lot of these agents like you build a business on top of them and then like it's crashing, you know.
They start out small, they start out fast, you know, this actually what initially made me fall in love with Claude. It was like, 'Oh my god, there's almost nothing to it.' And then over time, I think you really need to fight off this temptation to add more and more and more. But it is true that people should be able to extend these things. That's why they're open source to begin with. And so FX is actually simultaneously this like super minimalistic CLI and also a library that you can embed in other programs. So the classic sort of C style paradigm here is that you have your CLI and then you have lib in that thing. So there's lib FX that goes together with FX. I'll tell you like this thing is so addictive that you know I started using it for my own personal research. I realized also that it's not only that we were gonna have to like clear this probably in post, blur it. But I started realizing that like, you know, another thing that AI was sort of bringing to the foreground is going back to the fundamentals of Unix and the fundamentals of computing. It's not just a bunch of small programs. It's also the fact that the file system is basically all you need. Today prior to this interview we announced Vercel Drives which is a way of sort of like modeling a file system in the cloud and that you can sort of like launch a bunch of agents around. So you can launch a sandbox and attach a drive.
I don't know if you saw some people started reverse engineering Muse and Instinct or how OpenClaw works. OpenClaw was so simple that it was just Claude Code plus soul.md.
Yeah. A Linux program plus the file system is basically all you need in order to run an agent. So I'm using FX for a range of things. One is anytime I have a question about virtually anything instead of going to like, I actually it's replaced Google for me incidentally because instead of having to go to like this cloud thing I can keep a lot of my sort of experimentation in the file system. I can have the agent look at it. I also, you know, and this I think is also like a little bit of like the future of software is it becomes a lot more like English driven. The fact that my agents.md file in this research folder sort of like creates the convention for if I ask the agent a question, reads up the instructions and then dumps my research into this folder. And I can sort of basically improve it over time. The other thing that I've done is that I have this alias which is like literally one letter R, which goes into this directory and launches it. Basically, there's not much to do here. It's like if I said like tell me more about tech podcasts. There's basically nothing to do, right? Because like it's cding into a folder and opening FX, but it still cuts down on all the latency. Like it gets addicting because anytime I have a question that I want to go deep into like I just fire it off and then move on with my life. The other alias that I created is P. So this is similar to what Toby does with try. I don't know if you've seen his CLI, Toby from Shopify. It's very common that I'll have an idea to create a new project. And so anytime I have such an idea, I do the same thing, but it creates a project in my projects folder. And then yeah, I can do anything here from like I will do a lot of experiments with like local Mac apps. I will create new Next.js applications. I will you know create new Zig or Rust programs. I do a lot of benchmarking to help our teams.
So anytime we created a new piece of infrastructure, I try to validate it from first principles on like how an agent would approach it. A good example is when we announced Drives, I had an agent go and try to like break it. You know, try to find race conditions, try to measure latency, try to compare it with alternatives. I do a lot of what I call agentic inquiry, which is like I'll have an agent go and try out one of our technologies and then I'll have it reflect on tell me what Vercel could have done better for this experiment and it's been doing wonders for me.
So FX is just the harness you use. What's your default model right now?
Yeah, so I try to test out everything over the last couple days. I was trying out I mean Opus 5.5 fast. I've been very impressed.
It's by the way I should show you something that I mentioned. One of the ideas of FX is that you should be able to embed it, right? And one of the inspirations for lib FX was when we created V0, we needed to build a harness from scratch. If I were to create something like V0 today or any kind of application like imagine like Gamma which like has like builds slides, if I were to build something like Notion or anything that requires sort of agentic intelligence to be embedded I would want to pick a harness off the shelf and so we created this example that we're going to open source I think you're going to be one of the first people to see it so I'll give you a prompt create a simple page that says height. And so we made the default model Opus 5.5. So the easiest possible V0 that you could build is just creating HTML pages. So we did this thing where we help you choose between design, app, or agent. And I like you, I've been extremely impressed with the speeds. Like change background to red. It's insane. It feels like you're designing in real time. And the level of accuracy like design an icon, design a set of SVG icons, make it wide again and describe a pitch for a cool tech podcast.
So what was really cool about building this demo of FX0 was you know back when we built V0 for the first time I think the amount of time that we spent just on the harness was disproportionate like 90% of the engineering time went into you know like figuring out AI SDK and the models and the prompts and the tool calls and all of this whereas this example was probably like I don't know 100 lines of code on top of the create FX agent API. And so right, so I was mentioning this is just like using the ability of the model to like output HTML. But the second thing we did was like, hey, using this mental model of like brain and hands. I don't know if you've heard of it, but like the architecture that we recommend at Vercel is that you run the harness and then you give it tools. So in this example, the tool that we're giving it is the Vercel sandbox. So this prompt said build a landing page for a CLI coding agent. It's I mean it kind of looks like the previous one that I showed you but behind the scenes actually running a full Next.js application.
Just HTML. And then we gave it a third one which is agent and it creates an EVE agent when you give it a prompt. Now, what's really cool is what perhaps took us like a couple years of engineering with V0, like it's now part of the infrastructure that you can use off the shelf. And we've had a lot of customers come to us saying, 'Hey, like I would love to add like app generation capabilities within my system of record or I like what you're doing with V0, but I want to have one that's sort of like internal.' Like imagine sort of having your own quote unquote software factory where one of the capabilities is design.
I think what AI is showing us is that you know what used to be software that you purchased is now becoming like extensions of the infrastructure that sits on top of the model. And so you know the level of creativity you can have on top of this is kind of limitless. Another thing I noticed Opus 5 is really good at is like, build a pixel art scene of a dinosaur jumping up and down and making magic. Let's see what it does. But it's been just, you know, the idea that you can embed this harness and sort of pointing it in a new creative direction gets me really excited because, you know, we have a lot of customers that are, you know, building like gaming platforms or education platforms or like healthcare, finance, etc. So lib FX is sort of even though it's a coding agent, what I love about it is that at the end of it is just a thing that's really good at working with a bunch of models calling tools producing outputs reflecting on it. For example for this particular FX0 example we have not given it a browser but you could and the model only gets better if you give it a browser. You can run it inside the Vercel sandbox or you can use one of our marketplace partner services like Browserbase or Kernel and yeah it's been super fun to build.
Yeah, this is crazy amount of progress in like three years. V0 was 2023, right?
Yeah. So, I remember prototyping it took like minutes to get anything up and going and now you have, you know, it really is only limited by your imagination.
Right. Like it's amazing to me that, you know, we needed a big team of people building something like V0 and it's now a library that you can import and you can build basically vibe code your own.
Yeah. So then human is the bottleneck really.
Because like knowing what to accept and also like unblocking these agents because you could have like you know I believe that the harnesses and the models are good enough to have thousands agents running productively but nobody has really thousand agents running productively right the thing is to figure out like how does that work and I think a lot of the lessons are like borrowed from what works for humans right.
Yeah, I mean neural nets inspired by the brain so I think like there will be some way of you know having a manager or like some director agent.
There's also a lot of interaction models I think because people I think in the different like eras of AI kind of overindexing one modality or the other but like let's look at this example for example like if I want to create a video for X where I had this idea of this dinosaur doing something really cool at a laptop like you still need this systems that are like super fast and have you in the loop very much like making decisions and like trying things out etc. And then there's another interaction model which is more like the you know go and solve problems for me. And there is also the one that is sort of coming off of a signal like we talked about, right? Like something is broken. So, maybe for fun, let's look at the so make me a to-do list with persistence to disk. So, this would be the case where like if I'm building a full stack app, by the way, like we've made insane progress in like making our sandboxes extremely fast. I think the future of dev tools like Next.js would be very much making these use cases work really really well where like you have an agent driving the experience rather than a human being. And but still like when you give an agent a computer like there's almost like the assumption of like I'm going to go and let it cook in the background, right? And you know I think more and more of these hardcore go and build applications solve problems etc. they might not require you to be in front of them every single time. So I guess it depends on like what the input is, right? If somebody reports this button is failing and the model can check it and ship a quick fix, that's obvious. If you can define like we want to optimize the boot up time and you launch your auto research for like seven days running, but what you're describing here is like you need to figure it out with the agent, right? You need to sit there and really work with it. And if it takes two minutes to get a response, you cannot be doing deep work because the cost of that two minutes has never been greater, right? You can unblock to another agent. That's why I think people really liked plan mode, even though now it doesn't really have to be a mode per se, is that there's an element of you going back and forth.
With an intelligence that is really important that it feels real time. In fact, I would not be surprised if more and more goes into voice for this reason where like you just talk back and forth with the model a bunch on figuring out the plan and the model pushes back and like improves your idea, etc. And there's also the whole premise of what I've been calling verification engineering, which is what you're trying to do is set up the set of guard rails or the proof system around the thing that you're building. And that also requires that you and the agent agree on a set of axioms and potentially a set of tradeoffs because I can tell and this was true actually in the development of FX, right? Like we can do amazing things in FX for startup time for making the binary smaller etc but everything comes with trade-offs and some trade-offs are for example like how much time you spend in the CI/CD pipeline some trade-offs are about you know speed versus safety versus binary size and there's a bunch of things there and so I think it's very important that you as a human spend time so almost in the edges of the testing system. Someone was saying today on X that they don't believe in unit tests anymore. And I've been kind of personally been there for a long time because over watching engineers and working very large teams I started realizing that a lot of human engineers who spend a lot of time in these micro unit tests that give you a false sense of confidence like and by the way the models are incredible but sometimes I see models just write the most inane test that you've ever seen like export const something equals four And then they write a test to ascertain that the constant is four which is kind of silly right and so I think the more we can spend our sort of design tokens into the edges of the system and less on like the little tiny implementation details the more effective you'll be at leveraging the agents and these might be things like again what are the set of actions about the correctness of the system you know what is the aesthetic sense that you want to embed into the system. What are the performance constraints like you know what are you not willing to trade off in terms of speed or you know maybe you don't have a certain performance budget but if the model doesn't really know what you want it'll take you in whatever direction it wants.
So what is that skill right? Like the skill is like system design, architecture, kind of a mix of like software design, core-based architecture like how would you describe that skill?
Yeah. Yeah, it's a really good question. I think on one hand I think there's this whole field some people call it like symbolic systems which is instead of like focusing on like the specifics of syntax and programming languages can we reason about systems at a higher level of abstraction. So maybe to make it concrete, what I think people should be able to reason about is the concepts of engineering, what is a library, what is a framework, what is a compiler. So understanding how the pieces fit together. Maybe a better word would be the ontology of engineering. And another thing that's true about engineering is that there's algorithms and there's data structures. And so understanding what data is, where it lives, how to get it. You know, understanding how long things take is also very helpful. You know, even prior to agents and AI, I would recommend people to look at this resource from Jeff Dean at Google. Let's see if I can pull it up. It's called latency numbers that every programmer should know. I think it was from Jeff Dean at Google. So this table I've shared it so many times with engineering teams. It gives you rough senses of scale and orders of magnitude of how long engineering systems take. So and by the way funny enough because Moore's law is kind of still alive like these numbers have to get updated frequently. So this is 2012. They're probably already but the orders of magnitude are probably still the same. But like an L1 cache hit on a CPU is half a nanosecond.
A branch mispredict like for example understanding how CPUs work at a high level super helpful. Like branch prediction and you know there's tiers of caching even the concept of there's tiers of caching. There's certain things that are true about systems of every scale. The Vercel global CDN is a multi-tiered caching system. When we first designed the CDN, we came up with an L1 cache and L2 cache and L3 cache at large scale, which is that when you go to a website really fast, like when you go to rauchg.com, the reason it's really fast is I probably hit the L1 cache of the Vercel CDN in San Francisco. And if I were to miss on it, there's an L2 cache. And then if I were to miss on that, there's an L3 cache. And there is a pretty fast underlying storage system behind that. So, I would recommend people understand these things. You know, you don't have to memorize everything, but one that I particularly love, oh, look, they've been keeping this up today. That's so fun. They've added LLM stuff. But I love this one. And I've used it occasionally as an interview question or I used to do like sort of level set if the person that I'm talking to has experience in like systems engineering and networking. How long does it take to send a packet from California to the Netherlands back to California over a well-tuned high performance network? So it's about 150 milliseconds. I think there's also one here.
So what type of answer would you not accept?
Oh, people get this very wrong sometimes. So, you think it's seconds, they think it's 10 milliseconds. It's very important to know that it's somewhere in between 100 to 200. You know, when people say 200 or 250, like I almost like I'm okay with it because like this is if everything is like perfect. You're dealing with, you know, Vercel spends a lot of capital on the interlink between regions because we need encryption in the network stack in between like so for example like you put the Vercel CDN in front of a certain workload the communication between the edge that's closest to you and the origin has to be encrypted. It has redundant it sends redundant packets. The packets get cloned and it goes over a high priority network. So when packets go over the internet they shuffle around routers and ISPs. If you run trace route you can sort of follow the life of a packet. The worst case scenario is that your package just gets tossed all over the internet and you end up in like low priority queues and in a high priority bandwidth system you're basically like you know taking up space in this very very expensive backbone that goes that is underneath the ocean between California and the Netherlands. So knowing I think how these systems work, it's almost like, okay, you don't want to have too much encyclopedic knowledge because the AIs are really...
good at that. But if you're also completely clueless, I think you might fail to push the AI harder. So if an AI, you know, ships a system and I'm hosting it in the Netherlands and I notice that, you know, the page is taking like a second to load, knowing this fact can help me say to the AI, make it faster.
Yeah, make it faster. Make it faster.
And so now the counter argument that you could create there is, well, the AI will just create optimally fast software in the future. I don't buy that because when you give it a task or a prompt, etc., the AI itself has to work with a system of trade-offs.
Yeah. You need to set what matters for this project.
Exactly. And you can't spend infinite tokens on every task. And so I think what you need to know as an engineer is what are my levers? What levers can I pull? I'm not going to do the work myself.
It's a little bit like giving the plan to the AI and the AI can sort of go nuts. But you're still in charge of creating that plan to some extent. So I guess it's being very technical on a high level in your domain, like whatever you're building in that domain, you need to know like...
Your domain is also a thing, right? Like I've become more savvy thanks to AI about areas where I wasn't. You know, if I went to my research folder right now, like I'm constantly researching things that were so out of my scope in the past. And so I do think you can get... you want to be a sort of... you want to dominate the conceptual space of your domain. And then as you build more confidence in certain domains, I think you can get more comfortable expanding and going to other places.
I think the key here is really getting both the most out of the AI but out of yourself. A lot of people want to maximize their skills and MCPs and stuff, but they don't use the AI to get more technical. I mean, I really encourage... I mean, again, this is sort of my take of the world, but I encourage people to understand. And there's not just understand, I would say reject non-understanding, because I do see folks proposing like...
