Have you had Guinness before?
I have actually never had a beer in my entire life.
All right, well, you're starting with the best beer— So that's good. You order your Amazon packages with Devin?
So you're just in Slack and you ask it to buy something for you?
Yeah, just, @Devin, can you go buy some more whiteboards for us? Or something like that.
I really enjoyed math competitions and going and competing and doing these things. And this is stuff like, if I ask you, "What's 694 squared?" It is 481636.
I have shuffled the cards. I am not collaborating. We give them to Scott. So now you have six cards and you're trying to make 163, right? And one way that you could do that here is two times 8 is 16. 9 divided by 3 is 3. 3 plus 16 is 19. 12 times 12 is 144. 144 plus 19 is 163. And so almost all combinations can be— But you're probably thinking like, "I could have done that. That's too easy."
Let me just tip it upside down like that.
Very good. Scott Wu is the co-founder and CEO of Cognition, which makes Devin, the AI coding agent. Scott is a triple IOI gold medal winner and kind of famous for being a math whiz. And now he's at the cutting edge of agentic software development. Cheers.
Tell me about your upbringing and all the math stuff. I feel like you're known for the math stuff these days.
Yeah, so I grew up, I'm from Baton Rouge. My parents were both chemical engineers and so they immigrated from China for grad school. And then naturally when they were looking for jobs, they were doing air emissions permitting and things like that. Louisiana has a lot of oil and gas and so that's kind of how we ended up—
I love the air missions too, actually.
Yeah. And so that's how we ended up there. I always loved math as a kid. I had an older brother named Neal. Super, super close, the whole way through. And Neal was about five years older than me. Neal started doing math competitions when he was in middle school. And so he would've been in sixth grade and I was in first grade at the time. And naturally I, as a little brother, would go and just watch what he was doing and try to learn some of the same math too. And that's kind of how I first got into math. And then, I found that I really enjoyed math competitions and going and competing and doing these things. And this is stuff like, if I ask you, "What's 694 squared?" I think it's probably not quite things of that nature. It is 481636. But it's things math puzzles, things like, the frog that's like going up and then every night falls down the well. And how many nights, these kinds of things where you get to—
Yeah. Where you kind of get to do the critical thinking and come up with interesting ideas and stuff like that. So I started doing math competitions in second grade. I remember there was a contest at the local college that I went to, which was for middle schoolers and high schoolers. I competed in the seventh grade math division as a second grader. I did the competition. It was my first time doing any of these. I just really liked math and stuff. And then they were calling out third place, second place, first place. And none of them were me. I still just remember, I was so upset.
That's your super villain origin story.
Yeah, exactly. That's how it all began, basically. So then I trained a bunch, the next year I was in third grade and I competed in algebra one or something and I won that year. Then I basically kept doing math competitions from there. My last year of high school, which would've been my junior year. I left a year early. But I did IOI the programming.
I did IOI three times, and I got gold, yeah.
So, where'd you go to school?
I took a year off actually. I left high school a year early. I wasn't that good at school, I guess. I left high school a year earlier—
Obviously, that's surprising, you weren't that good at school.
Well, I I wasn't that good at finishing school. I have a middle school degree, but, I didn't really make it through high school or college. So I left high school a year early. I spent a year actually in the Bay working at a company called Addepar. And I did that as a software engineer. That was back in 2014.
Yeah, wow, that was a blast from the past.
Yeah, it was a while ago. And then after that I decided, OK, I will go try out college after all and see what that's like. I went to Harvard for two years and then I dropped out.
How did you end up at Addepar? And that's very forward thinking of them, obviously, that they took on a high school aged, high school dropout.
Yeah. It was a fun group. Funnily enough there were four of us who started at the same time as high schoolers. It was myself, Alexandr Wang is another one. We started on the same day. Eugene Chen, who's now running Phoenix DEX, and then Sreenath Are, who's most recently at Sandbar as the CEO.
Wait, sorry, this is a real small group theory moment. So you and Alex were in the same—
That's right. So we knew each other, we met in middle school.
Now of Meta. That's right.
We met in sixth grade. He was from New Mexico, I was from Louisiana, but we met in this math competition called MATHCOUNTS. We were both at the national competition, and then we started talking. Google Hangouts was the thing at the time.
It turns out there's some math in AI.
Yeah, this may be an— Yeah, it's a fun thing. Well, a lot of the folks, as it turns out from our vintage ended up being— I think there's like a real infectiousness of, being entrepreneurial too. I think Alex deserves a lot of credit for, I'd say being the first of our group.
Alex Wang got you into the idea of starting a company.
Yeah, somehow I think there's definitely a bunch of that involved, for sure. But also, a lot of folks, Johnny Ho, who is one of the cofounders of Perplexity, for example, Demi Guo, who started Pika, a lot of these... Jesse Zhang who started Decagon. A lot of us were actually competing in these math and programming competitions in the same year. And we all knew each other.
OK, so this gets something I was wondering. There's this topic that people talked about a while back of, where are the young founders? There always used to be people in their early twenties working on breakout companies. Michael Dell was 19 when he started Dell. 23 when he took it public. Obviously, Mark Zuckerberg was very young when he started working on Facebook. And when it was a real breakout, he was still very young. And there was a period where there was no young founders. And now there's many, many more, like a whole bunch of the people that you mentioned. You're 28, running Cognition. Is the presence of young people as founders of leading companies a biomarker for industry vibrancy? Where, Michael Dell was young during the takeoff of the PC era and, Mark Zuckerberg was young during the takeoff of social networking, and now we're in the takeoff of AI coding tools.
I should just say, I appreciate you calling me young. I think relative to being 18 or 19, is still a long way. The test is in your twenties. So I have a take on this, and I'm curious to hear yours on this. I've been thinking about this question as well, and my take is actually just that overall being a founder has just gotten harder. That's probably the biggest, the highest order of it. I think the reason that young founders who are just really sharp and really determined, did very well is because at the end of the day, being a good first principles thinker does beat experience, and just a lot of being a founder is doing something that has never existed before and coming to your own conclusions. The thing is now there's a lot of people who have both, the first principles thinking and the experience. And I think things have gotten a lot more... call it mature as a space. And so basically it's gotten harder. So there are fewer that are literally coming out of college.
It feels hard to make the claim that, it was easy to start a leading business in prior eras. Facebook faced lots of competition. It's not like Dell was the only PC maker. I don't think they had it easy by any stretch of the imagination. However, I think you are getting at something where clearly all the large companies these days, they're very aware, they're very connected with the ecosystem. If you look at Satya or Mark Zuckerberg, they are very aware of everything that's going on AI, and they're paying a lot of attention to it. And so yeah, maybe there aren't giant opportunities that are just being left on the ground by the big established companies.
Yeah, and maybe harder is not the right word. It's more just that the space is a bit more mature and there's more of a playbook and more existing knowledge. There's obviously something unique with every business, but lot of the details of, "Here's how you should structure equity, here's how you should figure out, fundraising, here's how you should hire your initial team." Many of these things I think do carry over a lot with experience. Where, I think in previous areas where the book wasn't written at all almost, so it just came down to how sharp you were and how good you were at making your own decisions. I think now there's a lot more experience to draw from. Maybe that's part of it. I also do just have a theory of like, I guess I would call it, the "Moneyballification" of everything, to give a few examples, one of the things that I do casually for fun is playing poker. Poker's a very fun game. It's actually much more mathematical than a lot of people realize. It's very, of course people kind of— I think people know that, like the poker solvers in the odds tables and everything.
Solvers and such. Or is it more mathematical than that?
No, I think that's right. Well, I think there's a first sort impression of, it's all about—
I'm all in. Yeah, exactly. Pay the person on the other end.
It obviously is much more mathematical than that. But the one thing that's kinda interesting is you see it in the evolution of the top players in space as well. Back in the day, in the 80s or 90s, the top pros, I don't think the idea that is less competitive, but the skills that made someone really great poker player were just really great intuition. They understood a lot of the mathematical concepts, but just at a very system one level of just being able to kind of think about them and obviously they had a good feel for the game and a good sense of how they should be able to kind of improve their own play. Now it's just all math nerds. At some point when the space gets mature enough that, you know what I mean? Where for a less mature space, when people don't know what the right questions to ask are or how to even think about it, like what is the right frame of reference. Then I think there's something about just, having a really sharp intuition and coming to your own conclusions. Then at some point, as these things get more mature, the conclusion of it is math. That's been the case in a lot of different fields. And I feel like it's happening a little bit for startups as well.
I see. More spaces have resolved to their underlying, like a chess engine, just deciding that the position is, mate in 41 or something.
Yeah, and chess was totally the same way, by the way, back in the 1800s people—
The romantic style of play is gone.
Yeah, exactly, the romantic style of play. And now it's like, there is a right sequence of moves and, just seeing how close you are to that optimum.
What are other domains where the "Moneyballification" of everything is shown?
One of my other hobbies, which I played at least before, the advent of Cognition was a game called Super Smash Brothers. I used to play tournaments for Smash. You saw very much the same pattern where the, it's a game called Melee in particular. I don't know if you played Smash.
It's for the Game Cube, which came up 2001. So it's a very old game, but, people keep playing the same game. For the first six to eight years of the game, the personality was really wily, sharp thinkers, people who are just quick on their feet and coming up with these ideas. Now it's just all math. And that the people who play and do really well are—
I think some of the RTSs are a little bit that way as well. Hopefully has gotten less creative as people have gotten better at them, so.
Yeah, and it's a funny thing where it's like, there's a lot of beauty in the nerd side of it too. It's just a difference in what skills get most selected for, is maybe the way I describe it.
OK, I'm getting distracted from asking you about Cognition.
What is Cognition? What does it do?
Yeah, so we're building the AI software engineer. We've been building Devin for, the last year and a half, and most recently just acquired Windsurf. Devin, the agent in Windsurf, the IDE, but at a high level, we really want to build the future of software engineering.
Is it confusing for people that you have two brands, you have Cognition, the company, and then Devin, the slightly anthropomorphized instantiation of it. We've been talking about that. Now there's Windsurf as well, and so now there's a third thing. But I think some consolidation is probably good. Got it. OK, and so people are maybe familiar with the, the GitHub copilots or the IDE-style paradigm where you're there writing code in your IDE and it helps you auto-complete it, or you can give some instructions in the IDE. That is not the Cognition-Devin paradigm. Instead with Devin, you're in a Slack channel with Devin, and you're prompting it to go off and "Build me an X or a Y," but you're talking to it as you would a coworker in Slack.
That's right. Yeah, and so you can call it from Slack or Linear or Jira. Or you can call it from your IDE as well, but you don't have to, right? I think that's exactly right. There's been this paradigm, in the past, I would say GitHub copilot was really the biggest and most well known originator of it, of IDEs. I would describe it as basically when you are typing at the keyboard as an engineer, making you a little bit faster at it and giving you the tools and the shortcuts and everything to do that faster. Devin is a very different paradigm of what I would call like an async experience. Where you have an agent and you delegate a task. And so Devin naturally operates a little bit more like at a ticket level or project level or something like that. You have some issue and GitHub or something and you tag Devin and then Devin gets to work on it.
Yep. Yep. What level of task is Devin doing a good job of today?
Yeah, we like to call Devin a junior engineer today. There are some things that an AI, of course, is way, way better than all of us at, especially encyclopedic knowledge and just pulling facts and things like that. There's some things that, that it still makes terrible decisions on. But I think that's the right average overall. What we see folks typically using it for are things like bugs, for example, or like simple feature requests and fixes and so on where you're talking about an issue and you and your team are figuring out what you should do and you're just like, "hey, @Devin go do this." Or on the other hand, a lot of the more, I'll call it the repetitive tedious tasks that come up often in engineering work. That's often, migrations or modernizations or refactors or version upgrades. It's crazy how much testing and documentation, it's crazy how much of the software engineers of the world's time is, more things like going and fixing your Kubernetes's deploy than it is building, and coming up with really—
Dependency management, all that kind of stuff.
What metrics can you share on where the business is at?
Devin is deployed in thousands of companies all over the world. We work with some of the biggest banks in the world, like Goldman and Citibank, all the way down to, startups with two or three people. In general, a lot of how we look at things is in terms of merge pull requests, and getting Devin to the point where it is a significant percentage of the merge pull requests in your org. Typically in a successful org, Devin is merging something in the range of 30% to 40% of all the pull requests that come through.
You talked about this async model, but isn't it the case that as I look at other, the Github copilots and the Cursors and everything like that, or Claude Code, they're not fully synchronous because you prompt them and they go off and do something. Are these distinctions a moment in time thing? Do they go away where everyone is synchronous in the cases when they can do it instantly and asynchronous in the cases where they don't? Is this a durable distinction?
It's a good question. I think the two experiences continue to exist for the next while. Then I actually think figuring out, the shared experience between them actually is the really interesting thing, right? And that's a lot of recently with Windsurf and things like that. It's something that we've already been thinking about and now are pretty excited to ship some things in the near future on.
Do you know the concepts of essential complexity and accidental complexity? Have you heard about this before?
Yeah. OK. Yeah. I think there's a real thing where, maybe one way to describe it is the ethos of a software engineer. What it means to be a software engineer in my mind is basically just somebody who solves problems in the context of code, right? It is somebody who tells the computer what to do and makes all these decisions of, it can be big decisions like "What is the right architecture that we want to use for all of this?" Or it can be like a lot of these micro decisions like, "Oh by the way, there's a case where this balance is less than zero, and what do we want to do here? Should we show an error or should we request this?" or whatever. All of these decisions are what people typically call the essential complexity what is all of the actual underlying logic of the decisions of what the software is doing, right? The accidental complexity is basically everything else. All the things that you have to do to support things as they scale. Or all of your standard, for example, anytime you have a class, you probably have all the standard carotid features along with that as well, where everyone knows that you need to have that in your class, but there's no real decision that needs to be made in terms of going and doing that. It's an interesting thing, which is, up until, AI coding has come along, The meat of software engineering has been in making the decisions, and yet you spend 80% or 90% of your time doing more of the latter, of just going and doing the routine implementation and so on. I think this merged experience that comes up is basically something where for anything that actually needs you in loop where you can go and make the decision, and you are looking at the high level strategy or deciding what you want to build, you are involved and you're doing that synchronously. Then for all the parts that are pure execution, you are able to hand that off asynchronously. So, the interesting thing is that obviously for an individual project, there are typically a long stretches that actually are one or the other, and it alternates between both of them, right? And I think what that will effectively look like is, the synchronous experience is the IDE where you are looking at the code directly and you see each of these things, the asynchronous experience is the agent that will go off and do each of these things, but to be able to go back and forth between your IDE agent.
So you want the engineer to be interactive with the agent as it's going and working, but on the high-impact moments of important choices as opposed to all the grunt work? How do you get large enterprises comfortable with giving Devin's sufficient permissions to be effective? But like you talk about the migration use case. Super boring. And so you change the table and get it talking to the new table, and then eventually you delete the old table. And that last step is kind of scary. I think people still have a... models hallucinate way less than they did, but people still have fear of the model just making something up—
...doing it. And so yeah, how do you get people comfortable with giving it enough power to be effective?
So we pretty strongly recommend that people using Devin don't give it, broad database access, for example. That's one way. I don't know of any instances where it has been an issue or things like that, but obviously, you'd just rather not take that chance. The framing that I would give honestly is, we have processes for these things because humans make mistakes too. And that's why we have pull requests and review. That's why we have CI and that's why we have all these things already, right? So Devin naturally slots neatly into all of these things. Typically the way that folks will work with Devin is, they're doing some big code migration and they'll break up the task, or maybe they have 50,000 files that all need to go, upgrade from this version of Angular to that version or something like that. And Devin will go and do each one and it'll make pull requests, right? So you will go and review the code and make sure things look correct, but there's still this human—
It's back to your point of incidental complexity, where the reason a migration is time consuming is not the actual single deletion step, all the time cost comes in other places.
Yeah. Exactly. I think in practice what we see with folks is, especially in these like enterprise migrations is, is when folks measure internally, they see something like an 8 to 15x gain for a lot of these use cases with Devin. Because, as you're saying, you're just reviewing the code, you're not going and writing every single line or going through every single reference or things like that.
So let's talk about that, because I think all organizations around the world are trying to figure out the productivity impact of AI coding. I think what everyone sees is engineers for sure want to have access to AI tools for coding. It's not totally obvious on the, PRs per dev type metrics, what's happening. Generally, you see some increase there, but of course it's not clear how good even a, pull request per dev metric is. Then maybe you can say that there's some, ongoing maintenance cost if you're shipping low quality code or something like that. I feel like everyone right now is looking for some slam dunk productivity data on what is the impact of... There's probably some CTOs looking at the slam dump data to justify, the spend to their CTO. What's your view on how big is the productivity impact? Is it actually measurable?
Yeah, for sure. I think this is something where this gradual shift towards agents actually will help a lot, as it turns out. If anything, to be honest, I think IDE productivity is often underrated because, how do you state it to your point, right? Like, you look at the numbers and it's, of our engineering org on average people took the tab completion 238 times this week. It seems quite clear that that should be worth something and it should make you faster. But how much faster does it make you, it's a bit harder to say. On the other hand, with agents a lot of the workflow obviously is going and doing the task for you, right? So if it's a Jira ticket or something, or a migration or things like that where you typically do have a good sense of how many engineering hours are going to be needed for this and what's going on. And because it's doing the whole thing end to end, it's a lot more clear of like, yeah, you didn't have to do this migration anymore, you reviewed the PR in five minutes and that's all done.
As time goes on, I think these things will become more and more and more clear.
There is a view that some people have out there that coding tools are a moment in time thing that get run over by increasing model performance, GPT-6 or GPT-7.
Presumably you do not hold this view.
How do you avoid getting run over by the labs?
Yeah, for sure. So look, I think the labs are obviously, I think they're incredible businesss. As best as I understand it, I would describe this view as a, call it the nihilist computer use take. Which is, of course all of these different things that we do in the world, in knowledge work involve using a computer. And the AI is going to get better and better and better at using the computer until someday there is nothing left except just the AI going and using your computer and doing your work for you, to the best of my understanding, is kind of that the argument there. I see the wisdom of it. This is the kind of thing that's very hard to disprove. But I think that, in practice what we've seen in the space is naturally there is a lot of contextual knowledge. There's a lot of industry details, there's a lot of... And so, as we were saying, going and doing some angular migration or doing some... It's not to say that these things can't get better. In fact, I think they will continue to get much better. But I think that the way that we make models better and better at them is by giving it the right data of... How good can you be at angular migrations if you've never seen angular, and you've never done an angular migration yourself? And there's kind of a cap on that. And obviously there are all sorts of these things of, using your Datadog to go and debug errors. The biggest thing I would say here is software engineering in the real world is so messy, and there's all sorts of these things that come up. I think in practice, most disciplines look like this. I would say the same thing about law or medicine or, and so on. So, while the general intelligence will continue to get smarter and smarter, I think there is still a lot of work to do in making something both, on the capabilities side, really good for your particular use cases, but also in actually going and delivering a product experience and bringing that to customers of how that actually happens in the real world.
So it's not a general intelligence task, it's a specific intelligence of, working in the Stripe code base requires some general intelligence, but requires a bunch of context, requires working within the workflows we have and everything like that. And you think that persists as an area where you need to specialize?
Yeah, exactly. Maybe one way to put it is, I think the argument is something like a super intelligence. I think in some sense, yes, I think we are, you could consider us short super intelligence. I think what we're getting to with RL as this thing is improving and improving, and we see more and more of the gains and people are developing the techniques. I think of RL and this paradigm of AI as basically, the platonic ideal of it is the ability to solve any benchmark, right? You have exactly a dataset of here are the things that you want and here's how we measure success, and here's how we do that. And whatever that benchmark is, it can be the hardest thing ever. It can be a unsolved math problems or whatever. Someday we want to get to the point where we can just take that and train a model that will just get a hundred percent on it. I think, frankly, we're moving towards that ideal a lot faster than most folks would've expected. I mean, there's been some pretty crazy, developments like the IMO Gold Medal or the scores on Sweet Venture or things like that. The thing is, when that happens, I don't think what we end up with is just pure ASI, end of humanity, human knowledge work or whatever. I think the thing that we end up in is basically a point where the hard question is, "All right, now, what is the benchmark?" Right? I think defining the benchmark in all of these spaces is a lot of the practical, real messiness of the world, right? And so for a software engineer, obviously, what are all the tools that you interact with on a day-to-day basis? How do you use those tools? What does it mean to build a representation of the code base over time? How do you decide whether you shipping the feature was successful or not successful? All of these various things and creating the right environments around them.
And so can there be a good benchmark for a model's performance on the kinds of things that Devin wants to do? Or is Devin's business model and... Devin's revenue is the benchmark essentially?
Yeah, it's a good question. From our perspective, we have a lot of benchmarks internally. The biggest is one that we call junior dev, which we might need to upgrade to senior dev pretty soon. But it is basically the ability to do a variety of just random real world junior dev tasks. We've shared some of the examples. Obviously we don't publish the whole benchmark because then it would, get obviated. But a lot of the tasks are things like, "Hey, you need to go and fix this Grafana dashboard and get this going and then pull up the results." And this is a very common thing that a software engineer does, right? And the thing that's hard about it is perhaps not some algorithmic coding thing itself, but it turns out on the setup actually the server that's hosting this is running the wrong version of some package. So you have to go through the errors and figure out what happened and then say, "OK, I need to downgrade the package to this. Other one, which is actually the right dependency for this thing, and then I need to run it and pull this up and make sure the numbers look correct." Things like that, which are basically as close as we can make them to what real software engineers spend their time on.
And so how have the newly released Claude 4.1 and GPT-5 done this benchmark?
Both of them are better at this benchmark than any of the models that we've seen before this week.
As you think about the AI business and industry over the next five to 10 years, like you think about all the different layers of the stack. You have the data centers, and then you have labs, and then you have the application layers such as yourself. Who benefits? What gets more competitive, what gets less competitive? Are all these just classic competitive oligopolies? Yeah, what's the market structure?
So everyone always makes fun of me whenever I say this, but I think all the layers are going to do very well. I think all the— There's just going to be a lot of AI. I think the prices are cheap everywhere. I've been saying this at least for the last six to 12 months, and I think, we've seen prices go up at decent bit across all of these. But no, at a high level, first of all, there's going to be a lot of AI. It can't be understated in the sense that, I think we're kind of coming off of a decade of a lot of various, B2B SaaS and so on. I think there was the internet obviously in the '90s and early 2000s, and then there was the mobile phone and cloud, which were kind of like late 2000s, early 2010s, right? And those were some of the biggest things in the last 30 years. Over the last 10 years or so, there was a real time where most of the stuff that was being built was a lot more incremental. Each next thing and building for a particular niche or for a small part of the workflow and making that more efficient. And AI now I think is the total opposite of that in the sense that, now we're talking about the entirety of knowledge work and perhaps the entirety of physical work as well, depending on what happens with the robotics, right? First thing is there's just going to be a lot of AI. The second thing about where does the value accrue? And my honest answer on that is, value accrues wherever there's meaningful differentiation in the layer, right? Simple, like if there's NVIDIA and there's TSMC, for as long as NVIDIA needs to work with TSMC and for as long as TSMC needs to work with NVIDIA, of course there'll be some rubbing up on each other's shoulders, but like they will continue to do great, right? And you kind of see this down the stack as well. I would argue that the problems that are being solved in all of these different layers are very, very different problems that have pretty meaningful differentiation.
You're saying this prevents too much vertical integration, basically, where the layers keep each doing their own thing?
Exactly. And I think there's a real diff where, as soon as you go from hardware to, obviously foundation model training is its own can of worms, and very much like the DNA of the company is finding exceptionally strong researchers, giving them as many GPUs as you can afford to give them and setting up a culture that orients around that. Then the application layer, I would say, is really focused...I'd say obviously it has a lot of the elements of research as well. But I think in particular is really,
Really focused on just figuring out how to make one use case work for us. For example, the only thing that we care about is building the future of software engineering. Maybe one thing I would call out is, people often talk about AI code abstractly in a vacuum. I think there are a lot of companies that think about code in the foundation model layer or things like that. We uniquely really think about software engineering, and all of the messiness that that comes with, and all the product interface, and all the delivery, and the usage model, and of course like a lot of these particular capabilities that come with that. So I think there's a real... Everyone has their own DNA, and everyone has their own things that they do best.
That makes sense. We at Stripe have been thinking a lot about building the economic infrastructure for AI and what is required. You can have an agent acting on behalf of a person, and you want to be able to just be prompting or doing stuff in your app. And part of the tool use that your AI can engage in is going off and conducting commerce in the real world. So we're building infrastructure for that. And then we noticed that because of the economics of AI, everyone has usage-based models, right? Per token, per what have you. And so we're building out usage-based billing infrastructure, and again, we find the billing systems people are building on Stripe, they're very different from the... Classic SaaS is per-seat pricing, whereas again, everything in AI is per-unit consumed. And you can get into how the agents engage in commerce with each other, where there's no human in the loop. So there are all these ways in which our product roadmap is being informed. But I'm curious what you think the economic infrastructure for AI needs to look like. Are there things that we should be keeping in mind?
Yeah, for sure. Yeah, seat-based to usage-based, big one for sure. I think on both sides, right? From the perspective of one, seats don't really make sense when the AI themselves are arguably seats as well, they're doing a lot of the labor too. And then on the other side, usage obviously just goes so naturally with the COGS themselves because a lot of it is effectively GPU spend on how much you're spinning the models, basically. I think that makes a ton of sense. The other big one which comes to mind obviously is just for there to be an entire agent economy as well. Today I would say is still probably more of a talking point than a reality. But things are pretty rapidly changing and getting to the point where your agents are... Funnily enough, we use Devin. Devin is obviously entirely focused towards software engineering, but we order our DoorDash on Devin, we order our Amazon packages with Devin. There are pieces of that that turn out to work nicely anyway.
So you order your Amazon packages with Devin?
So you're just in Slack and you ask it to buy something for you?
Yeah, just "@Devin, can you go buy some more whiteboards for us?" or something like that.
At a certain point, do the real-world things you ask Devin to do run into just blockers with sites trying to block bot activity?
A lot of Devin working really well, obviously, it relies on Devin being able to do these things and get through it. But some of these things, I think, are quite natural with the model, which is, you often have API keys or secrets or things like that that you want Devin to be able to hold onto. So that works for credit card numbers as well. Obviously there's a lot of work of... Real-world software engineering doesn't involve a lot of just going and browsing the web and finding different sites and clicking around on them, even if you're just testing your own front end or putting a documentation or something. Good browser use is an important piece of that as well. And I think it's just kind of something that's—
So shouldn't you build a consumer app? Doesn't everyone want this magic wand app where you can just have your virtual assistant? There's a million virtual assistant startups. It seems like none of them have really gotten to any scale.
Yeah, it's a fun question. I think from our perspective, on the one hand, it's fun seeing Devin go and do these DoorDash things. At the same time, we also just know that our team is so small, we just don't have the focus to be able to do that in addition to doing software engineering. You're pulling up Devin and you're seeing this, and then on the other side there's the IDE there, but Devin's just going on DoorDash or something. It's a very fish-out-of-water experience, and it's fine for us to keep—
