Together Talks | Ep 3: Scott Wu on Agents, Cognition AI, Competitive Programming, and the Future

Thank you for having me. I'm excited to be here.

Yeah, you know, I think I was always very, very competitive as a kid. My first math competition was actually in second grade. I was like, you know, seven or eight years old. I went to this local math competition. It was a competition for middle schoolers, actually, so I competed in seventh grade math. And then I remember this very vividly still. I went and then I was like, you know, I loved math and stuff. I did the competition and then I didn't win a prize, like third place, second place, first place, and then none of them were me. And then I was just really upset about it. And then I trained a bunch and then I came back the next year and then I did like it was like algebra one or something the next year and then I won. And then it was like ever since then it was always just, you know, math competitions, programming competitions were kind of my whole life, honestly, like as a kid. And yeah, I mean, what are the main things I carry over? You know, honestly, I think a lot. I think there's obviously some, you know, the math itself and the computer science itself. You know, there's a lot of kind of just like the technical details that are obviously all kind of correlated. But I think even beyond that, you know, just the willingness to be very brutally honest with yourself, I think is very important. You know, the ability to just examine your own flaws and be ready to work on it. Obviously a lot of the kind of the general things of working hard and, you know, figuring out how to really put your best into it and to go for success. But yeah, yeah, yeah.

Yeah, I mean, so a lot of our team actually, you know, our whole initial team was kind of, you know, many of us knew each other for many, many years, actually. A lot of us first met in these like math and programming competitions. And, you know, we wanted to work together. And a lot of us have all had our own journeys in AI, you know, for the last like five or ten years. You know, one of my co-founders, Steven, was very early at Scale AI. My other co-founder, Walden, was early at Cursor and also had messed around with a lot of his startup ideas as well. I personally built a company called Lunch Club. And the whole team was kind of like that, you know. And yeah, I mean, I think we had a lot of ideas of what we wanted to do. I think code was obviously very, very exciting to us from the beginning. I mean, we all love coding, you know, we're all engineers ourselves. But also, I mean, I think code is, I think, you know, one of the most powerful direct applications of better reasoning and better logic, right, out there in the world. You know, I think code is probably one of the main drivers of progress, honestly, in the world, you know, in the last fifty years, call it, right? And so, yeah, so that was always really exciting. But I think more than that, I think to your point, like, we, yeah, I mean, we were really excited, I think, to kind of explore much deeper what it would look like to not just have a coding assistant, but actually, you know, a more autonomous decision maker. Like, I think one of the other big things about code is that it is honestly inherently a very agentic task. You know, in theory, for example, you can have the whole codebase and have an explanation of, all right, here's what I need you to do, and you just spit out a perfect diff and it's just correct. But obviously no human can do that, and, you know, at least today no AI can do that either, right? And in practice, what coding is like is more like, you know, you read this file, you read that file, you pull up the documentation, you know, you run the code yourself, you test it out, you add some print statements, you debug, you know, you pull up the front end, you know, you do all these kind of like iterative things, and then you come out with a diff that hopefully builds what you want it to build, right? But like, yeah, I mean, I think naturally having agents that can do that was something that was really, really exciting to us from the beginning.

Well, I actually think they're quite complementary. And so, you know, I think a lot of these products out there, like the Copilot-style products, like you're saying, I think are actually really great products. You know, you're coding and you're implementing things, and it gives you the suggestion, and you can autocomplete, right? And kind of all the little tools, you know, like that, are super, super useful and make people faster. I actually think it's, yeah, it's very complementary in the sense that I think for certain tasks that you want to handle yourself and you want to be, you know, very specific about how you're going to implement it and do it, I think Copilot and the kind of assistant products are the tool for you, right? Because it will take place in your environment, in your IDE, while you're building. Whereas I think the way I almost describe agents is more like tasks that you want to delegate entirely, right? And so maybe one way to put it is, you know, Copilot-type tools make everyone a faster engineer, and agent tools make every engineer into almost an engineering manager. You know, you have your own team of interns, and you can assign your interns tasks. And so, yeah, you know, I think for quite some time, you know, I think we'll have both of these, actually.

Yeah, you know, I think it's going to change a lot. I mean, I think the, and by the way, I think it's always changed a lot. You know, I think that is one thing that we often underrate. You know, it wasn't that long ago that programming, you know, was like punch cards, you know, and you go and like fill out the right, and that was what programming was, right? And then somebody invented assembly, you know, and assembly, when that happened, there was this question of like, well, what happens to all the programmers? And of course, the natural thing was there's actually even more programmers, and now they write assembly, and they can do way more things, right? And then we had, you know, C, or, you know, there was COBOL or Fortran or whatever, and eventually now we have, you know, Python or things like that. We have frameworks and clouds. And I think the, I think it's interesting because I think the history of programming has kind of been like an exponential curve of programming getting more and more effective, maybe I'd put it, but also of there being more and more programmers, right? And so I think in the long run, what does a software engineering job look like? I think it's, you know, I think even today, what makes someone like a really great engineer, if you say, wow, this person is a really amazing engineer, usually you don't mean that they type very fast. I mean, obviously that's nice too, and sometimes it's cool, but I think the biggest thing is like the ability to really, you know, to solve problems, to break things down, to have really great intuition about what works and what doesn't work, to build things in a way that, you know, is very like future-ready and understand how things might change in the future, right? And I think all of those things are still going to be really important. And basically, I think the engineers of the future will be more like, yeah, almost like these technical architect types, or halfway between a technical architect and even like a product manager, right? Where, you know, you get a problem and you're just thinking about how do I break down this problem, what exactly is what I want to build, you know, what are all the details and flows and the cases, and, you know, how, what is the solution that I want to build exactly? And once you can describe that, then it'll just be built for you, right? So, yeah.

Yeah, yeah, or eventually you'll just have it, you know, Neuralink plugged in where you just think it and then Devin will build it for you. Yeah, I mean, I think like, um, maybe the way what I'd say is, um, I think it's going to be pretty different from anything that we've seen so far. Uh, and I do think, you know, in general with AI, um, I've always loved this quote where people were saying, you know, like the first intuition with AI is thinking it's going to make better movies, but I think the reality of what's going to happen is we're going to make something that's like a totally different experience that offers what movies today offer, right? And it's going to be different. It's going to be like way better and more capable, right? And I think the same is true in general. And so, you know, I think the closest comparison today, I'd say, is kind of like having your own team of interns who can just do whatever you tell them. But of course, I think there are a lot of things about it that are going to be totally unique and, you know, kind of things that we haven't seen before at all. But yeah, I mean, I think at a high level, it does basically look like that, like a very conversational nature where you can kind of, you know, whether it's voice or typing or whatever, but like you can kind of just dictate, okay, um, you know, we're going to have this page and it's going to look like this. And, you know, if you click this button, it goes here, you know, if you're building a front end, right? Or if you're building, you know, figuring out how to architect stuff on the back end, you're going to say like, okay, I want my database to be this technology and I'm going to set it up with X and Y. And, you know, here's how I want the tables to set up. And then kind of just be describing that out. And yeah, one of the cool things is that you're going to be able to dynamically make changes easily, right? And you can say, oh, wait, actually, let's change this up a little bit and let's do that, right? Or let's add this thing, you know, um, and you'll just be able to say those things and have those be implemented, right? Uh, and so I think the, I think at a high level, you know, it's kind of like, I think Devin, at least for quite some time, you know, Devin is not going to be the decider of what to build, you know, and so I think that still rests for in human hands, right? And from there, a lot of it is just making it so that, you know, software engineering is as easy as it arguably should be.

Yeah, yeah, yeah. I mean, I actually love thinking about this stuff because I think the, you know, as you can imagine, you know, we explore like a lot of these different ideas with like multiple agents and things like that. And, you know, similarly to the previous one, I actually think it's the kind of thing where our intuition obviously is like human organizations and how different humans interact with each other. But, you know, with agents, like a lot of these paradigms, you don't have anymore. And so, you know, one of the big things with humans is I'd say there's kind of this coupling between the action agent and the context, right? Where it's like, you know, one person knows one set of things and is acting kind of serially based off those things and they can communicate with each other, right? Which is a paradigm that we could use. But I think within agents, there's a lot of things, you know, the obvious things like if you're asking, hey, like, can you find where in the codebase, you know, is like, you know, like, I don't remember exactly what it's called, but I need the variable that, you know, corresponds to this gradient or whatever, right? You know, a human will go and click through different files, maybe go through and do some fuzzy search and stuff like that, right? But it's all serial, right? Whereas with an agent, you can kind of just spin up ten different agents that are all acting and all, you know, kind of searching in parallel to figure it out. And not only that, they can actually share the context with each other, right? And so it's kind of like, I think you will be able to do things where you have like a lot of parallel agents, maybe some with different capabilities, but they actually all share kind of underlying knowledge and context and, you know, kind of are able to share learnings that way. Yeah, that's my sense.

So I think the, it's an interesting question. I would personally guess right now that they will share the same expertise. Obviously, you can imagine having targeted models for each one. I could see it either way, but if I had to guess, I'd probably guess the former. But I do think there are a lot of other kind of details that might change, right? So, you know, a very simple one is there are a lot of tasks that perhaps you can do more straightforwardly with just pure execution where, you know, you don't need the most compute-intensive model, right? And there are some tasks that are kind of the really deep planning ones and seeing things very far in advance where you want the most intensive ones, right? And, you know, you can have a team of different agents where the agents are, you know, kind of, there are different flavors of the agents, right? And, you know, there's a hierarchy where they're splitting up tasks like in the appropriate way and figuring out how to do that, right?

Yeah, that's interesting. I mean, so the economic argument obviously I think makes a ton of sense. And I think in general, for sure, like I do think, especially over time, just building up the kind of Pareto frontier, right, of different capabilities where like some things again will require like the absolute most powerful reasoning or something, you know, and you will need to pay more for those. But like some things that are very kind of niche, just kind of like knowledge recall or, you know, execution type things that can be done in less compute, I think will be done, you know, with smaller models. Yeah, I mean, I think as far as the specialization thing, I mean, it's interesting. Yeah, I think it depends. I think it largely depends on like, yeah, what happens with like the shape of these transformer models, right? If it turns out that, you know, having a really big model that contains all the knowledge is actually, you know, able to kind of like selectively pull out each of the different specifications, like the specific skills, or whether it makes sense to have specific ones. I honestly, yeah, I could see it either way.

Yeah, I mean, I think it's great. And I think, you know, open source models will continue to grow and to expand. I mean, I think the, my sense overall is, I think, first of all, I mean, I think there's just a ton of growth to come in AI. And there's a ton of use cases more, you know, and tons more to unlock and tons more to productize and everything. But yeah, I mean, I also, you know, on top of that, I think that, I think that the existence of open source honestly does, you know, promote a lot of this and make it easier to do a lot of this. And, you know, I think a couple of the biggest reasons right now, I think, to do open source, one are because of the economics and two because of, you know, people who want the most kind of custom control over whatever post-training, fine-tuning, you know, whatever inference-time tricks or things like that that they want to do. And so, you know, I think those will continue. But then I actually also think, we were talking a little bit about this earlier, but like I think as the number of all the models increases, you know, I think there will be a lot of natural things where several models will suffice, right? And given that, of course, you would prefer to work with the open one all else equal. And so, yeah.

I mean, I think for us, I think the, yeah, I mean, I think the biggest thing for obviously is logical reasoning. And I think that's kind of what underlies both the code but also the decisions around code, you know, that Devin makes. And so, you know, we do a lot of work with this and we also, you know, work with a lot of others in the space on it. And I think the, I mean, I think both in open source and closed, I mean, I think there's, I think we're right at the cusp of actually a pretty big wave of reasoning capabilities, which is super, super exciting. And so I think it's going to be a fun kind of next six months or so when we see the new models coming out.

I think there probably are still a lot of gains to be had from these synthetic techniques of just generating data, and especially the kind of data where you have clear, verifiable ground truth signal, right? And so, you know, I think Google DeepMind recently announced the result where they were able to get an IMO silver medal, right, training models specifically to reason with math and doing Lean. And I think one of the big things about that is that you have this programmatic way to verify proofs, and that makes it way easier to generate a ton of data to get the right feedback signals on what's working and not working. And I broadly think that, you know, over the next few years, we'll pretty quickly get to the point where any field that we have a good ground truth data label for and abundant way to create more data, I think AI will generally be able to kind of get to like peak human capabilities or even above human capabilities on. And so, yeah, I think a lot of the work to come actually is, you know, in things like that and figuring out, you know, the right data loops, the right synthetic data and so on. At least for the time being, you know, I think in the far future, there's obviously, I say far future when I'm talking about six months. But yeah, I mean, I think there's a lot of interesting ideas on, you know, how to build models that generalize better beyond data, you know, which I think there is more to explore there. But I do think we are going to get a lot of gains even just by setting up the right synthetic data loops and just getting these, you know, targeting the right utility functions with these.