If you're not using AI, you are just slower. The future of software engineering is you telling your computer, 'Here's exactly what you need to build and it just happens.'
It's a trillion dollar market. How do you think of the possible value and opportunity of something like Devon?
We can do not just like 10% or 20% gains, but more like 10x.
Why is it all these math people are so good at AI?
I go to Glasgow Middle School and I'm in seventh grade. What is the value of 255?
5,000. 5,000 is the correct answer.
This company you're building, you're hanging out with these top minds in math and physics and competitive programming.
So we do a lot of math games for sure.
First person to solve it in the comments, we're going to send you some swag. Scott, what's a harder one?
So, coding is honestly the fastest growing use case of AI out there.
We've been spending our whole lives in Minecraft survival mode and now we're going to get to play Minecraft creative mode.
Scott Wu was the world champion in competitive programming at the II after winning multiple gold medals. He used to work for me 10 years ago at Adapar and today he's the CEO and founder of one of the fastest growing AI companies in the world.
He's surrounded by a lot of the top minds in math, physics, competitive programming. They're building one of the fastest growing companies I've ever seen.
What do the top minds in the world think about AI? Where's the world going to be in 5 or 10 years? And can you solve Scott Wu's math problem?
Welcome to American Optimist. Really excited to have my friend Scott Wu here with us. Scott, you're the CEO and founder of Cognition AI. You were the world champion at the II competitions, top math competitor, and your company's doing awesome. Thank you for joining us.
Yeah. No, thank you so much for having me. It's exciting to be here. It's for a few reasons. First of all, I am actually an American and an optimist, so it's exciting. But also it's — you know, we've known each other for like 12 years now, and Joe actually gave me my very first job. I was an engineer at Adapar 12 years ago.
You're a superstar at Adapar. It's cool to see you go on to build like one of the giants in Silicon Valley hopefully, it looks like. But let's step back a bit. So about a year ago, you guys released a demo of Devon, which is what the product of Cognition is called. Did you have any idea it would go so viral? Like what surprised you?
I mean, it's — look, no, not at all. We hoped it would go well. You know, we put a lot of energy and effort into it and it was really awesome to see.
I mean, I think we have always had the view of just skating where the puck is going, basically. And around end of 2023 we were working on AI and specifically AI for coding and we felt that things were going to shift a lot more agentic, basically. Everyone's talking about agents now but back then it wasn't really a term. You know, at a high level, we had the view that instead of pure text interfaces, there's going to be a lot more full interaction where AI systems could not just read text and answer questions but actually just go and interact with the real world.
And in code, obviously, that means debugging, pulling up the logs, reading documentation, testing code yourself, and so on.
Well, it's amazing. It's a trillion dollar market. I want to dive more into that. But before we do that, Scott, let's go to your background. You're from Louisiana originally, right? That's not exactly the epicenter of the math and programming world. Like what made you interested in these topics?
Yeah, your dad is from Louisiana as well, right? You spent some time there, managing a plant, actually, when he was younger.
That's true. Yeah, similar story for me. Both my parents were chemical engineers. They immigrated from Shanghai in the 80s and yeah, naturally there's a lot of oil and gas emissions and so on. And so my parents both worked in basically the emissions industry in Louisiana.
I grew up in Baton Rouge and it was — yeah, not a lot of people who did math and programming in Baton Rouge, but I always loved math. I had an older brother also that did a bunch of math and programming competitions and first got me into it. But yeah, it's something that I knew I wanted to do from very very early on.
So in our family, my dad ran a plant there and he was a chemical engineer too, is why he was there, and then my wife's grandfather ran an oil plant there.
So you do get a lot of talent, I guess, coming through that way, but it's not seen as a center for these things. So you got into competitive programming. How old were you when you first started doing that?
So, my first math competition, I actually still remember, and my first math competition was in second grade. So I was like seven or eight years old. There was this — you know, the local college held a math competition for like middle schoolers and high schoolers. And so I was competing, I think, in the seventh grade division. So it was like me and a bunch of seventh graders basically doing math competition.
What kind of problems was that?
It was like, you know, it's a lot of the typical stuff for seventh grade math is probably things like, I don't know, exponential or something, fractions or whatever. You know, like the standard problem where the frog is at the bottom of the well and goes up and falls down every night and stuff like that, basically.
But yeah, it was — you know, I remember it because they were calling out the awards, and so then they called out, you know, third place and second place and third place, and none of them was me. I didn't win, you know, and I was really — I was like 8 years old, but I was really...
Really disappointed. I was really upset about it. And so then the next year I trained a bunch. I competed in algebra 1 the next year when I was in third grade. And then I won and then I kept doing math competitions from there.
And then programming competitions, it was — I started around middle school. Around sixth grade. I learned to code when I was in fourth grade or so and then I started doing some of these programming competitions when I was in sixth grade and it was — I always loved them.
So as an aside, right now in the Bay Area, I think you have a lot of the top competitors happen to be Asian, happen to be based in the Bay Area, but it's interesting because they're actually dropping a lot of the honors courses in math and other things in the Bay Area. Have you followed this? What do you feel about that?
Yeah, I mean it's sad. I think — I'm only here because of advanced math classes, is kind of what I would say. And I think growing up in Louisiana, you know, folks were really nice and really supportive. Honestly, there were a lot of people basically at all levels that kind of allowed me to, you know...
I was taking — I grew up in — I went to a school called Buchanan Elementary School in Baton Rouge and it was right next to this high school called McKinley High School and because of a lot of hard work from folks, basically things were set up so that I could actually have someone walk me over every day to go to the high school to take math classes at the high school.
And it's obviously — I think it all obviously starts with math and with just technical education. And so no, it's very near and dear to my heart.
My friends and I actually got walked also to go to the higher school. So there was a group of us, six of us, and they helped me build Palunteer later. So it's interesting — I don't think we were quite as advanced as you, as many grades, but it was really important. And I'm told nowadays they wouldn't allow that to happen anymore. So I guess equity means that they have to pull us down now, which is too bad.
But you were crushed in these contests. Where'd you go to college? How'd that work out?
Well, so before I went to college, actually, I worked at Adapar. I was there for a year and a half.
Why did you do that? Why didn't you go right to college?
Yeah. So it's after high school. You know, coming out of it, I had done a bunch of these competitions. I think at that point already, I really felt like I wanted to start my own thing at some point, which is funny because I don't think I knew what that meant, you know. There was apparently — people, I don't remember this but people tell me this — that when I was in fourth grade I had a math teacher at the high school and I told him that I was going to start a really big company someday and then I was going to give him a position as a software engineer.
You still have the position, Scott?
Well, we got some work to do to get there first. We're going to build the big company first and then he'll have the software.
But it's — yeah, I obviously had no idea what that meant. So I'm not sure what I was talking about at the time. But yeah, it was really exciting to just see technology move and see real companies get built and I wanted to learn what that was like and what that was in the real world. And so I took a year off — a year plus both of the summers around it. And I worked at Adapar and it was a lot of fun. It was basically — I was on the computational architecture team, was figuring out how to compute a lot of big like financial metrics for a lot of these...
Well, we have $8 trillion on the platform now, so hopefully some of your insights are getting used at scale.
I appreciate it. Yeah. And no, it was a lot of fun. And then after that I decided that after all I would go to school and try out school for a little bit. I wanted to less so, I think, to learn software engineering and more so to basically meet interesting people, take classes and just kind of develop personally. I was at Harvard for about two years and then I dropped out and started my first company and it was a great experience.
And your first company was called Lunch Club, which I remember writing a small check and you saying 'I think it's a bad idea,' but actually you figured out eventually something really good there. But tell us about that.
Yeah. Absolutely. So Lunch Club — it was, you know, we started in 2017, which is way before kind of generative AI, but the whole idea was we used AI to make the right professional connections. And so folks would come in and they would tell us, you know, their interests and their goals and then they would also give us access to their contacts and their calendar. And based on that we could figure out who were the right people for them to meet.
It was all in-person meetings up until COVID happened and then we had to support video meetings as well, which was a fun journey.
But yeah, I was there for about 5 years. You know, we made millions of meetings. It was a really fun journey and I feel like it taught me a lot of the lessons that I take.
And Vlad who you built it with was obviously someone who's an amazing recruiter of talent and I think he's pivoted to something that's working so no surprise he's going to make it work. What was the inspiration for starting Cognition?
Yeah. So, it was a fun journey. Actually, a lot of it came back to these math and programming competitions that I did when I was a kid. And so I was still super, super tight with a lot of these folks. You know, we would do these math competitions in middle school. We would do these programming competitions and math competitions in high school. You know, a lot of folks went to either MIT or Harvard, so people were pretty close by as well. And then naturally many of those people were working in AI.
And so, you know, there was a core group of us who we had all spent our last 10 years basically building in AI and thinking about AI and building different companies. You know, one of my co-founders, Steven, was the first engineer at a company called Scale AI, which is doing incredibly well. My other co-founder, Walden, was the first engineer at a company called Curser. And yeah, so we had all done a bunch of different things and there was a point where we basically all got together and said, 'Let's try and just do some projects together.'
It wasn't a company at the time. It really was just more of like a research project, you know, messing around with some ideas. We rented a house here in the Bay Area and then just got a bunch of people together for like two weeks and hacked on stuff together. That was November of 2023. And then in December we got another house in New York and we got everyone together and hacked on a bunch of stuff. And then in January we got another house and in February we basically just kept doing the hacker house model and eventually, I guess, we incorporated a company, but that really came later.
That's awesome. And you know, you really obviously brought together an all-star cast. You mentioned founding engineers at all these top companies. These are your smartest friends from the math, from the competitive programming world. Like why is it all these math people are so good at AI? Is this just raw IQ? Like what's going on here?
I think it's — to an extent it's always been the case in tech, in Silicon Valley, you know, that a lot of the smartest people — you know, one of my favorite things: Mark Zuckerberg actually was a competitor on Topcoder. You can find his profile on Topcoder. He was a competitive programmer. And it's a lot of the initial folks who built that with him, like Adam D'Angelo for example. Adam D'Angelo was like an international medalist at competitive programming. And so it has always been here to an extent, but I think it's especially true in AI. And it's actually crazy to the extent to which it is. I mean I think like OpenAI, Anthropic, Google DeepMind — a lot of these folks, the folks running all these big labs, and then of course the folks running a lot of these startups are actually people that I've known for more than a decade now just from these communities.
I think there's a couple reasons for it, but one thing I would just call out is I think the balance in AI does just inherently swing a lot more technical. You know, there are a lot of purely technical problems that if you can just solve these really, really hard problems, it is incredibly clear, you know, what happens and how you turn that into a business.
I want to stop you there and get one of these — not all of our audience is going to understand, but try to start at a level that's like advanced but not crazy advanced. Like what's one of these hard problems you've solved? Can you tell us about this?
Yeah, for sure. So in these olympiads, for example, a lot of the problems that come up are things like, you know, you're in a network grid and you have to go find the shortest path between a bunch of places, but certain roads get blocked over the course of the day and you're adjusting and you're still solving for these paths over the course of the day, for example, and you have to support all this and make this run really, really fast, right?
That's an NP problem, right, as I'm told, right?
So this one, actually, it turns out there's some really sick things that you can do. These are the kinds of problems that folks think about and the cool thing is that every problem is unique. It's not about being asked something that you've already seen. It is all about coming up with a new algorithm, which could be based on things that you've studied or understood, but it really is like doing the first principles thing.
So just for intuition again for the listeners, why was that road problem tied to a real life thing that matters? Like, give us some more. You're obviously not doing literal shortest path problems in AI, you know, but...
I think that a lot of the purely technical problems that exist in AI are a lot of these things of training AI models that can solve really tough problems or understanding exactly what the right architectures, the right systems are for AI to be able to go and do these things.
And I think with software engineering in particular — I'm obviously a programmer myself and so it's a fun thing to get to spend time on, you know, teaching AI how to code as a programmer nerd myself. And it is like, I would say a lot of it is really like thinking about thinking, you know. It's how do we — somebody hands you a bug and how do you fix the bug? And maybe the answer is, you know, you go and run the code locally. You reproduce it yourself. Once you figure that out, you go and find the error in the logs. You look at those files and you understand it. But a lot of it is teaching AI systems to do those same things, you know, and there's a lot of kind of thinking about thinking that goes into it.
I love it. I feel like your social IQ has gotten like a lot higher the last 12 years and you're being really careful not to tell us anything that's too complicated for us not to understand. But I want you to go past that for a second. Like what's an example of something that not everyone's going to understand that you solved with this?
Yeah. Absolutely. I think the broad — in generative AI, you know, everyone talks about kind of the whole generative AI wave. I actually think it's two waves. The first wave was kind of what I'll call imitation learning. And the second is more like RL, reinforcement learning. And the really cool thing about RL is you basically can...
I think the natural conclusion of RL, which is what we're kind of getting to, is you basically can solve any benchmark, which is insane to think about.
Which means like if you have a clean set of environments, if you have a good feedback loop to decide what is correct or incorrect, you have all the necessary context that you need to make those decisions, you can train a model that just aces that. Which is, you know, we're seeing that in practice on a lot of these things actually.
Funnily enough, a lot of the benchmarks that people evaluate models on today are actually just the math and programming competitions themselves that we all did when we were kids, which is partially maybe because all of us are math and programming nerds, but partially also because it's, you know, a clean way to just kind of see how good the models are at reasoning about hard problems. But yeah, it's one of the cool things there is, is it just means that, you know, if you think about what happens next, a lot of what happens next is really just like understanding and really defining the benchmark. And so in practice what that means is figuring out exactly what are the use cases and tasks that we want to solve for, what are the right set of tools that an agent should have access to, you know, how are you going to evaluate success or failure?
You know, one of the reasons that code is growing so quickly is because you have this really great feedback loop of success or failure, right? In healthcare or in law or something like that, it's a lot harder to say whether you did the right job or not. And code — you can run the code. And so that's often the feedback loop that gets things to work, but that's kind of the core.
Scott, not all of our listeners are in the computer science world here in Silicon Valley. What does Devon do? Just basics, what is this?
Yeah, so Devon is an AI software engineer. And basically what that means is it's an autonomous system that you can work with the same way that you'd work with a human engineer. And so you can assign a bug to Devon. You can give Devon a task. Devon will go — it'll know your codebase, it'll be able to test itself, it'll be able to iterate and work on things. It'll produce a code diff pull request for you to review and look at. And you basically work with it as your own junior engineer.
So Devon's being used by a lot of large companies. I'm noticing there's a lot of large enterprises doing like these big giant, you know, multi-billion dollar adoptions and whatnot. And how do you iterate and figure out what's working or not? Like what are the milestones to see what's working or not that you're learning from in these environments?
Yeah. For sure. So with these, a lot of the general setup is, you know, we want to look for the right use cases where we can do not just like 10% or 20% gains or 10%, 20% ROI, but more like 10x. And so we pretty routinely see spots where we can make things like anywhere from 6x to 20x more efficiently.
What does that 20x mean? You built something over the weekend that would have taken you months, is one example.
Exactly. So it means every hour of an engineer's time at that company on this particular use case using Devon is equivalent to 6 to 20 hours of an engineer's time not using Devon. And practically, obviously, what that looks like is basically just reviewing the code that Devon puts out and just making sure things look good and then just kind of signing off on that.
But in practice, you know, in enterprises, a lot of what that looks like is — the cool thing about it is it turns out it's a lot of the kind of more tedious, repetitive stuff that human engineers don't want to do, which is kind of a nice — I think there's this really beautiful synergy of AI and humans, which is that AI has always kind of been good at the more boilerplate side of things, right? At taking on these more repetitive tasks, which are also nicely — the humans want to be able to work on the higher level strategic decisions or the creative side of tasks.
This is interesting though because this is man-machine symbiosis originally with like Licklider in the '60s — you do the automatable things with the computer and then the higher level with humans. And in chess we always talked about this, but then eventually it got good at everything and it was better than humans at everything. And there's this funny joke where if you're not married yet — like the husband and wife will agree the husband gets to make all the big important decisions and the wife can handle the small things, but then after like 60 years of marriage the guy says, 'It turns out there were no big decisions.' And so it's like if you give something the small things, eventually it's going to get everything right. It's like how do you think about that?
Yeah. No, it's — you know, Andre Karpathy has this phrase, jagged intelligence, which I really love. And it basically just means that, you know, you can think of humans, with our intelligence, we're kind of uniform, or at least from our perspective we are, where as a kid you grow up and you kind of just get smarter and smarter in a lot of different fields. Whereas with AI, there are certain things which it is incredibly superhuman at and then there are certain things that it is incredibly bad at.
Right. And one of the cool things about jagged intelligence, I'll just say, is it means that humans plus AI really is the way to go, at least for the foreseeable future, right? Because I think there are certainly — there are more and more things that we're seeing that the AI is just able to do incredibly, incredibly well. But as long as that kind of tail exists of all the things that humans can do, that is what we'll own. And if anything, we'll be able to just do a lot more.
How long do we have, Scott? How long till it catches up, too?
I think the long tail will take quite some time. And I think there's some things that are honestly inherently human. You know, I think it'll always be up to us to decide what to build, for example.
And I kind of think that the future of software engineering is basically that. You know, we talk about programming languages and architectures and all these things, but at the end of the day, the whole point of software engineering is just to tell your computer what to do, right? And you should be the one deciding what to do and you should be able to express that cleanly and just have it built. And I think that's kind of what we're going to get to over the next — I'll call it like five or so years — is a point where software engineering, we might even still call it software engineering or programming or whatever, but it really is just you telling your computer, 'Here's exactly what you need to build. Let's add a new page on this website or let's add a new table and store this and that information and just make sure this is ready to load, or this button could be a little bit rounder, let's go and do that' — and it just happens, you know.
