The world going forward, there is going to be nothing that no one can build. Everyone is trying to commoditize the other. Value accrual is a time-dependent phenomenon.
Meet Matan Grinberg, CEO and co-founder of Factory. Before Factory, he was a physicist. He spent 12 years trying to be one of the best string theorists in the world. Now he's changing the world of software development. He works with some of the biggest enterprises in the world. He does look like Matt Damon from Good Will Hunting, but he is one of the best founders I've met.
So many of the tasks that we're doing, we don't need the very frontier to do it. We might see a short-term contraction of usage of the very frontier models. I think it's pretty embarrassing that we don't have frontier open models in the United States.
Name a legendary company that has a [__] sales or marketing team.
You can't. The age of the polymath is back. We will see the best companies treat teams more and more like SEAL Team 6 or like professional athletes. Ready to go.
Matan, it is so good to have you in the studio. You've just insulted my continent with the suggestion that we've only come up with bottle caps while you came up with Transformers. Not wildly untrue, but this is going to be a fun show. So, thank you so much for joining me.
Thank you for having me, Harry. It's a pleasure to be here.
Now, I was just doing a show yesterday with Rory and Jason, and Rory was basically saying, you know, the fundamental question is, will we see an increase in GDP coming from AI and the coding developments that we're seeing, and will it lead to GDP increasing above the 2% average for the last 200 years? Do you think we will see meaningful productivity gains from the AI tooling that we're seeing, or is Uber's concerns validated?
So I think yes, absolutely, we will see tremendous growth from these tools. I think it takes time to permeate through because you can tell, like on an individual basis, almost on a problem-by-problem basis, we can solve problems faster with these tools. Now, companies generally are organized around solving problems. And if you're organized around solving problems and you have some set of personnel, you might say this is the number of problems we can solve at a given time based on how many people that we have. Everyone is now going to be able to solve more problems with the same number of people, or solving the same number of problems with fewer people. But it takes time for the resource allocation to adjust. A lot of businesses will have to ask: do we want to solve more problems now because of the increased leverage that we get, or do we want to solve the same problem but now we can do it in a more efficient manner? That's, I think, a question that a lot of businesses will be grappling with.
Do you think we will have fundamentally smaller teams, which ultimately suggests that number two is the option that most people take, or do you think we will have actually the same size teams and we'll just go after a more expansive area?
It's really not obvious because there are dynamics that it's hard for me to predict. But what I will say is, again, bringing it back to problems. All of these companies are now going to have to think: okay, we have all this new leverage. Do we want to solve the same problem? Do we want to increase our ambition and solve a bigger problem, or do we want to solve more problems that our users maybe have?
I was watching Andrej Karpathy and he was talking recently about, you know, the 10x engineer actually is wildly misunderstood and you won't see the 10x engineer, you'll actually see a smaller number of 100x engineers and kind of the rest, and this bifurcation of engineering talent. Do you think that is the right way to look at the future of engineering talent?
Directionally, yes, because what is a 10x or 100x engineer? I don't necessarily agree with the language around it, but I think it just implies as if 10x of what is it? Pure output? Like if you say 10x, it means like how much code they're writing. Yeah, now I can write a billion lines of code with these tools. It might be [__] lines of code though. So maybe the way that I like to think about it is like load-bearing individuals in an org. It's kind of like if you remove this person, things fall. Where there, in some orgs, there might be people where if you remove them, nothing happens, and they're, you know, not load-bearing in that case. And so, you know, basically these people who have very high leverage are now being handed a tool that gives them even more leverage. And so using the language of 10x or 100x, yes, they're levered up, they can have even more impact. And I think, you know, with that leverage language, those who know how to use leverage well will be able to have even more impact, and those who don't will kind of, on a comparative basis, be that much less valuable to a business.
When we think about kind of the two different parts you said there, hey, you have the option of you can do more with the same size teams or you can reduce teams and do what you already did. If I am thinking as a leader today, what would be your biggest advice to me on how I should think about resource allocation for tokens internally?
Yes, this is a great point. This resource allocation problem of tokens — it's not just tokens, it's like dollars, it's tokens, it's people. This is, I think, going to be the thing that over the next 24 months every C-suite is going to be thinking about. And I think the right way to go about it is: what is the core competency for our business? What actually matters for the business that we are doing? And then how do we allocate resources accordingly? In other words, if you're a logistics company, your core competency is probably not software development. Now, you might have added a lot of software engineers as a means to an end to deliver on your logistics goals, let's say. But that might not be your core competency. And so what you should be thinking about is not how do we get more engineers to make more features, because that's what engineers have in the past been judged by — like how many features do they ship in a quarter. Instead, it's like what are the actual output metrics that matter for our business, and how do we now allocate resources — whether it's dollars, whether it's tokens, whether it's headcount — to more dramatically move the needle on that business outcome. And I think this is great for the world because I think part of the reason why so many organizations got so bloated is because we were in a period of time where everyone was focusing on intermediate metrics. If you're an engineering team, we wanted to ship three features this quarter. Did you ship three features? We shipped four. What a great quarter. Like, that doesn't necessarily matter for the business at all. And so now it's like finally coming back to what matters in the first place. Like, what are the business metrics that we want to, you know, move the needle on? Is it customer satisfaction? Is it revenue? Is it market share? And you can kind of tie back every individual's work to that, whether it's marketing, sales, engineering, all of it.
Kirkland announced a $500 million spend. You're friends with Winston from Harvey. Fantastic guy. Who obviously, I'm sure, has — I don't know if you guys have spoken about this actually — but like, that's a big spend. $500 million across five years to internally build their own Harvey or Lora. How did you think about that?
I mean, it's fun, you know, talking about core competencies. Kirkland spending half a billion dollars to build their own AI tools. My understanding is that building AI technology is not a core competency of that firm. I think I was surprised to see it. Now, I actually think this is good for Harvey because there's nothing like trying to do something yourself to make you realize, oh [__], this is actually really difficult. This doesn't actually matter for us to have the in-house ability to build this ourselves. Let's go and have someone who is an expert in this to go and build this for us. That is my sense.
My favorite is also the amount of people that are like, see, we told you how easy it was. And you're like, it's so easy. They're committing half a billion dollars. That would suggest the opposite. I had Brendan on from a call the other day and he was fundamentally saying that the next 12 months would be the most value-accruing 12 months for AI infrastructure companies. We would see that the models, the products, and the AI application layer companies would be most at risk. Would you agree with that?
I would disagree. I'd pretty strongly disagree. For a couple of things. One, actually, sticking with the Kirkland thing. I think as an example, we're so used to a world where moat in software was I know how to do this and you don't, and so you're going to pay me because I have the engineers who know how to build this and you simply cannot. Now, the world going forward, there is going to be nothing that no one can build. Every single piece of software anyone will in theory be able to build. Now back to the resource allocation though. Is it worth your time and your energy to go and build it, or should you go to someone else who has already built it or can do it faster? To me, kind of an example of this is like suppose we had a very busy day at work. I could probably go and pick up lunch for everyone on the team. I know how to do it. I know how to walk out the door, place an order, hold the bags, bring them in. Now just because I know how to do it, is that an efficient use of my time? Probably not. I'm probably going to say, you know what, for my resource allocation, I'm going to pay someone to go and do that for us because at Factory, our core competency is not that the CEO goes and gets lunch for everyone. And I think it's somewhat similar here, which is like just because you can build a lot of these things does not mean you should. And in fact, oftentimes you want to be really ruthless about what are the few things that you and your team own and do end to end. And then if it's not relevant to your core business and your core competencies, outsource it.
What would you like to do, but it's not core competency for you, and so you don't do it because of focus?
Oh man, I enjoy like making breakfast. I haven't done it in like three years. There's nothing like — it's just not time efficient. It just doesn't make sense to spend my time doing that.
But it is, you know, it's something that I do enjoy. But to your point on models, applications, infrastructure, I'm not sure if you've seen there's a meme of the Microsoft org chart and it shows like, you know, different segments and they all have guns pointed at each other. Just to show like in Microsoft, you know, there's a lot of bureaucracy and everyone's kind of fighting for who gets to do what. I think that image is pretty accurate to what's happening right now with the models, the application companies, and the infrastructure companies, where everyone is trying to commoditize the other. Everyone is trying to say, oh no, this one is irrelevant. This — all the value is going to be here, all the value is going to be there. The reality is value accrual is a time-dependent phenomenon. So like it's not like there is one person whose steady state gets all of the value. That's not how it works. It's maybe for this next year, this person is who has the pricing power, who gets the value. This next period of time, these people get it. We are all, you know, whether overtly or not, and maybe I'm saying the quiet part out loud, like everyone is trying to commoditize the people that are not them. So, for example, we're model agnostic. We want to give our customers the best pricing, the best performance, the best speed for whatever task they want to do in their software development. And we want to make sure that OpenAI, Anthropic, Google, Microsoft are all under pressure to make sure they give the best models for as cheap, as quick as they can, and don't feel like they can just, you know, charge whatever they want. Now similarly, the model companies want to make it such that the applications are all trivially easy to build and really the product is the model, and then the infra companies have their own spin on this. But the reality is like everyone's trying to commoditize the one that's not them. And so, you know, from Databricks' perspective, you know, it's very much in their interest that models that have access to proprietary data get differentiated value and capture a huge amount of value because that validates their business model. It's a big push and pull of who can get the leverage.
What is the belief that would invalidate yours?
The bear case against Factory is if one model provider gets significantly better than all of the others. So basically, I think a key thing for us is that all the models are going to be roughly as good as each other. They'll be good at — one is a little bit better at review, one is a little bit better at testing, one's better at Python, this and that. It all kind of fluctuates every week. Like even already, people have a hard time keeping track of what model is number one, what's the latest thing that came out. If one model ends up going way above all the others, that's a case where it's like, okay, we want to put — you know, companies might want to just completely go in with them. But then that's a monopoly for the entire economy to be worried about.
Is the rate of model development sustainable? And what I mean by that is like, you know, I was with the founder of Nebius the other day and he was talking about it on, oh, every few weeks we see new models. And I said, you're wrong. Every few days, especially when we look at Chinese open source, it's like three or four a week. Is that rate of model development a feature of the time that we're in, or is it an ongoing characteristic or trait of this environment?
I think eventually we'll stop seeing them as model releases and they'll feel more continuous. Like just how before it was, you know, GPT-2, then GPT-3, then GPT-3.5, then 4, 4.1, but like, and then you get more and more decimals and like, you know, 4.523. Like eventually they're just going to not announce it and it's just like, hey, look, here's our model that's continuously getting better. Because already people have fatigue. Like engineers at the enterprises that we work with can't keep up with every single model that comes out, nor should they. And that's the whole, you know, the case for the application layer, whether it's us or like a Harvey or whoever else, is we're going to figure out what model is best for what use case, where the trade-off is between cost, quality, speed, and we'll just deliver that to you based on the task that you have, because it's hard to focus on what matters for your business and then also keep track of all these models that keep coming up.
A big question that people have is around the rise of open source and whether everyone is concerned by the amount they're spending on tokens just being so much larger than they thought. Hey, we spent our annual budget and it's May [__]. Maybe we should move to open source. And we're seeing more and more great companies use frontier models, see where they can get to, and then move to open source to get as close to that as possible. How do you feel about that being a considered threat to maiming the market for frontier models?
I think it's a really important counterbalance because it basically allows you to make the trade-offs of what tasks do you want to put what level of intelligence on. And I think it's a really important counterbalance because a lot of enterprises will realize so many of the tasks that we're doing, we don't need the very frontier to do it, and we can do it much faster, much cheaper with these open models. And again, it's part of the resource allocation. And to do good resource allocation, you want to be able to be anywhere in that cost, quality, speed trade-off.
I love the GIFs on Twitter or memes on Twitter when it's like, you know, me naming a file and it's like the massive cigar with the blowtorch. Yeah, I love that.
No, because it's such overkill. But also there's a funny dynamic that emerges, which is there's kind of an ego thing where, oh no, no, the work that I'm doing only a frontier model could handle. Oh, this mere open model can't deal with the work that I'm dealing with. And this is like even admittedly when I first started switching over, I'd be like, I don't think an open model could handle this. And it's like, no, it probably can. And it's kind of a funny thing to mentally deal with of deciding, you know, manually or then having the router do it for you.
My question is, enterprises like security. They like reliability. They like ease. And when you have frontier models which are packaged perfectly, priced clearly, and it's secure, they not just go for that, the easy option over trying to be smart and intelligent routing to different open models.
So, a couple of things. One, it's easy when there's only one of them, but again, as we said, a new model comes out every week, and if you have to go through the full enterprise process to get every new model in, it's not very easy. Two is it's also really expensive, and if you're seeing your costs go up like crazy and not having an ROI case, it doesn't make as much sense. And I think something that's interesting is there's kind of like three phases that we're seeing happen in these enterprises. So phase one, this was a couple of months ago, was board yells at CEO, 'Hey, Mr. CEO, what's your AI strategy?' CEO is like, 'Shit, I don't know.' CTO, 'Hey, what's our AI strategy? Let's make sure we adopt AI.' And so then phase two was kind of AI at all costs, token maxing, part of your performance reviews. We're going to measure how much you guys use AI. Everyone, you have to adopt. That was phase two, right? Get as many people to adopt as possible. Phase two happened a lot faster than people might have expected. And so now we're entering phase three. Like phase two was kind of like the debauchery, the long night, you know, taking shots, having a great time, using all the AI. Phase three is the hangover where you go and look at the bill and it's like, 'Oh my god, we are spending so much. I have no idea what the ROI is. Does this — is this helping our business?' That's where a lot of these companies are at now. And I think this is why routing is so important because they're realizing — and this is a true story. One of the CIOs I was speaking with realized we've been spending hundreds of thousands of dollars per month on people asking Opus 4.8 questions like, 'Hey, how's it going?' Like, what are my macros from the food I ate today? Like, what's the weather like? And it's like, guys, like we don't need the frontier of human intelligence to be doing this stuff for us. Let alone it's not even work-related in some cases.
But will we see a contraction then, given the hangover period being realized?
I think there are — we might see a short-term contraction of usage of the very frontier models. But I think it's healthy. I think it's healthier to do that than to like be blind to it and then have a real sudden kind of change there.
Uber announced last night, I think it was, or yesterday, that they were having like a $1,500 budget per individual. How do you respond or think about that?
We — I've literally seen this with dozens of our customers where initially, and this is a lesson on our post-sales team, where initially we came in, we were like, 'Oh, by the way, we have these user limits, but here these are the models. Go crazy.' This was before we had routing, and it happened a couple of times with customers where the usage would go crazy. They hadn't spent the time to actually determine what parts of the codebase do we want to dedicate these tokens to versus not. And then they were like, 'Oh my god, we're spending so much. This is crazy. We need to put in token limits.' And at first, when we weren't like — the first time this happened, we were like, 'Oh my god, their usage went down. What's going on?' But spending time with them, we realized, wait, we need to make sure with every customer we are having a very clear conversation with them of, you know, it looks like you guys are spending a lot of tokens on some of these things. Have you thought about consciously, yes, we want to do this? Sometimes we'll proactively set in those user limits just so it's better to be aware as you're going up as opposed to just going crazy and then kind of realizing. And so what's happened with Uber publicly has happened privately with a lot of customers of ours. And yeah, there's a little bit of shock where it's like, okay, wait, let's put in these user limits. But then you come into a question of, well, wait, this team is really important. They should have a different user limit than that team. And we're just getting towards this world where you have very nuanced resource allocation throughout your org.
To me, the biggest question that I ask myself, and I think we need to ask ourselves as an ecosystem today, is if Mark Benioff says that he spends $300 million on Anthropic, okay, for his devs, that is 3.8% of salaries. Okay, great. What will that number be in three years' time? Because if it's still 3.8, [__]. If it's 20, [__] again. But [__] positive. And if it's Brendan at MosaicML who says that he's spending more on tokens than he is on headcount, [__] again, but even more positive. What do you think that percent of dev salary is in three years?
I think it's actually a more nuanced question than we might think. I actually think it can be as low as 0% for some individuals, and it can be as high as like thousands, tens of thousands of percent for some individuals.
And what's the dependence there?
It depends on what the unique skills of those individuals are. And I'm saying individuals and not devs in particular because I think the way we even organize roles is going to be very different, where I'm not sure like dev as a word makes sense. Traditionally, it's like custodians of code, right? The people who do anything relating to code are engineers or developers. I think everyone is going to be loosely interacting with code in your org, whether they're sales or marketing. But I think the difference is there are going to be certain people where, again, we're coming back to resource allocation, they get more leverage by using more tokens. And then there are going to be certain people where actually they don't really need tokens at all. And that's not how they deliver value to the business. Like for example, you know, maybe our best salesperson, the way we use them best is not by having them use tokens, but by going and meeting people face to face, right? That's an obvious example because they don't write code in the first place. But I think similarly, maybe there are some engineers who they actually do their best work by spending time with users, spending time with their customers, maybe doing some data analysis that's not very token expensive. But then there are going to be others who are delegating to dozens of droids in parallel working on a ton of different crazy features and refactors and migrations. But I don't think it's going to be a consistent number across the board. In fact, I would argue that if your org has a standard number where it's like we want every engineer to be at this percent of their salary in token use, you're probably painting with way too wide a brush.
If I were to say give me an average number, what will that average be? Like what will the median be?
I would say order of magnitude. It'll probably be comparable to salary.
Yeah. Like on the same order of magnitude.
Within a three-year timeline.
What percent of tasks today being using frontier models could be done with open-source models?
Probably like 80 to 90%. It's typically the planning that really needs the frontier models.
But is that not like the most — I'm sorry. I'm really, you know, dim. That's nonsense. But if it's 80 to 90%, does that not just present the biggest bear case ever against code or Cursor, because you're just taking away 80 to 90% of that time?
Well, it depends because that 10 to 20% could be the most important tokens. It's maybe like right, like 10 to 20% of the tokens, but those are really, really important because it's kind of decision-making tokens perhaps. Sure. But it's very similar to how we structure human orgs, which is like oftentimes leadership makes very key decisions that determine the fate of the company, and they don't spend the most hours. Like if you look at the human hours of a company, most human hours are not spent on making the decisions. They're on gathering data or implementing things, but then there's a select few hours where it's like here is where we're going to, you know, make this irreversible decision on the strategy. And those people that make those decisions are also typically paid a lot.
Yeah. But the assumption there would be then that you'd have to increase that spend for that 10% even higher.
And that's what's happening already. It's like the frontier models are sometimes they're getting more expensive or you're using the ultra-high reasoning or you know this type of thing. And so it's like okay, if this planning thing is the very key thing, we'll spend on it, but it doesn't necessarily mean that most of your tokens are going there. It's just for certain key steps maybe you want to spend a lot and it's worth allocating the budget there. But then once it comes to okay, we have the plan now, let's implement, the open models are typically really good.
When we think about what you're willing to spend on, I asked Brendan how much it costs to hire great AI researchers, and he was like tens of millions of dollars. Have you found the same, and is it impossible to hire great AI researchers in competition with Anthropic and OpenAI?
We are a very opinionated organization, and so it's very self-selecting — the people who like the opinionated stances that we take are willing to not necessarily go and try to, you know, maximize the dollars that they can get out of in the market. That said, it is still, you know, pretty competitive.
What do you think is the strongest opinion that you have that most people disagree with?
I would say the opinion that we have that I think in the space that we are in is the most controversial is the way that we treat what product is at Factory. I think there's some very commonly held beliefs at the labs or at some of our competitors who are also doing kind of software development. And this is honestly growing up in the Bay Area, there's a very common Silicon Valley fallacy, which is there's like research is like the pinnacle. And then there's engineers who implement the research. You know, they're not quite there, but you know, they're still great. And then there's sales and marketing and all that dirty stuff. Oh, if only we could build a better product and it would sell itself and we wouldn't need to deal with, you know, sales and marketing. And it's just completely delusional. Like the product at Factory is the entire journey from the very first time they hear our name till their 10th renewal after a decade of being a happy customer. Now the software is a big part of that journey, but so too is the marketing that we do and the people that we have running that. Same with the sales process. The people that present themselves in discovery calls or in demos or in solution engineering. Like that entire thing is the product. Everyone is first class. It's not like we have engineers who are like holed up at the office and you're not allowed to speak to them unless you're an — It's like no, no, no, we have engineers and salespeople sitting next to each other. There are no like engineer corner, sales corner, any of that stuff. It's like everyone is completely intermixed. When salespeople close a deal, engineers say we closed a deal. When engineers ship a feature, salespeople say we shipped a feature. It is entirely one team, entirely cohesive. Everyone — there's no like first class or second class. And this is shockingly controversial. In particular in the Bay Area and in particular in coding or in AI, it's messed up. It's so messed up. And I think that the reality is it will come to haunt some of these companies one day because I think right now where there's a gold rush and everyone's like desperate to sign, you know, and get more tokens from these people, it's easy. They're kind of like, in my mind, it's kind of like they're astronauts in space where there's no gravity. Your muscles will atrophy. Gravity will come back. And if you don't have a good sales and marketing team because you don't give it respect, the second gravity returns, all of your muscles will be atrophied and you won't be able to compete. And I would say this: name a legendary company that has a [__] sales or marketing team. You can't.
No, but I can name companies that have [__] products but great sales and marketing teams. That's the ironic thing on the flip side. And in fact, it seems like many more legend — like I'm not going to name them because I'm going to jumble.
I'm sure we're thinking of some of the same ones.
I mean, yes, if Chad Pets were here, he would say them.
Does what it takes to be a great engineer change when you essentially become prompter and manager of agents versus creator and doer of tasks?
Yes, it very seriously changes. And this is actually why we're selecting for very intentionally — like this culture that we just mentioned is really important because the best engineers are going to be the ones that don't see sales and marketing as dirty work but as again an important part of the product. Because as an engineer, you're no longer, you know, just your job is ship feature. It's no — you are owning full end-to-end outcomes of here's the way the customer is behaving. Here's how maybe we can change that behavior that makes them, you know, a better user long term. It makes them more agent-native. They get more out of our product. We can then follow them through that journey. Enable the salespeople so that they know how to talk about it or they know how to demo it. This is like a full-stack engineer that goes way beyond just engineering but into sales, into marketing, into enablement and all that. And those are the parts of engineering that really, really matter. Those are the parts that have made engineers typically good founders is when they have that. And the parts of engineering that become less important are funny enough the things that Silicon Valley has really bragged about a lot, which is like competition winning or like Olympiad type — are you like as fast as possible at coding or do you memorize all the different nuances of these different languages? Those are the parts that don't matter. Like if you memorized some coding language or some, you know, syntax of a coding language that someone else didn't, it doesn't matter.
People like — not the credentialism, but I think they misunderstand VC mindset right now. And as a VC, I'm happy to share how you feel, how we feel, which is just like fundamentally there is intense uncertainty around what Anthropic and OpenAI will do and who they will kill at the application layer. And so in a world where we desperately seek —
Certainty, we look for validators, and the validators of someone being a math olympiad or you name it, whatever that is, that validation in the wake of not having other certainty that serves as a good crutch.
Yes, but it's a crutch. It's a crutch. It's helpful. It's a good indicator. Like generally, you can't win competitions if you're dumb, right? Like it's pretty rare. However, for these types of engineers that we're looking for, it's like that's cool, but that's kind of irrelevant. Like, what have you built? How have you taken ownership and agency of things end to end? And this is not like we have people on our team that have won Olympiads and I think they're great and it's fantastic and a lot of my friends have, but there's also a certain like especially there's some high schools that like really focus on like you must do the math olympic, like you must do this, the AMC to then go to the IMO and like this is the path to success where actually that's kind of anti-signal because there it's like you're not owning your fate or choosing your agency. You're kind of going through the funnel. But then there are people on our team who are like from the middle of nowhere where no one else in their high school ever did this stuff. And they kind of took the agency of like I think this is really fun. I'm really competitive. I want to compete at this. And they kind of go and do those competitions on their own. Those are when the signal is still positive for this kind of engineer of the future.
I don't think I've told you this, but do you know who you always remind me of?
Matt Damon in Good Will Hunting.
Oh, that's so... I mean, I'll take that to the bank.
Have you not been told that before?
Let me say that one more time. You look identical.
That's very... We're going to put up like an image here and you're going to do a side by side.
All right, we have a clip. This is the clip. This is the starter to the show.
And they're going to be like, 'Uh-huh. I totally get it. That makes absolute sense.'
Can I ask you, when we go back to actually like what makes devs great and how we think about structuring the team, what role does not exist today that you think will be incredibly common in the next few years?
I think so. It's starting to exist more and more, but I think it's kind of this like GM or general manager like role for someone who used to be an engineer where basically you own end to end an outcome that is not just a shipped feature but like a business outcome. So even at Factory we have this now where there are people who will own the marketing copy if they're going to be releasing something. They'll own the outcomes in the product metrics. They'll own enabling the salespeople. So it's way beyond what a typical engineer does and it kind of feels like again owning more of a business outcome, more entrepreneurial, higher agency, just like spreading their reach.
I think that's just like every function. It's... I, you know, believe it or not, I sometimes post on different social media platforms. And I just said like my biggest advice to any students here would just be just be full stack in whatever you do. If you're doing marketing, create the copy. Make sure that it's ready to post. Post it at the right time. Amplify it. You have to be in every element from start to finish. Not just, oh, I just do the copy and then I hand it over to designers to create the visuals and then they hand it over to a social team.
Is that not just the same for every function? We're expecting everyone to be full stack in every function. The age of the polymath is back. Like growing up I was so like I was obsessed with math and physics and I was so jealous that in like you know hundreds of years ago people like Da Vinci or Euler or Newton could be polymaths and it was because their fields were relatively shallow. So like chemistry wasn't that built out, mathematics wasn't that built out, physics wasn't that built out in Da Vinci's case like art and engineering and sculpture. And so you could get to the frontier of these disciplines within multiple disciplines within your lifetime. And then growing up in, you know, the early 2000s and 2010s pre-AI fields were so deep in my case, theoretical physics and string theory. It was so deep that you could spend literally 50 years catching up on all of the literature and academia that's existed before you contribute anything new. And so it was like this was infuriating to me because it was so frustrating. With AI, we're now completely the opposite. These tools can get you up to speed to the frontier. Obviously, with a lot of uncertainty about certain details, you won't have the depth of other people, but it'll get you to the frontier way faster than ever before. And so now, if you're someone that's good at thinking around constraints, thinking about systems, holding uncertainty in your head, and being okay with that, like knowing there are unknowns and knowing that you can still push the frontier forward despite that, you can be a polymath who can push forward and create innovations on how to do developer marketing while at the same time pushing forward the frontier of like token caching for software development agents at the same time as like you know being an incredible solution engineer. Like these are things that you can now do all at once and so this is something that's very top of mind for me and in our hiring process we want to find the people that can be those polymaths. The era is totally back, this polymath era is back.
I've had a lot of people say that agent operations will be like the leading function that doesn't exist today that will be very common in 3 to 5 years. Do you agree with that?
What is the definition of agent operations?
Agent operations is the creation of agents and the maintenance of them. So to be able to go into different functions and say, ah, social media, I'm going to create agents that allow you to create, distribute, share posts. Ah, marketing and design. I'm going to create agents that allow you to create visuals, share them amongst each other, edit them, collaborate on them.
I think to some degree everyone should be able to do that on their own, but I could imagine a world where there's kind of someone whose job it is is to like find places that aren't as efficient and similar to operations now, like in organizations, but now it's just agentified. So, they're using agents to make the organization more efficient wherever possible. But I think in general, if you have people that in certain functions that aren't proactively doing that, it's probably a bad sign.
What do we do today that we'll look back on and go, 'Oh my god, I can't believe we did that.'
I mean, for an engineering team, like writing release notes. That's crazy that people used to spend hours of time writing release notes or like writing documentation.
So, not everyone knows what release notes is. What is it?
So, it's like, you know, basically cataloging the changes that you've made in the last whatever month or so. You know, and sending it out to either internal or external to your users and generally like this and like documentation. Like Stripe has a really great reputation. They had incredible documentation. Like so many APIs had horrid documentation. Stripe was like the pinnacle. They were so good at this. Spent a lot of time doing it. 5 years from now it's going to be like, oh my god, I cannot imagine, cannot believe that these people that get paid so much money spent hours of their time doing this. I think that's something that, you know, we definitely won't do.
Does that reduce the impact of Stripe's great documentation if everyone is equalized?
Uh, yes. But I think Stripe has plenty of places that they can differentiate and I think it's a better world where everyone has documentation as good as Stripe's.
Totally agree with that.
How does the product review and especially like code review process change in the next few years?
Yeah, I think what's cool about this agent-native software development is review has been a big problem because basically, you know, first phase of rolling out AI coding tools was, oh my god, look how much code we can generate. You know, it's incredible. I'm generating a ton of stuff and then phase...
Yeah, phase two was some poor staff engineer who has to review hundreds of these slop PRs that are like don't adhere to your standards or are like completely misformatted and all this stuff.
Um, but what's great about having this kind of like full end-to-end software factory as it were is it's now very clear the ROI of investing in things that make your agents more kind of ready for production. So, examples of this are making sure your agents have access to up-to-date documentation, making sure agents can spin up a remote machine so that they're not just generating the code, but they can actually run it and see what the outputs are and iterate based on that to make sure that it's actually good. Setting up things like CI/CD or good linters or good pre-commit hooks. These are all things that the best organizations at like developer experience would invest a lot of resources in. But they would do it because it makes it easier for engineers to work, easier for them to onboard. But the impact of doing that well is just like one-to-one kind of correlated to how many engineers you have. With agents though, the impact of that is now like 10x or 100x depending on how many agents you're using because the better your devx, the better your agent ends up adhering to your standards, which means there's less time that that poor staff engineer has to go through reviewing your PR, which means you're kind of faster throughput in your software development.
When agents are the buyers and you're selling to agents, how does the world change and does the value of great API increase?
I think that value is increasing especially because the thing that makes it easier for agents tends to be the same as the things that make it easier for humans. At some point in theory that could change where, you know, if you're actually training models or training agents to be as efficient as possible communicating to each other. But then the downside there is it's not as human readable. But if you think about agent to agent, agent to agent doesn't give a shit about UI or design, but it does fundamentally care about data structures, potential integrations, documentation, do you know what I mean?
So, I think one thing that if you don't have careful standards in place, it can get bloated pretty quickly. But I think the best organizations who are the most agent-native actually put in a lot of guidance on like here's like the UI side of things and how things need to be, being very aggressive about like pruning anything that's unnecessary, making sure there's not like, you know, bloated like I don't know comments in all of your code that's like kind of gratuitous or there are ways around it but that's kind of where the human's job changes a little bit where their job goes from... I mean part of our name, our name is Factory. Part of why it's called Factory is because the future of software development is where these organizations instead of having engineers that build the software, they're going to have engineers that build the factories that build their software. Visually whenever I say this I always think of Tesla's factories. I don't know if you've ever seen videos of the inside of Tesla's factory. It's all these like robotic arms going and you have the assembly line going through. And there might not be as many humans in that assembly line, but you know damn well that humans designed this process to optimize the throughput to, you know, produce more Teslas in this case. And so in this new world of software development, human engineers are not going to be involved as much in like writing the actual code, but they're the ones that are going to be involved in how do we make sure it's not just creating all this bloat or it's technically getting the job done and passing tests, but doing it in a way that is really dramatically increasing debt. So they're kind of like building the scaffolding around this factory that produces their software.
Do you worry about labor displacement when we move from working in the factory to working on the factory?
Short-term, yes. Long-term, no. Short-term, yes. Because it's just a shock to the system where, you know, there are all these big layoffs that are happening that are pretty aggressive and, you know, these are thousands, tens of thousands of people that had a job that no longer do. And so, I think that does worry me. Long-term though, I am very not worried because the reality is there is a huge number of problems in the world, ridiculous number of problems in the world. And a large percent of them can be solved or can be helped with software. And very few of those problems that can be solved with software are we currently solving with software. And so if we are going to be flooding the job market with tons of engineers, that means that we can now allocate them on the broader economy to solve more of these problems in the world. And if we have more engineers who are going and solving more problems in the world, that is a net good.
What problem is not currently being solved with software that will be enabled by this new technology? Because everyone's like climate change and I'm like great, you know how many people I've found doing climate change technology? Well...
Yeah, well, and maybe part of that is because all like the Googles have been hiring all these engineers so distributing great engineering talent to more problems I think is going to be a good thing. The economy has to match though and properly incentivize them and that's something that I think will take a little bit of time which is like the intermediate period. But like so many health problems, like so much of pharmaceutical research can be advanced with better engineering. And like the thing that really upsets me with some of the people who are talking about, you know, pausing AI development or any of this like, oh it's, you know, it's a bad thing and it's going to, you know, harm society. Dementia is kind of a go-to example where everyone understands how big of a deal that is. That is something that can be solved with better AI and better software. Like it's a matter of time, like we will solve it and we can solve it. And by saying you want to slow down AI, that's saying like these people who have relationships with loved ones who have dementia. You're like, 'No, no, no, sorry. You guys, you got to maintain that relationship for a little bit longer. We're scared. We don't know about AI.' I think it's like it's pretty harmful and it's pretty selfish to say that. It's something that to me it doesn't make sense. It doesn't make sense.
Do you agree with government intervention?
Uh, in what capacity? In free markets when you think about like the allocation of resources there are times when it is suboptimal from a human morality societal standpoint in a lot of cases to see engineers at Anthropic working on optimizing Claude Code when they could be working on optimizing healthcare systems or optimizing more critical or mission-critical things immediately.
Governments can intervene, offer subsidies, offer economic incentives. Do you agree with that or do you believe in Adam Smith's invisible hand?
Um, I think it's certainly useful in some cases. Like I don't think anyone would argue that the government should never intervene ever in the economy because there are some things especially as it relates to like military uses or safety or things like weapons like you're definitely going to need, you know, some involvement there. Um, I think there's some incentivization that can be helpful just because there might be some problems for a society that maybe capitalism doesn't see the immediate feedback loop of and so you might want to juice the incentives a little bit to get an outcome that you're looking for. But I think generally I'm pretty reluctant. I think you need to have a very good case for why you need to do that. Even like the example of climate change, you know, talking about that one, it's obviously a very sensitive subject or a very important subject for a lot of people and you know you could make the case that the faster we develop AI the sooner we solve climate change because AI, you know, can help us solve a ton of these problems but to develop AI faster you might need to consume fossil fuels and emit them and you know that emits CO2 into the atmosphere and so the question is like, you know, short term it might be slightly worse but it ends up getting us to solve the problem way sooner instead of dragging it out over 50 years or 100 years. And so there's some of these cases where the natural kind of free market will incentivize it the right way and there's some cases where it won't. But I think you need to be very very careful about the cases where you do want the government to say, 'Hey, we want to step in here.'
Do you think we are in an AI infrastructure bubble?
Maybe there's like some short-term blips, but like long-term absolutely not. Like not even close. I think there might be similar corrections to like this thing at Uber where, oh, we were going a little haywire. We weren't allocating it appropriately and it's like okay let's lower consumption a little bit but like on the net absolutely not.
What bottleneck do we have today that will be completely solved within a few years?
I think the biggest bottleneck by far working with all these organizations is the human side of it. It's just like behavior change like especially if...
What you're saying there is like selling into large enterprises and how they do change management.
Yeah. Or even on an individual level like if you're an engineer who's been an engineer for 30 years it's hard to change those patterns like you're stuck in your ways to a certain degree. But there's also a funny thing where some of these engineers who have been engineers for a very long time or who have been engineering managers they might be more reluctant to use these tools but sometimes they're better because they know how to delegate. They know how to deal with some of the like junior engineers where if you tell them the wrong thing they're off in the cave doing the wrong thing for seven days they come back with something completely useless. And then on the other end of the spectrum, there are people earlier in career who don't have as much of a standardized workflow that they're used to. So they're more eager to adopt these new workflows, but they don't know how to manage people. They don't know how to delegate as well. So there's kind of an interesting balance there.
