The Future of Innovation: AI & Lean Startup with Eric Ries

Great. Well, let's hop right into it. So one of the key themes in season one has been the importance of identifying kind of the right assumptions to test at the right time. Several of our guests have emphasized the need to test assumptions about customer and their unmet needs, maybe engagement with a solution versus just testing technical feasibility and model accuracy. Could you give our listeners your perspective on how to identify the right assumptions to test at the right time? Sure. So try to imagine that you've built a business plan for whatever the new AI cool thing you want to build is, and it's really simple. Let's ignore the fiction writing part of the business plan that you wrote in Microsoft Word or Google Docs. Just throw that out. That's not important. But let's look at the spreadsheet. Everyone's always got a spreadsheet in appendix B that shows how the new idea is going to become worth, you know, billions of dollars, hundred million dollars. It depends on the size of the company, how big the promises have to be to get the thing funded. But the spreadsheet is always the same: the beautiful hockey stick-shaped growth curve, customers flocking, the high revenue per customer, the extremely high lifetime value, etc., etc. And now the key to identifying a leap of faith assumption is just to ask yourself: what are the inputs to this model that, if they went to zero, the model would be catastrophically destroyed. So let's imagine that, you know, there'll be some cell in your spreadsheet, if not, you better redo it, that says something like percentage of customers who try the product who convert to paying customer, or percentage of people who see the brochure who pay us, or, you know, the revenue per customer in the first transaction, or whatever those critical metrics are. There's going to be a few, and all we want to do is, like, can you visualize in your mind the weapon of mass destruction rippling out through your whole spreadsheet, destroying the whole thing if you change it to zero? And it's uncomfortable when we're doing something new. We're not supposed to talk about the possibility that it might go to zero, but that is a real possibility. And the most important thing is this is not an exercise in skepticism. You know, if you're a product manager and you want to sabotage a rival product manager, it's really easy to just say, well, let's imagine all the things that could go wrong and start enumerating them. You don't even have to say that they are going to go wrong. You're like, have you considered the possibility that customers won't want to buy this? Have you considered, like, those are considered attacking questions in most corporate contexts. But here we don't have to tell anybody else. We just have to know for ourselves these are the leap of faith assumptions of things that must be true in order for this business plan to make sense. And then you have to do one further step because this all sounds like a bunch of spreadsheet jockeying. I understand, but the real thing you want to do is understand what are the human behaviors behind the metrics. We always say metrics are people too. What is the human behavior that this represents? And I think this is where people get really tangled up when they're doing innovations. They say, ah, this isn't about quality. This is about spreadsheets. But I want to build a really high-quality product. Okay, but what does quality really mean? Quality means that when the customer is given the chance to use your product, they find it delightful. Well, delight is a human emotion, hard to measure, but when someone's delighted, what are the behaviors that they exhibit? Oh, of course, they use the product a lot, they refer their friends, they use it again, they come back. Those are the behaviors that the leap of faith assumptions represent. And all we're trying to say in startup is that when we're doing something highly uncertain, something highly ambitious, something with long-term potential, we want to just know what those assumptions are so that we can find out if they're wrong as soon as possible, not because we think they're wrong. In fact, we ideally like to find out, just double-checking that they're right. When they find out they're right, that becomes the basis for making much more rapid and decisive investment in those products based on more than just a hunch.

Well, I want to stick up in defense of the technologist for a second because one of the cool things about AI is it's stretching our imaginations. So, you know, in the previous era of technology development, we went through a kind of a long interregnum where we pretty much could build anything that we could imagine, and we started in innovation circles to deemphasize the importance of imagination, mostly because we had more good ideas than we knew what to do with. And in software anyway, anything that you can't imagine can be built. There was no real cost. You know, it's really just a question of time and do consumers want it. Well, now we're confronted with a technology that has cost of goods sold all of a sudden. So our friends in manufacturing have a big leg up on people who've only ever worked in pure software. But I think a lot of people have jumped to the conclusion that they understand what generative AI is capable of doing. And I think there are a few people on this planet who can say that, but I probably know most of them, you know, and it's very unlikely to be anyone I listen to. This, no offense, and I would even include myself in that category. There's truly deep experts, and one of the things the experts have that we don't is they have a much better understanding of what is the possibility of what this technology could do in the near future. The rest of us better play catch up. We better be figuring out the answer to that question. And so I don't think there's anything wrong with focusing on the technology, what it can do, as a way to generate new bold ideas that are worthy of testing. But we can't assume that just because we've conceived that something is possible, like a futurist, we say, okay, this can happen in the future, that doesn't necessarily mean that it's good that that should happen. It doesn't necessarily mean that it will happen. It doesn't necessarily mean that customers would like it if it did. And, you know, you and I have been around this long enough. We've both seen situations where people have spent millions and millions of dollars, you know, training a model, deploying it, figuring out how to build software around it, all to discover when they finally get to the moment of deployment that the people that are deploying it to won't use it. And so there's nothing that people, I feel like, defend that process with the idea that it's about quality. It's about, you know, decisive action. It's about this. It's about that. Well, building something that nobody wants is not praiseworthy in any dimension. So instead, we should try to learn what people actually do want and build it for them, which is not to say that we should just ask customers what they want. I think we all know that old canard. That's not, it's not effective at all. It's that no more important than if you said, oh, oh, sorry, I can't do chemistry because I can't ask the electrons what they want. It's like, hello, it's a scientific method. We want to form a hypothesis and test it. And no matter what you believe about craftsmanship, about quality, about product, whatever you believe about what works, market power, some people are very finance-driven, some people are very technology-driven, very marketing-driven, the importance of brand, I can enumerate so many business theories that I've been pitched over the years. The idea here is that whatever you believe, let's just double-check that it's true and test it.

Everyone knows what a Venn diagram looks like, and the problem with teaching kids Venn diagrams is that we teach them that there is always an overlap zone between the two things that are being put into the Venn diagram. But actually, in real-world situations here, we're trying to figure out, right, like the circle of what is possible to build and what is desirable for customers to have. We're assuming that there's an overlap zone and that our product, our MVP, our experiment is going to sit in that overlap zone. But there's no like cosmic law of the universe that what you can build overlaps with what customers want. In fact, far more often you wind up with headlights, you know, not a Venn diagram at all. And so that's okay. Like we have to accept that that's a possibility. But I do think that in a corporate setting, because so many corporate cultures are so anti-imagination, anti-creativity, we oftentimes don't allow our teams to actually explore the full space of what is possible. We've pre-constrained it. And this was true before AI. I never forget, I was working with a company in, I shouldn't say a company, but they were a company that competes with Amazon. And so they were really, you know, constantly talking about how Amazon was eating their lunch. They're fighting with Amazon in multiple fronts. You know, big company with many divisions. Amazon is in competition with them many times. And they were like this particular little team I was coaching, they're trying to build a new software product that could compete with Amazon. And they were like, we're finally using the tool of the enemy. Like we're really going to, they were focused on software. I'm going to do it this way, and we're going to try to figure it out. And I remember they were sitting there talking about the constraint of the problem. Well, we can't do this because of that, or we have to go get an out, we have to outsource this piece, we have to get this person's permission. They were like enumerating all these extra things. And I finally said, look, just can you guys tell me if Amazon was actually having this meeting right now, is this the way they would approach the problem? And they all laughed like, oh, no, they don't waste time on any of this stuff. I was like, so are we competing with them or not? Like, what's the deal? You know, they were like, oh, well, you know, I guess they're just way better at this than we are. But it was like, are they, or could you just choose right now to bring forward like the full power of software and unleash it here? And what's interesting is a lot of companies that don't, they're not software native, always felt like they were inferior or behind these other platform companies. And it's kind of a funny situation now where actually, even if that's true, and if you're feeling that anxiety right now, everybody's on a level playing field with generative AI because absolutely nobody knows anything about what to do with it. You look at these big companies, how many of them have successfully deployed a generative AI product at all? Even if you look at the top most valuable tech companies, even the ones that are reaping the massive rewards from the generative AI boom, how many of them have actually deployed a successful generative AI product? I can think of three or four, right? Like maybe you can think of five or six, but like we're not talking about ten, we're not talking about a hundred. So we're all, we all have a chance to leapfrog. So I think it's ever more important when we build these teams to really bring cross-functional discipline together, like get people, you know, as we've always said at Lean Startup with team construction, full-time, not part-time committee, full-time dedicated members. The old Amazon canard, the two pizza team, right? A team no bigger than you can feed with two pizzas. Get people in a room together and say, look, you've got to figure this out together. You've got to find the full exploration of what's possible. And I have done this exercise in startups, in corporations, from two guys in a garage up to big multinationals with hundreds of thousands of employees or big government agencies with millions of employees. And it's always the same. You have the least important person in the room at some point will speak up and say, well, you know, if you really wanted to do that specific thing, you know, do you know about this? And everyone looks and they're like, well, no, never. What are you talking about? What, you never heard of it? It's like, oh, well, everyone in my field knows that XYZ. Or like, oh, everyone in the marketing department knows that customers, what they really would pay for is this. Or, you know, everyone, you just, you pick a field, people know it. Everyone in customer service knows that this is the number one thing people call and yell at me about. Does everyone else in this room know that? No, no, that's actually really useful information. Thank you for sharing. So, yeah, I think it's, um, that cross-functional collaboration is right on.

This is an exceptionally misunderstood concept, and every, you know, I don't know, every six months or so, someone sends me a message that's like, you know, if only you'd name the concept something else, then people wouldn't get confused. And I've been through lots of iterations of trying that out with clients and with customers. And let me just tell you, the issue is not in the terminology. The issue is not in how we define it. The issue is in human nature and our extreme business culture that we live in that's all about, you know, claiming you can predict the future when you really can't and just hoping that it will work out. And unfortunately, in too many companies, middle managers have gotten good at setting up these long-time horizon projects where they can basically get promoted into another job before the bleep hits the fan, and then they never have to be accountable for the beautiful plans that they worked on. And it's funny, talk to executives, I'll just be like, well, there are these certain managers just have this golden reputation that everything they touch turns to gold. And if you dig in, you'll find out what they're really good at is making these beautiful plans that don't have to have contact with reality. And the ding in the performance file will be whoever had to execute the plan after they were gone. They're the ones who get, you know, they get screwed up. And they, you know, I remember doing this negotiation once where someone said to me, it's going into one of these meetings with a very cocky middle manager. And he said, look, the magic key, just so you know, yeah, I want you to have my back in this meeting. Whatever they say about what happened with this beautiful plan that we built out, make sure you ask them, did you follow the plan exactly? And then the answer to that question is always no. And then whatever they say, they deviated from the plan. That's the reason it didn't work. If only you followed the plan exactly, you would have. And people are good at that, you know, it's just an incredibly effective way of evading accountability in too many company cultures. If you have a friend's company that's doing that, you got to put a stop to that. It's terrible. So minimum viable product, it's a really, really simple idea. It's just given the thing that you want to learn with your experiment, what is the least amount of work we have to do to discover the truth of the situation? And people immediately like, right, you're talking about cutting corners. You're talking about ignoring safety. You're talking about lacking quality. You're looking at it's like, no, no, no, no, no. If you tell me those things are essential, put them in the MVP. You tell me what's essential. But most times, most companies have these elaborate plans that do not require so much work. And so by the time they find out that something's wrong, as you said, it's too late to change or it's very expensive to change. So in a minimum viable product, what we want to do is just find ways to reduce the scope. And actually, in a lot of cases, we will by doing so increase the quality. So in the, you know, CL, here are some of the classics. Then we can talk about AI-specific classic MVP. You're manufacturing something. To manufacture a million units is a lot harder than manufacturing a thousand units is a lot harder than manufacturing one unit. Well, if we didn't care about the cost per unit at all, could you manufacture me one unit that is exactly what customers want? And could we go sell it to them? People think, of course I could do that, but what's the point? Aren't we supposed to be a mass manufacturer? Well, if you can't sell one unit, you can't sell a thousand. So let's first find someone who wants the one unit. In software, we have all these techniques, call them the concierge MVP, for example, where a human being works behind the scenes to make sure that the results are really good. We see that a lot with AI, where people think they're interacting with AI, but it's actually an AI-human hybrid behind the scenes. We do that to make sure that customers actually like the deliverables that we're creating. And the reason why MVPs are so powerful, we've been emphasizing the negative, obviously, because in corporate settings right now, we have this epidemic of people wasting money and time on these products that nobody wants. But there's a positive side of it too. Go talk to your favorite technologist and ask them, what would you rather have, a blank sheet of paper and a specification document with total freedom to make whatever you want, or would you like three examples of actual deliverables that customers have already been willing to pay, and you know exactly how much they would pay for this specific deliverable? The technology problem, I say this as a technologist myself, the technology problem is dramatically easier to solve with examples, and not the kind of fake examples we put in project management, product management, PRD documents, stuff like that. The real example of here's a real customer who actually paid us money for the real thing. And that's incredibly important with AI because it's just so much easier to generate the deliverable. So, like, the cost of doing the MVPs is a lot lower, especially if we're willing to use off-the-shelf existing components, you know, use existing foundation models. Like, there's so many things we can do that reduce the scope of the problem. And then you say, well, but we can't afford to have a human being generate the results for a million customers. That's okay. Let's do it for ten, and then let's figure out how to do automation. Let's figure these things out.