One of the most depressing things I see when I talk to boards of very large companies and to C-level leaders is a lot of them have come to believe they just can't change, right? So, I show them what AI can do and they're like, 'Well, I guess we'll ride this out.' And I'm like, 'That's an insane answer. You have 100,000 people working for you. The smartest people in the world are working for your company. You have customers and resources. The idea that you can't be flexible and compete seems to be overly baked in right now.'
Ethan Mollick, professor at Wharton and author of Coexistence. Welcome to the IdeaCast.
Yeah. So, you've been closely tracking advances in AI and constantly experimenting with all the new tools. What is different today than even a few months ago in terms of how new developments should guide the way that leaders are thinking about strategy in particular?
One is the exponential curve continues. AI just keeps getting better. There isn't really a sign of that stopping right now. And the second is that agents are real, which are AI tools that can go off and do work on their own autonomously and do real, real process work, real important things. And that has become a viable approach to work in the near past.
It seems like a lot of companies right now are thinking of AI as really a productivity exercise. As you know, you're automating tasks, you're saving time, you're reducing costs. How does, you know, a genuinely strategic AI transformation strategy differ from that? What do you think companies should be doing now?
I think it's actually pretty urgent to do a few things. One is viewing this purely as a productivity tool gets you in trouble in a lot of different ways. People are getting ROI from AI, right? They're getting ROI from that productivity gain. Productivity gains are real. The problem is that those productivity gains get eaten up by the organization, which is if you don't change a part of the process, AI is mostly a single-player entity and you end up with more of everything. So if you don't change your KPIs, people produce more PowerPoint, they produce more code. That's not actually what you want. So leadership needs to decide what this thing is. And the second is AI is getting very capable and I worry that if our default is automation, we end up with a situation where we just keep handing tasks to AI and don't think hard enough about where humans fall into this process, and that leads to a lot of danger as well.
Is your argument that leaders need to move from asking sort of where can we use AI in this existing business alongside humans to a question that's more like, you know, how does AI change our business going forward?
At this point we have ample evidence that AI can regularly do the kind of work that would take a human two to 18 weeks in a reasonable amount of time, reasonable amount of money, with high success rates. That implies a lot of different things. It implies that what your organization does has to change. How you think about the boundaries between jobs is changing. I talk to C-level leaders all the time and one of the first effects they see of AI, and we saw this also in our research, is that jobs start to blur. Coders do design work, designers do coding, product managers do everything. How do you start thinking about what the boundaries of your organization are? What's the product you're trying to produce? Is it an interim product of a PowerPoint? What was the point of the PowerPoint in the first place? You start to run into really fundamental questions about what the goal of the organization is, how you accomplish that, what are your values for getting there, and they're urgent. This isn't sort of theoretical, you know, let's sit down in an academic way. It is your organization is already changing from the inside. What do you do about that?
So what are some best practices that you recommend to leaders who are grappling with these issues right now that need to start having those really urgent conversations and not just have them but sort of make them progress enough so that they can actually start making some real decisions?
There's some individual feedback and there's some organizational feedback. On the organization side, the model I've seen works best is sort of like a meta model, which is leadership, lab, and crowd. So you need three things to succeed. You need the leadership team to actually lead in this, right? That's not just say use AI. They have to make decisions about how AI is used, what the organization does with AI, how to incentivize people. The crowd means giving people access to these tools. We're at a point now where giving organizations inferior AI access, either just because you haven't bothered to improve to the most recent models, leads you into trouble, like you actually become very deluded about what AI can and can't do. And then you need a lab. You need people doing 24/7 work who are not primarily technical, by the way. There'll be technical people there, but also organizational people who are thinking about how do we restructure the organization around AI. They're benchmarking what systems could do, building systems of the future, taking ideas from the crowd and ideas from leadership and helping build them. So, you need to be doing this as a whole organizational response. At the individual level, the factor that I'm seeing that differentiates leadership that succeeds at AI use and organizations that succeed and those that don't is leaders who are willing to eat risk, who are willing to say, 'Look, nobody knows what's going to happen here, but unless we do active experimentation, we're not going to know anything.' And if you just let everybody in the organization have vetoes over AI use because it might be risky, it is. But not doing anything is risky too. And that also means actively using it yourself. So leaders who don't use the latest AI systems, who wait for reports from other people, who are waiting for training some week, fall behind those who just are experimenting.
So, I think the last time that you and I talked about this, you sort of had this model for human-AI collaboration that was either a centaur, you're half using your human skills and then half using AI, or a cyborg where you're going back and forth between the two. Talk about how that is evolving to one that you now describe as conjurer work that you think, you know, leaders should not only be doing themselves as individuals but also enabling in their organizations as part of their strategy.
The biggest change, if you really look at the sweep of the last year, was that in every independent study we have, basically AI went from regularly being able to do a half hour of work, 45 minutes of work, to you can get 16-plus hours of work out of a single AI instruction with good chance of success. And that means you don't walk work next to the AI anymore. You don't work back and forth with it in the same way you did before. You are actually assigning the AI to tasks almost like you had a team you were assigning and it does the work. So a conjurer is really about conjuring a team into existence when you need it. It's almost like writing an RFP for a project you need done, or SOP, or any of the other standard TLA three-letter acronyms that we use in business to assign work. And you send that out and the AI just does that work. That's a very different situation than I prompt the AI, look at the results, I prompt it again. This is a very different, you know, action-oriented field.
How does that change how leaders approach, you know, the structure of the organization? What it even means to be a manager?
So, everyone's a manager to some degree right now. Everyone has AI to manage and AI skills look like management skills. In fact, there's a great study out of Anthropic that shows that the people who are the most successful coders at the first pass of coding are actually managers, not coders, because they do a good job specifying what they want and AI executes on that. So, that is the big problem, right? Your organization suddenly capacity increases. And I think organizations that don't have imagination are going to retreat to automating away the work. And automation is a well-worn playbook in the business world. I think it's a recipe for disaster. I mean, it's a disaster for all of us. If all of our work becomes automated or we just become verifiers of AI content, that's like not a good situation to be in. But also, it removes all the differentiation. Like leaders need to be building companies that blend AI and human work together with process and with culture in ways that accomplish things that neither could do alone. If you just move to automation, you're going to have the same cloud-based company everyone else does.
Well, that's an interesting point. So, you know, when we're talking about strategy, we're also talking about finding competitive advantage. So, if every company has access to the same frontier models, where exactly can leaders differentiate their organizations from competitors?
That's where the key kind of comes in. You are probably not using AI enough inside your organization, not trusting enough, but you then want to bring people back in. I mean, we're already seeing as the most extreme version of this, right, in terms of organizational change, are things like software dark factories. So a dark factory is a factory where all the work is done by robots. You don't need humans in it. You don't need any lights. Software dark factories, I know one company, Strong DM, that has been shipping product to customers and there's no human involved in that. The rules are no human could look at code. No human can write code. It is 100% AI automated. I think that's a dangerous approach because you're just letting the code do the work. I think we need to think about building twilight factories, situations where the AI proactively reaches out to people and brings them in because they have variance and views, because their process or expertise matters, because their sign-off approval is needed, or because they find the work interesting. And I think we need to start restructuring work around the idea that these systems really can accomplish many things and will be able to accomplish more in the near future. So it's integrating with humans at the right places, at the right time, in process, that's going to give you a differentiator.
Where are the specific ways in which you think that, as fast as AI is evolving, humans still have a role, and particularly leaders still have a role?
One thing is you can't really be in the loop in the same way you were, right? It was easy to be in the loop when it was a chatbot. It's not easy when, I mean, I literally have projects where I have a swarm of a hundred agents solving a problem for me. Like I have them report out to me like I'm a manager. So I am in the loop to the extent that, you know, they give me a PowerPoint about their progress and they show me demos of what they're building, but that's very different than, you know, I'm approving every action, right? And so leaders actually, one of the outputs of AI is leaders are super empowered. I keep seeing organizations where leadership basically takes back control over their company because they can now directly ask the AI for, you know, the information they need. I have talked to people who are like C-level leaders who've actually diagnosed problems at plants that no one brought up to them because they could just literally query all of the information at their company's disposal and ask questions and have the AI go off and do tasks for them. So it's a really interesting change in the span of control if you set this up right, which lets leadership actually think more about things rather than hearing things through three or four different layers of leadership.
So would you say then that AI agents are making strategic decisions on behalf of leaders and organizations with the right context, you know, given to them?
I mean, they already are. If you have an AI agent that is doing any long-running work, right? Epoch, which is an AI benchmarking company, was reporting that they could get regularly sort of between two and 18 weeks of engineers' work out of a single prompt to Claude for a couple hundred dollars, right? That involves tons of strategic decisions and judgment calls and creativity calls. They may not be your judgment or creativity, but like we can't say anymore the AI isn't making strategic decisions. It is. The question for you is where do you want to make decisions and where not? And just like any team, you can give it those instructions. You could say, listen, I want you to come with the five best options, your defenses for each of them, surface those, pressure test them for me, and then deliver so I can pick between them, and this is the format I want them in. Or you could say just go ahead and execute, right? You're hooked up to code, build a prototype for it, and test it with a few people. Those are now options you have available to you. So some of this is reconstructing what we've already done with humans. So what's the process by using humans and AI together that I get a working idea that's much better than my ideas before? How does that get done much faster and better?
Where exactly though does a leader know where they add the most value and where they don't? Like what are the points at which they should be involved?
Absolutely. Every time. I think part of this is you have to make decisions now about how to best accomplish what your company wants to accomplish. And you obviously don't want to give this all up, right? You don't want to give it up because the AI is not as smart as you or what you're best at, because the AI doesn't have enough variation, because it doesn't actually absorb your culture in a deep way. Now, you can give some of that as context, but that's part of process design. I mean, we do this all the time. What are the limits of your product engineering group versus your marketing group versus your legal team? We have to think about the same kind of process design here, which is where are those limits and limitations? I think the danger is that by skipping doing that, you either move to an all-automation world where all those choices are made by AI, or you underestimate what these systems could actually do. So it is actually disciplined decision-making that gets you there.
And what are the things, the aspects of strategic leadership that you don't think can ever be delegated or outsourced?
So I'm always careful about bright lines where we say something can never be done, right? But these systems are jagged. They're good at a lot of stuff and bad at what stuff you would expect. And that means, by the way, there's different things the jobs do. Think about coding, right? You were hiring coders to be the best at writing elegant lines of code. Then AI could write the code. So then it became about sort of systems engineering. Could you see the bigger picture? Now AI is increasingly doing some of the systems engineering work. In some ways, it's very upsetting if you're a coder because the nature of the job is changing in ways that are good or bad all the time, right? And what you learned as craft no longer is. I think we're going to see the same thing about leadership. Like right now, maybe your judgment calls are about, you know, the details of pricing strategy. AI might be better at pricing media. You won't know till your lab tests it for you. I can't give you the answer in your industry or not. You'll only know by experimentation, and you don't have to do that all alone, right? That's where your lab and your crowd comes in. Your crowd is surfacing use cases and values of AI. You don't have to do this all yourself. But I think a big danger of companies is they think someone's solved this problem for them. Someone's figured this out. We're all faking it. Nobody knows anything. Like the AI labs, you know, don't know anything. The vendors are all making this up. This is a three-year-old technology. So, you have to sort of seize control to some extent of this yourself to figure out what you want to use it for.
Are there capabilities that you suggest your students, your MBA students, your kids, aspiring leaders, that you suggest they develop because you believe they'll be increasingly important in an AI-dominated world?