Inside the path to Enterprise AI Adoption with Ethan Mollick | EP 21

I mean, so the most extreme version of this, which has been floating around in sort of AI circles for a while, is when will we get the first one-person billion dollar company, right? But we don't have to go that extreme to say, okay, well, we already know AI improves performance for individuals, but it also fills in skill gaps for them. Whatever you're worst at, the AI is probably better at that than you are. And so, as a result, most people are held back by skill gaps. You know, they're good at some stuff, not good at others. Productivity gains, right? I can now have basically an infinite army of interns to manage. And as a result of that, you get very large improvements in individual ability. Again, the moderating factor is organizational structure. It's not individual ability or even AI ability at this point.

Sure. I mean, so the way we work with AI has evolved. I mean, the book came out two years ago, which is infinite time in AI world. But the methodology still works for a lot of kinds of work. There are new ways to work with AI as well. But the idea was that you basically there's two kinds of ways of working with AI. Either a cyborg or a centaur. So a cyborg is blending their work with AI. So you go to AI to do something, you go back to human work. And a centaur divides up the work like the half person, half horse where you do some things with AI and some things not. And I think that those models are helpful because they kind of clarify how do I want to approach AI in this situation.

Sure. And I want to focus, it's not just productivity. I think productivity gains become a focus and in some ways you end up just doing more of what you're already doing, right? So part of what's interesting about AI is that it is transformative. Transform doesn't necessarily mean replacement, right? It means it does change and transform things. So for example, I've got some research and a lot of my colleagues do showing that AI is really good at idea generation. So if you ever need to generate ideas, have the AI generate 100 ideas after you've generated your own. So you have your own thoughts first. It's obviously very good at sort of the usual editing, writing, feedback, but it's actually quite good at strategic advice, at thinking through hard problems, obviously coding. I mean, it's a general purpose technology, so it does many different things.

So there's a lot going on there. I mean, one thing is we don't know the shape of the frontier. You don't know what AI is good or bad until you use it. I mean, OpenAI has no idea what the value of AI is for a midsize Midwestern insurance company, right? Or a car dealership or anything else. I mean, no one's thought about this. So there's opportunities to find use cases. There's also probably problems, things it's not good at. The only way to know is to explore. I think that there is too much worry right now about some of the sort of information security things. I mean, they're very worth worrying about. But the idea that the AI is going to leak your data or your private secrets is less of a concern than it was. Although these systems really have secured themselves a lot more than they were. There's still kind of cloud-like risks, but no longer the AI will train on your data without you knowing it and send it to someone else. For enterprise systems, there are a lot more esoteric security risks appearing as a result of AI. So it's actually quite good at hacking on its own. It's great at social engineering. It itself can be manipulated through things like prompt injection. So there's a lot of new risks, but I think the privacy concern or that we shouldn't use AI because it might leak our secrets is generally something that a lot of companies have figured out ways to address.

Yeah, I mean so on the ROI front I actually don't think this has been as much of a problem in the last few months. My colleagues at Wharton have a survey they've been doing of companies. 85% report positive ROI from their generative AI efforts already, right? I talk to companies all the time doing this. Like eight months ago it was a very different situation, there was a lot of uncertainty. I do worry though the ROI focus as the first thing to do is a mistake because this is fundamentally about transformation, about an opportunity to do different things in a different way. And if you're just trying to focus on ROI as the only outcome you're not doing R&D. So there's going to be some expensive expenses. If you're only trying to be like, how do we write more code? I think you end up in trouble.

I mean I think almost everything. So let's take something that we know from the research that AI has a big impact. So even before Claude Code and all the new coding apps came out, there was a nice study of Cursor which is another coding helping tool. When that came out it increased the amount of code that people were writing by about 38% and did not increase error rates. So that's just like right off the bat, right? I think we have multipliers on that right now. That's fine except that the way we do coding tends to be built around agile development of the early 2000s and it's based on a whole bunch of assumptions about what work people can do, how you divide up the work, how you do a two-week sprint that no longer make any sense. So the issue that I have is that unless you're rethinking how do we redo coding as a process, what does it mean when people produce 100 times more code? You end up being stuck, right? So it's not just about a multiplier effect on productivity. This has knock-on effects everywhere in the organization. If I could produce an infinite number of PowerPoints for you, that's not actually what you want in a company. You don't want 100 more PowerPoints. That's not a productivity gain, right? What you want to do is think about what was the goal of this. Can I accomplish it in a new way with AI? So part of what I think the exciting thing but also a trap for companies is let's do more of what we do without thinking about it in a different way.

It's a really interesting question and one that's being hotly debated right now. I talk to a lot of CTOs of Fortune 100 companies who are thinking about how do I start using AI to replace some of the functions that used to be enterprise functions. Even if you're not thinking about that, you start to think about how do my agents get access to these tools when it's not just a human but I want my agentic systems to be able to use it. I think people are thinking hard about what the boundaries are. When do you want to assign a task to, outsource it to a vendor versus when do you want to think about how do I bring in the right consultants and people to help me do this internally? And I think that it's a lot of flux right now.

Tons of them. I mean, I think there's lots of use cases where I've been seeing people get value from agents. I don't know how many I'm allowed to talk about, so it's always hard to do. I mean, coding agents are an obvious thing, right? One of the big shifts that's happened recently is if you talk to the team at OpenAI that's coding, they're not touching code anymore. Most of the time they're just instructing agents to do the writing. Same thing at Anthropic, all the Claude Code stuff is being written 100% by Claude Code at this point. And when you talk to senior developers they're getting these same sort of benefits. It depends on what the company policy is about whether you gain from that or not. So I'm seeing agentic systems being used in internal tools a lot. A lot of simplifying of what used to be very complex processes or bureaucratic processes with AI. Really lots of change happening all over.

So, not to upset anyone, but I actually think that the emphasis on data is a little bit of a hangover from old school AI, which means AI from four years ago, which is every bit as valuable as it was, right? Old school machine learning, for customer insights, for finance, for anything else, every bit as valuable as it was. Predictive models, random forest, whatever you're building, they're great. Keep doing those things. And those need clean data. Large language models operate a little differently. Your model should be a person doing something, not a machine that's learning from patterns. So accessibility is very different. If a human can use a tool, often the AI can use it. If you have a knowledge base that's searchable by humans, the AI can do a search of that knowledge base. The idea that you need really well formatted data before you can do anything with LLMs just doesn't make sense. It's a hangover of a previous era of machine learning that's every bit as important, but is a different meaning right now.

Yeah. So I mean some of the most important data is not in any data set, right? It turns out that one of the great things you can do with AI, especially with things like skills which are sort of these prompt libraries, is you can distill out expertise and have the AI act like an expert. That's not written down anywhere. But what's our process for handling this problem? If you explain that really well to a modern frontier model, it often can execute that process autonomously using tools on its own. Like I'm able to ask the AI just for a fun example to build a game from scratch and I don't have to do anything else. It will figure out how to create the graphics, host the game, play test it itself without me having to do anything else. In a more serious way, I've had my students launching startups in two or three days and they're without ever having coded before, without touching code because they could just give the AI instructions to go do this kind of work. So I think people overemphasize the amount that it depends on that kind of structured data. Now, in the space of data analysis, AI does a lot of things that are useful. One of the big things is taking unstructured data and structuring it, right? You can have it read through stuff, helping you organize data, help you write code, but it's a very different thing than using LLMs day-to-day.

I mean so the most extreme version of this which has been floating around in sort of AI circles for a while is when will we get the first one-person billion dollar company, right? But we don't have to go that extreme to say, okay, well, we already know AI improves performance for individuals, but it also fills in skill gaps for them. Whatever you're worst at, the AI is probably better at that than you are. And so, as a result, most people are held back by skill gaps. You know, they're good at some stuff, not good at others. Productivity gains, right? I can now have basically an infinite army of interns to manage. And as a result of that, you get very large improvements in individual ability. Again, the moderating factor is organizational structure. It's not individual ability or even AI ability at this point.

So I think agents as employees is a little less useful than thinking of agents as operating in the organizational structure as a whole, right? So I mean agentic work is now a real thing. This was something we were making up and talking about and there's a thousand definitions of agents and you'll still see that hangover. By the way, lots of companies say an agent is a prompt that's connected to another prompt by a deterministic system. That's not what agents are. You can see in Claude Code and the coding agents how this works. They're agents that independently plan what they're going to do, get the skills, tools, and information they need to execute it, execute on their own, and then return with a product for you. So delegating stuff to them, delegating things to sub-agents becomes really important. That's something you could do right now. This isn't a science fictional future of like you'll manage a team of agents. I typically have five or six different agentic tools running at a time when I'm doing other things, building software for me and doing research and all kinds of other tools.

So I don't think it's either or. What I think you need to do is you have people in your organization. I usually say you need three things to make your organization succeed with AI. You need leadership. We've been talking about that. How do you build the organization around this? How do you transform things? You need crowd, which are people with access to these tools. We've also discussed that. But the third piece is a lab. You need people who are doing 24/7 work with AI. So it may not be that the tools are all available to do what you need right now, but unless you have somebody actually trying to use these tools, you don't know that, right? So the issue is that there may be a weight calculation. You might decide like I feel like a lot of people who leapt on talk-to-our-documents RAG-style solutions ended up with things that are less powerful than if they had waited six months and built on the latest set of tools to do things. And that's okay, right? Learning happens. I think what you need to do is be close enough to the ground for some people in your organization to have a sense of like, yeah, we should wait to adopt it for another six months because this is... but that doesn't mean you're sitting back. I think it is hard to catch up if you're completely out of the game because there's a skill set that you developed that is consistent. If you've used Claude since Claude, you know, one, actually Claude 4.5 Opus feels like that. You'll know what it's going to be better at or worse at than ChatGPT because I know the model's personality.