Ethan Mollick on Coexisting with AI, at Scale

Yeah, that's a great, good, hard opening. I mean, so I think that co-intelligence is still useful because it's kind of a description of a mode of working that still exists. It's just fading importance, right? So I think the things in coexistence that make the most sense and will hold up durably is there sort of a list of times not to use AI as a human, the human pieces do, right? I'm a social scientist. So the fact that this is about interactions between AI and humans, AI might change, humans don't. So one thing I talk about in the list in the book is when you should use AI because it's important that there's a human involved, either for social reasons or learning reasons or societal reasons, and I think that that will persist, right? We have to make choices about when to use this and where not to. And the other is about that I think as you delegate things to AI, things change in durable ways, right? We don't get that delegation back in some ways. So if the book has a lot about how and when and in what circumstances and precisely how to give large-scale tasks to the AI, and then that is a thing we don't return, right? That just like we give something up each time with the co-intelligence model, we do with the agentic model as well.

I've been ahead of the curve generally in thinking about AI's capabilities, right? So my book actually starts with a conversation about can AI write this book, and I conclude not yet, right? And then I talk about will an AI write this book, and I talk about the weight equation, which is this idea from rocketry that if you were to set off an interstellar voyage to, you know, Alpha Centauri right now using today's technology, it would take you many thousands of years, but it's completely plausible in the next hundred years we have technologies that take half as long, right? That are just faster drive, not violating physics, just better. We have fusion power or something like that, and you get passed on the way by somebody who waited longer to launch the journey. And the same thing has kind of happened with AI, which is I have a list of things that AI couldn't do, but that I kind of check on, right? One of my first things I improvised when, you know, when GPT-4 came out was a tool that would auto-reply to low-priority sort of spammy emails, but it tended to over-accept those emails. It would say, 'Yeah, I'm not interested in this, but here's, I'd love to help you more in the future.' And I'm like, 'That's not what I want my spammy email reply thing to do is to keep volunteering my time.' And now I trust an AI to do that kind of spammy email processing for me, you know, while I write the more important stuff myself. I think that when there are areas of the jagged frontier of AI capability, when it's good or when bad, it's unexpected. I think if you look two years ago, a clear weak spot was math. Absolutely a huge problem. That changed, that changed literally about almost two years ago, almost exactly with the release of the first reasoner models, the o1 preview series, and suddenly AI is really, really good at math. So I think it's hard to make really firm bets in one way or another.

Right. So the big next line in AI development is usually called RSI or recursive self-improvement. Do AIs make themselves better? There's a related concept for us as users of AI, which is called continual learning, and they're a little bit different from each other. RSI is the idea that AI helps make AI better. It's really a thing the labs do, right? Like, you know, OpenAI uses AI to improve AI. They're already doing that with some things, but in a lightweight way. The continual learning is the uncracked problem in AI right now, of the fact that every time you talk to an AI bot, it's an amnesiac. It doesn't know what happened before or after in previous conversations. It can look up information, but it doesn't actually remember skills that you gave it, right? It's just reading a prompt that you wrote before and incorporating that in. So given all of that information, we're not really at either of those places yet, right? We don't have continual learning and we aren't in the early days of recursive self-improvement. So none of those things are having an impact right now, but the capability gains are happening without those two pieces fully operating yet.

A few different things going on. First of all, I want to credit my co-authors on these papers, right? Which are folks from Harvard, MIT, other Wharton folks, University of Warwick. So it's not just my work, right? I think that we are in a really interesting situation right now because those papers themselves are already slightly obsolete in some ways. Those are already pre-agentic papers, right? Those were co-intelligence era papers. I think there's a lot of lessons to learn from them, but things are still accelerating. I will say I think the nature of the firm is going to change dramatically. It already is. Like, I spend a lot of time talking to executives at companies and talking to large organizations about this stuff. And I think we're seeing the beginnings of this. Change is slow in organizations. Anyone who studies them knows, or lives in one. You know, I'm part of a 250-year-old organization, Wharton, that still does things by paper that could have been done electronically 30 years ago. So things change over time slowly, but I think we're going to see very large-scale changes happen in companies. In fact, I don't think you're going to see large-scale gains from AI at the firm level until you see corporate changes happen. You can take any example of this. Take the example of coding, where there's been the most change happening inside companies. And coders, according to the latest papers, we can see maybe 17 times more code they're writing without necessarily appreciable increase in error rates. But that doesn't matter if you're part of a team that does two-week-long sprints where everyone has to sit down and develop what the code's going to be, and then on top of that, then you also have, you know, interactions with the marketing department and the sales department, and those happen on a five-month cadence or whatever it is. That doesn't change anything about how the organization operates. So even the most basic processes, we need to change things. And then there's the more profound issue you're pointing at, which is the blurring of roles. What are roles in this world? What does an engineer do compared to a designer compared to a, you know, a coder compared to a marketer? That's a really hard problem too. And I think that people want this to be simple. They want AI to look like previous waves of technology, right? As an IT-style solution, which is, okay, we'll buy a product that will solve a problem or automate a process. This is a much deeper, more profound change than a SaaS solution in place.

I don't think it's pure science fiction at all. I know companies already that have entirely autonomous AI systems running in parallel to their own as a long-term test to see what's happening. I had my AI agents try to make me money at one point, and they successfully did, but it was my idea, so at least I get some credit for it. In general, that's still coming to some extent, right? But I also think we don't need more AI development as much as we need the harnesses, the technologies around AI, the processes to make this kind of stuff operate. I think a world where AI is doing fully autonomous work, though, is a company that's vulnerable if they're using AI for fully autonomous work, because as good as these systems are, they're kind of like having one person or a small team of very similar people run stuff. It's part of why AI can be technically a very good writer, but as you start to read more and more LinkedIn posts by AI, you get kind of nauseous because it's all the same writing. The AI makes very similar decisions. Variation is actually really important for this stuff as well.

It's a really good question. Okay, so we can separate ideation from other parts of this process. On the ideation side, that paper's great. It has one of my experiments in it, a bunch of other people's interesting experiments all sorted in one place. And it basically shows AI generates more ideas than people on average. They're more creative than people. They're less diverse than people. And you could do things to increase that diversity, but not quite as, you know, capturing a full population. Which I think is a useful set of stuff to know because it tells you, first of all, even in ideation, if you're an outlier, right, and you're very good at this, you will beat the AI. And as a professor of entrepreneurship, this early evidence even suggests it's true. This is like the golden age of entrepreneurship. I mean, entrepreneurship, when you think about it, is really about being in the top like 0.01% in something, right? Like that's your idea, that's the thing you're good at. And then desperately trying to make sure that all the other stuff you have to do to run a company doesn't kill you, right? You're really good at coding, but you're not good at people. That could take you down. You've never understood how finance works, that's a problem. You don't know when to start a limited liability company. All of this stuff. This is what we teach people in entrepreneurship classes, how to think through this, how to deal with it. There was a study by the person who's now the chief economist at OpenAI that suggested something like 50% of Americans have had a startup idea, not pursued it. Suddenly with AI, it becomes more possible to do it, not just as a loop that does stuff, but also a nice study out of Harvard and INSEAD showed it also makes performance better for entrepreneurs who are already doing well. So I think it's actually a golden age for this thing. There's enough gaps, right, that you still need human interaction, but it makes it easier to do the kinds of entrepreneurial tasks you did before, and you have the overhead of firms that we talked about. So it's a really exciting time.

All of those things. Ideation piece, I think you want the original ideas yourself generally, otherwise you're going to come up with a very AI-ish idea. They're not bad, but the AI comes up with very similar ideas in lots of cases. Some people aren't good at ideation. So I absolutely think that you should talk through your startup idea with the AI. I teach a class, I changed it to a five-found class. So everyone is launching companies in three days was my last experiment, and the quality of ideas was up considerably from the typical of my class. The outlier ideas stayed around the same, but the ideas that were always like, oh no, I know this is not going to work, but you're going to have to discover this for yourself, those disappeared, right? And I had higher quality ideas coming out of this. So you should absolutely use variation. You should absolutely be using it to help you with all the other aspects of running a company. And you know, but there's enough human pieces that creep into entrepreneurship anyway. I don't even think you need to be that disciplined about saying, you know, what you are preserving for yourself versus what you're leaving to AI. That's not always true in every field, though.

So I think there's three things we're talking about here. One of those is the processes within the university, right? So teaching, research, right? The second is the sort of nature of the institution as it will be, right? And the third is what purpose it serves. And I think it's worth breaking each of those down. Let's start with the middle one, right? Will the university change profoundly? I mean, just as you said, I think you will underestimate how much institutions change quickly. And so institutions that are not buffered for that, are buffered from the market, government, universities, those will change over time more slowly. The university is not built for rapid change. The oldest in Europe have been through three industrial revolutions, plague, like, and they still look roughly similar to where they are. So I think it's easy to overestimate how much institutional change happens. But let's talk about the two more interesting parts: the purpose and the sort of function and what we do all day. So teaching is already changing because it has to, right? Like AI has invalidated a lot of teaching techniques that they were really good, or at least undermined them. Not invalidated them. They were still good, but we can't use them anymore. It's changed our opportunity for what we can do, and it's created chaos in classroom settings. We'll get through this, right? Like you can already see a better model ahead. There's studies out of Harvard and Stanford and also Kenya that show that AI can be a really good tutor when used as a tutor. But so my colleagues did a study in Turkey, and there's a large-scale study in China that suggests it undermines learning if people are just using it to get help with homework. So that means we'll move to AI tutors outside of class, you know, eclipsing classroom experience. Inside of class, we'll have evaluative experiences, active learning. All the academic research has suggested that's a better way to teach for the last, we've known for 20 or 30 years that active learning is a better way to teach than all lectures. We'll move to that. It won't be fast. It'll be messy. In my own classes, I have AI tutors, AI simulations, AI-powered exercises, stuff that it used to require me to build the group, and I have done this before, of 14 coders and raise a couple million dollars in funding and build something I can build in a day now. Really great opportunities, but we're not there yet. We have to get there, right? Similarly, on the research side, we're in the early days of automating research. Now every journal is overwhelmed by slop. That will change over time, but the function will change in that way. But that five years will still probably be mid-transition. Then there's the question of what purpose universities serve, and I think that is the area that has actually weirdly doubled down. So one of the things I talk about in the book are two things you need to stay human in the world of AI: deep knowledge or wide knowledge. You need to know a field really well so you have enough expertise to work with an autonomous agent, and you need enough broad-based knowledge to understand the other opportunities available to you as fields blur. And universities are very good at teaching wide and deep knowledge and very good at evaluating it, and companies are less good at that. I think there'll be more of a role for universities in the future as a result.

I think you're in danger predicting a field's success. We just don't know the answer to that question yet. It's early days. We were being way too confident about job impacts, for example. And we don't know what it looks like. We certainly don't know what it looks like over the course of an undergraduate career. In my former life before AI stuff, one of the things I did research on was careers. We did large-scale career surveys. It turns out careers are long. People do lots of different things over their career. So I think building flexibility in is more important than thinking about like, I'm betting on a job, right? Especially in the US system, you're not put into apprenticeship directly from a company from college often. I think there's an exploration piece that I expect will continue. I would say on top of all of that, like, you know, you just can't make bets. So I think there are some fields that are probably obviously in bad shape. It's hard to know what those are going to be 100%, right? So if all you're doing is writing press releases all day, if you asked me a year ago, I might have said, oh, that seems like it's going to be 100% automated. But then you start to read that turns out AI language gets to a certain point, and then does it get better than that? Maybe press releases will actually still be a human job. It's really hard to know for sure.

Oh, so the Generative AI Lab at Wharton, which I am the academic director of, has a whole bunch of free resources available. They're all open source or Creative Commons so that people can modify them as they want. And the general idea is there's a lot of modalities of teaching that we know are very successful that have been hard to apply, right? So let's take some examples. Teaching other people is a really good way to learn, as a professor in the classroom has experienced. So using the AI as a naive student you have to explain things to is one modality. Simulations are a really powerful teaching technique that's been very expensive and hard to build. That's been a lot of what I've done over the years is trying to make that happen. Very hard to do. The AI is very good at building simulations. We've got a paper coming out on that as well. Tutoring and mentoring. I have AI mentors and tutors in place. So I actually let people do all these things, right? So we have, you know, there's a teach the AI piece. There is the AI helps you. You have to build a business case with the AI's help to build a case that you know of for a problem that is illustrative of that problem. The AI evaluates something you do. The AI asks you questions to do an after-action report. Like there's lots of places where if I would apply, if I had infinite TAs and I could force the student to sit with them, what would I have a patient tutor do that could also code? Like that is an unlocking idea. Now you don't want to just get excited about the technology, quad technology, right? Like the issue is that you want to find use cases for this that are themselves pedagogically sound, valuable, and then you can spend the time to make sure they work rather than just deploying them randomly.

I think it's every field because we move from a level of your own knowledge and guidance guides the AI to the AI's judgment to creativity and guidance guides the AI. And I think that that changes a lot of things, right? Because it makes all work, management work, all work is delegation. So listen, I'm a management professor, I would say that, but that is where the evidence is pointing as well. So I think that that changes almost any kind of complex work, right? If it was a team doing this, it will be a team of AIs assisting you in getting this done. All the evidence we have is that the most exposed fields change the most. Consulting, banking, there's a lot of incentive to get those things automated. There's not a lot of love for the workers in the space, right? They're not unionized. They don't have a lot of protections. I think we'll see a lot of changes happening there. But will that increase or decrease employment? I don't know. But the more sort of life of the mind knowledge worker job is, and the more data gathering, more analysis would help you, the more AI plays a role.