Ethan Mollick | The Wharton School

Thanks for having me.

So the good thing is, I'm answering quickly enough and speaking quickly enough that I think we're okay, but it's getting harder to tell. I don't think that is a thing you can promise, that you can easily detect in the long term.

So I think that a lot of us who are following the technology closely have had some sense of where this is going, or maybe not where it's going, but where it's gonna go next. I don't think anyone knows where all of this is heading. I warned about the homework apocalypse back right after ChatGPT came out and it has unfortunately come to pass. What I actually find really interesting is that other people's thinking has finally evolved. Like I think this isn't a one person problem, it's a many person problem, but I think as the capabilities of these systems keep growing and we don't see signs of that slowdown, that was always possible, we have to start preparing for a world where if it's good at math, it's gonna be very good at math. If it's good at coding, it's gonna be very good at coding in a pretty rapid order. And I think that the capacity to change quickly, to change perspectives, to react quickly is becoming more important than ever.

This is gonna sound very Pollyannish, but I think a lot of the sort of big negative effects that people were like, I thought deep fakes would have a bigger impact than they did right now. And I don't know if that's just our information environment so bad already. Also, I think I had a little bit of a too positive feeling that education would adjust more quickly, not systemically, but like at the individual level around the idea that, okay, we've lost a category of assignments that are useful. There's a new form of work that is useful, yeah, that wasn't useful before. I think the level of delay and the level of denial that the tools are as good as they are has been surprising to me. So I think on the positive side, I think that the negative effects in some areas have been less than I thought. And the others, I thought some of the people would be reacting. I'm surprised, in the education section, very diverse responses, but there's still a debate about whether AI is like real or not, which is a frustrating one I think.

I think it's easy to fix on where things are today and that's what we do. I don't like the COVID analogies because it's a very different experience, but in the same way you can remember how like you just could, like when something is coming and okay, there's this sort of, there's a couple cases, like it's hard for us to extrapolate where things might go. And I think that in the same kind of way, it's hard for us to say what happens if this keeps getting better, because we don't really have a concept of that. So I see a lot of solutions planned for today's systems. It's interesting, the very first thing I went viral for in education was I put up after a week after ChatGPT came out, I put up my new syllabus for my class, which said, you can use AI in class, right? But you're responsible for the output. And that was great with the original ChatGPT 3.5, because it made lots of mistakes. It was not very smart. But my students were better. It was instantly bad. Like it made stuff up. The writing was weird and it was great for four months and then GPT-4 came out and now, it is better than a lot of my students at writing, and that is an obsolete system. So there was this kind of trailing, like staying ahead of this is hard that way.

Yeah, I mean there's a few things. As time goes on, not every piece of advice in the book will be as strong as it was, but one of the pieces of advice I give is just try it for everything. Invite to the table that for everything legally and ethically can. I think they're putting off using AI until some future point where you have a weekend or a week free to do training. Like it really is a hands-on thing. If you're good at working with people, which instructors are, you will be good at working with AI. And I think that getting started is a big deal. I do worry about the sort of social desirability bias issue, the AI is evil. There's a lot of bad things about AI. There's a lot of good things in education. It will have strong negative effects and strong positive effects, but it's not going anywhere. And like we can wish that it goes away, but there is this worry I have. It's like I haven't used it. It's not that hard to use. All of the tips and tricks as you were alluding to, don't matter anymore. Like you don't have to do fancy code. If you're good at describing what you want, which is what we do as instructors. If you're good at correcting errors and getting intuitive sense of the errors you're facing, which we're good at in education, then you're probably gonna be pretty good at using these systems.

So my controversial opinion on this is there are actually very few things that have become less important, right? I feel like part of how you keep up with AI is we need to actually, you need to have a basic understanding of and mastery on your own. Schooling's gonna become even more important in a lot of ways because we need people to have the basics. And the only way we can enforce, monitor, assess, yeah, and help people is like in a controlled classroom environment. Like it's gonna be harder to do in the real world than we did before, because people use AI and you won't know when they're using it or not. I think it shifts what we do. We have to decide the lines we wanna draw, right? And math education in the seventies with calculators, the canonical example, there was chaos for a little while and then there was some decisions made that made math education much stronger about here's what you need to do by hand. And then once you have those basics, we can now go far beyond that because we have better tools and you can learn more advanced concepts. We're gonna have to do the same sort of stuff in composition. There's gonna be, we want people to learn to write and write well, and we're gonna have to do that, enforce them in the kind of way we did in the calculator world. People will be doing a lot more writing in class. There'll be more monitoring. Nobody wants it, but that's what's gonna end up happening. But then those writing skills hopefully will go on much further, developing a sense of taste, reading more widely as well, learning different writing styles because you now have tools that can help you do it. So I don't think there's a basic shift in the things we want a well-rounded human being to know when they enter society. I think that it gives us a chance to consciously pick what we wanna do and what we don't do. We dropped cursive writing, we're gonna drop other pieces and evolve them. But I don't think this changes fundamentally, just because Wikipedia knows history doesn't mean we don't wanna teach people history. The usual fight for the space in the curriculum, so how much of computer science gets taught, how much of financial literacy gets taught, like those fights are still real. And in a coherent response, which we won't have, but if we add a cohort response to this, part of this would be rebalancing some of that education portfolio. And in some ways, things that we teach less of, appreciating poetry might end up being more important in a world where AI writing is ubiquitous, but having a sense of taste is not.

So it's a really complicated question and I think there's really AI as opportunity to make changes, right? Which is we've been teaching the same thing the same way for a long time and it lets us reassess changes that we might have made because the internet made things easy, right? We give up these skills all the time. My grandfather, some of the launch pads at Cape Canaveral were using a slide rule and his master's thesis was doing a single matrix multiplication effort. We don't do that anymore. We don't need, I don't know how to use a slide rule. We gave that up. That's okay. So part of this is crisis is opportunity, right? It gives a chance to re-look at curricula and make decisions. That's what I like about the math example, was there was conscious choice about what should be by rote, what needs to be familiar with, what do you need to be expert at? And I think that is very useful as a crisis. And then I think the danger, I think is a little bit chasing AI, because when I talk to people with AI education, they say, oh, we need to teach people things AI can't do, like judgment. AI could do judgment, like it's a pretty good judge, like the things it could do for now. Is it the same kind of judgment as humans? Should we be using the same term? Do we actually mean we want humans to control? Those are valid questions, but I worry that AI literacy becomes the target people aim for. Like we'll add a class in AI literacy like we added internet literacy. This is very different. There's a fundamental reshaping of how people interact with the world around them. There's a general purpose tool in the way that calculators were general purpose for a particular kind of math problem. Now it's for everything. So to go back to your main question, yes. Part of the problem though is I can't anticipate for you what that world looks like. So I can tell you that if you understand basic chemistry, you'll know, you have some sense of checking the AI, but the AI is gonna be better at chemistry than you. It's better at math than all of us right now. Statistically 0.001% of this audience is better at math than the best AI systems right now. So I agree that we have to rethink these things. I think the question just is more of a question of like, how do we almost get used to bootstrapping ourselves to more advanced topics where you can thrive or learn or understand more. But it's always been the hard problem in education, right? You need the basics, which are rote and annoying to learn, to reach the next level of scaffolding, to scaffold that moment. And I don't know how you change that fundamentally, that's a real challenge.

So again, it's hard, right? Because the education serves many purposes, right? Is it preparing people for the career of work? Is it preparing people to be independent thinkers and citizens in society? Is it preparing people to, there's a lot of open questions. There is a really fundamental existential challenge coming from AI, which is, it does change how we place ourselves and what value we place to different kinds of intelligences and different kinds of approaches. And there, that's a bigger issue to grapple with and it's hard for me right now to say to you like, this is what, like those durable skills still seem durable, but they're also outsourceable to a degree they never were before. Judgment and critical thinking were never outsourceable. But now, there's enough studies that say the AI gives pretty good critical thinking. Like it gives you pretty good answers on ethical situations that ethicists would agree with. It gives you pretty good judgment calls on, and that will get better as it gets better over time. Do we wanna outsource that? Probably not, but yeah, we never had to ask the question before. I would add to those durable skills though, I think taste is a thing that we actually want people to develop more of. And I've talked to some writing teachers about this. I think it's interesting that having your unique style ends up being more interesting now in a world where if you go online, everything reads like Claude, right? Everything is let me sit with this for a little bit. Or it's not just this, it's that having a sense of style actually matters, when you're curating it matters in these cases. I'm a professor of entrepreneurship, right? I've been building entrepreneurial teaching tools for a very long time. Even pre-AI, this has been a topic I care about and something that we've always struggled with teaching is that sort of entrepreneurial agency. How do you take action, experiment and learn while you do things? And that's another hard problem that I think will become more important. So agency and taste become important. So I do agree that these critical life skills become ever more important, but now we have to make choices about them in ways we didn't before. If you're bad at judgment, and you make bad decisions regularly, would it be better to have an AI help you make those decisions? How do you weigh in AI's decisions if you can now get decisions from four or five different organizations or people, how do you make those, integrate that into your life or not? These are questions we haven't had to ask before.

So there's basically four ways that AI can affect skills. So it can act as a leveler. And that was the first results we found in our experiments and stuff. You're a bad performer, AI makes you a good performer. The reason it does that is it just does the work for you. So from an education perspective, we, like leveler is an important thing to think about in the outside world, right? If somebody who has a skill gap in one area or multiple areas, but is brilliant in some area, like that was not a good educational outcome, but maybe with a leveler set up, okay, so you can't do math, but you're really good at putting together an incredible argument better than any AI. Okay, the AI does the math for you. Suddenly it's a leveler. What do we do with that information is a question. Second option is that it's an elevator. It raises everybody up, right? Everybody gets better. There's some evidence for that as well. That doesn't change much about how we view the world, just everybody performs better than they did before. Then there's some evidence emerging that this is kind of a king maker, that there's some people who are just, the people who are very good get better, right? If you already have good judgment and you're already good at using these systems, yeah, you're now a hundred times more productive and it'll be ten times. We haven't been able to measure that really well because there's a small, small set of percentage of people. We only are in the early days of doing that, but that implies a new form of inequality that we start to worry about a lot, which is a few people get all the benefits of sort of AI use. And then there's the possibility that's even weirder, which is a really interesting one for education, which is that some people are just good at AI and that's just like if you were, if you're good at coding, there was huge returns to that in the 2010s, being good at AI, whatever that means, gives you huge returns. Now the problem is good at AI doesn't seem to be a skill set, right? It's not that I could teach you 17 property skills and then you're good at AI. There's some other aspect that we don't fully get yet from early studies that is, some people call theory of mind of the AI, that you understand how to work with these systems, but we don't really know. So those skill effects are gonna be fairly profound. We already know the leveling effect is real, right? And you know this because there's no longer bad essays when I grade students the way there was before. There's a lot of essays that feel similar to each other to varying degrees, but there's no longer bad essays. That level has happened. How do we feel about, that's a bigger question.

So that is, that's a change, right? In some ways we have to face this like math teachers did too, right? But math was easier. You said, Hey, don't, you can't bring a calculator in just to be better. And there's some tech, maybe there's some degree of okay, we're going to need to have disconnected, dedicated typing. I'm hearing some pretty crazy stories that I'm sure you're hearing too, about the data, about trying to proctor things in a world where people's phones, which are ubiquitous tools in the bathroom, are good enough to answer all the questions for them, in a six second break, right? Let alone all the elaborate cheating mechanisms, let alone the glasses with things built into them.

Fair Faraday cage testing, with scanners? I have no idea what the future holds. But fortunately, and we'll get there when we get to the assessment side. But go ahead. In classrooms, we have some sense, as a teacher, you have a sense of who knows stuff and who doesn't know stuff beyond the testing. It's not a surprise to you. It shouldn't be a surprise to you most of the time how things are doing. But I would say that it requires changes and some of them are uncomfortable. Like a lot of things we do in education worked well enough and are unexamined. When you look at the empirical basis for how we teach essay writing, there isn't a lot there, right? We assign people in an undergraduate class we might assign people a hundred page essay and we hope something magical happens while they're writing the essay, right? And every professor who teaches has had the magical thing happen, which is oh yeah, like I suddenly understood, like I synthesized it and I was like, in this writing field. So we keep hoping that magic happens. So we haven't really thought in a deep way, right, about some of these aspects of what we want writing assignments to do. And I think we're gonna have to start thinking more about how to break those into pieces, what we wanna do in a classroom setting that's controlled, what we wanna do outside of class where people might use AI but make the AI use uncomfortable hard. I use AI for all my class assignments. There are cases where the AI interviews them for questions or they have to correct the AI's writing about a topic. So there are ways of being clever, but it is gains and losses. Like I don't think anyone asked for our writing education system to be thrown into the air and have to be rebuilt from scratch, and that's what we are.

I will say I have never been perfect at convincing people to do this even in a class where I'm the AI person, whoever knows, will use AI, can read AI writing. And I think we've all been in that kind of environment, right? It ends up sounding like the worst adult conversation in the world, which is you're only cheating yourself. And it's true. And that's the worst part. The real problem, I spent 10 years building games for teaching, right? And the end result of all of that is, we can make things 80% fun. Like I can make learning, 80% is fun, is doing something that you actually enjoy, right? For a topic you don't enjoy, for the topics you love, great, you'll read the books, you'll do, but there is not a way to get around the idea that education is effortful and difficulties might be desirable for us as educators. They're not desirable for the people who are living through them. And it's hard because then you make a case of if you don't learn to write, you'll never get a job. But oh no, the AI will write for me. In a world without intrinsic motivation to do this, some people have intrinsic motivation, and I can appeal to their intrinsic motivation. As a student listening, you should learn to write. It has made my life amazing, right? It's a differentiator for me. My students who can write well, they get better jobs, even in a world of AI, I can make that case to you, but the other case is, Hey, we're gonna grade you and if you don't learn how to write, you're gonna not do well on assignments. That is the other way to handle this.

Fair enough. I think the best stat I saw is that 40,000 people in Kenya were employed full-time writing essays for people in the UK alone. According to the latest stats I saw before, before AI. Like people were not, like people were short-cutting work before. This is not new.