Allow me to introduce you to Pete. Pete is my Tamagotchi. Some of you may remember these. A Tamagotchi is a kind of simple digital pet you can care for. So with Pete, I need to push specific buttons at specific times to feed him and play with him, generally take care of him. And if I do a good job, Pete can have a long and happy life. And if I do a bad job, Pete could die. And honestly, that would make me sad because I care about this simple piece of technology.
If I was giving my talk right now and I knew Pete was somewhere feeling hungry and neglected, I would feel distracted, maybe even a little guilty, even though I know Pete is a toy and this is all a kind of make-believe.
And until recently, this is generally how we were able to interact with digital companions of all kinds. We might care about them. We might even empathize with them. But at the end of the day, we know these are objects we can treat as we like, not subjects to whom we have moral responsibilities.
However, the rise of advanced artificial intelligence has started to blur the lines between digital objects and digital subjects, even for experts. We now have generative models that can have long organic conversations with us, that can create artwork like poetry and movies and music, that can take and pass standardized tests in a range of academic fields.
And yes, the way they do that is different from the way we do that. When we create art, we like to think we do that by expressing our innermost thoughts and feelings. And when they create art, they do it by pattern matching.
Still, these models are already remarkably capable. And even now, even today, it can be tempting to attribute real thoughts and feelings to them. And by the way, yes, I brought out a stool specifically so Pete can enjoy this nice theater during my talk.
So these developments are now leading some experts to ask when, if ever, will AI systems acquire morally relevant, significant capacities like sentience, the ability to consciously experience positive and negative states like happiness and suffering.
Because if and when that day comes, they will be subjects and we will have moral responsibilities to them.
Now, why does sentience matter so much? Well, as the philosopher Jeremy Bentham said in 1789 about the moral status of other animals, the question is not can they reason, nor can they talk, but can they suffer? Because when you can suffer, you can be harmed. And when you can be harmed, I have a responsibility not to harm you unnecessarily.
So, will AI systems be sentient anytime soon? I direct the Center for Mind, Ethics, and Policy at New York University, and I spent the past two years researching this question with my fellow philosopher, Robert Long, and a multidisciplinary team. And I now feel convinced that the answer to this question is a clear maybe.
And if the answer is a clear maybe, then we have a problem. So let me take you step by step through a philosophical analysis of AI sentience that makes three basic points. First, we still have real uncertainty about the nature of sentience. Second, we still have real uncertainty about the future of AI. And third, when in doubt, we should exercise caution.
So, we can start by asking about the nature of sentience and the possibility of AI sentience. Imagine that we one day build a much more sophisticated Pete, a Pete 10.0, with a physical body and advanced and integrated capacities for perception and attention and learning and memory and self-awareness and agency. And now imagine that this Pete is having a conversation with you one day and in a quiet vulnerable moment expresses a deep fear of abandonment, not because of pattern matching but rather because of cognitive faculties that function like thoughts and feelings. Is that Pete sentient?
Well, it depends on who you ask. Some people say no. AI sentience is a physical impossibility because sentience requires consciousness. It needs to feel like something to be you. And consciousness requires being made out of carbon-based cells, not silicon-based chips. And there are different reasons why you might think that, but one is that only materials like carbon-based cells are able to produce the specific kinds of chemical and electrical signals that we find in human and animal brains. Maybe those signals are the key to consciousness. And if so, then maybe even Pete 10.0 is still only simulating fear.
But other people say, "No, no, no, not so fast. AI sentience is a physical possibility because sentience and consciousness require at most certain computational functions that at least in principle can exist in carbon or silicon.
And different theories focus on different functions, but the list standardly includes physical embodiment, perception, attention, learning, memory, self-awareness, agency, perhaps a global workspace that integrates it all together. And these theories may disagree about whether a Pete 5.0 that has some but not all of those functions is sentient. But these theories will generally agree that a Pete 10.0 that has all of these functions is sentient.
So how can we tell which of these views about the nature of sentience and the possibility of AI sentience is correct? We may never be able to tell for sure. Determining which other minds are conscious and sentient is famously one of the hardest problems in all of philosophy and science due in part to the problem of other minds. The only mind any of us can directly access is our own. And that significantly limits our ability to ever know for sure what, if anything, it feels like to be anyone else, even another human.
We also have a long track record of bias and ignorance about other minds, including a tendency to over-attribute sentience to beings like chatbots, especially when they talk with us and we use them as companions, and a tendency to under-attribute sentience to beings like non-human animals, especially when they have really different anatomies and behaviors than us and we use them as commodities.
So for me, when I think about how important and difficult this issue is and how much bias and ignorance we have about it, I think the only responsible stance we can take now and for the foreseeable future is humility. We might lean one way or lean the other way, but we should keep an open mind and leave room for doubt. And that means that even if I find it plausible, which I do not, that only carbon-based beings can be sentient, I should still allow for at least a realistic possibility, at least a one in 10 chance that I am wrong and that a sufficiently complex silicon-based being can be sentient, too.
But then if we do leave room for that possibility, that brings us to our second question about the future of AI and when these kinds of complex silicon-based beings will exist. Here is what we know. We are currently in the midst of rapid progress in AI and developers are investing billions in further progress and they may or may not be trying to build sentience specifically. But they are trying to build many of the computational functions we associate with sentience and the faster they make progress on that, the faster the probability of sentience will go up.
So how fast will the progress be? Well, it depends on who you ask. Some people say we will slow down from here because most current progress is due to increasing the scale of generative models and there will soon be diminishing returns with further scaling. There will also be diminishing support for further scaling because of how expensive these models are becoming and there will be no major breakthroughs that totally change the game anytime soon. But other people say no, no, no, no, not so fast. We will actually go faster from here. There will be increasing returns with further scaling, increasing support for further scaling because of how beneficial the models are becoming, and multiple major breakthroughs in the near future, especially when armies of highly capable AI systems become involved in their own further development.
So, how can we tell which of these predictions about the future of AI is correct? We might not know until it happens. Predicting the future of technology is always difficult. And predicting the future of AI is particularly difficult. But I will say this. If you right now feel confident that we will not in 2035 have AI systems with many of the same computational functions I attributed to Pete 10.0, please keep in mind many people felt similarly confident in 2015 that we would not by 2025 have AI systems that can have conversations, create art, pass standardized tests. And yet here we are ahead of schedule.
So this predicament is like the previous one. It calls for caution and humility. And that means that even if I feel confident that we will slow down from here, which I do not, I should still allow for at least a realistic possibility, at least a one in 10 chance that I am wrong, that we will not slow down and that we will in fact build these complex AI systems soon, within a decade or so. But then if we allow for both of these possibilities, that brings us to our third and final question. What do we do now? How do we respond to what we can expect will be ongoing significant disagreement and uncertainty both about the nature of sentience and about the future of AI.
I have suggested that humility requires allowing for conservatively at least a one in 10 chance that a sufficiently complex AI system can be sentient and at least a one in 10 chance that such a system can exist in the near future. But those possibilities together yield at least a one in a hundred chance that a sentient AI system will exist in the near future. What do we do with that possibility?
Well, as an ethicist, I can tell you that fortunately, this question is easier to answer because every expert in every domain agrees, when in doubt, exercise caution. In other words, when we are making important, high-stakes decisions involving risk and uncertainty, we should take reasonable proportionate measures to consider and mitigate clear risks in advance. We do this all the time. We do it in our own personal lives when contemplating taking a new medication that has a one in a hundred or one in a thousand or even one in 10,000 chance of producing a fatal side effect.
We do it on a variety of policy domains with respect to public health, when thinking about pandemic risks, with respect to the environment, when thinking about climate change risks. We increasingly think about animal welfare risks. If there is a realistic possibility that octopuses are sentient and octopus farming would harm them, we can and should and increasingly do consider that as one factor among many when making decisions.
We also increasingly consider AI safety risks. If there is a realistic possibility that the further development, deployment, and scaling of AI will cause massive and unnecessary harm to humans and other animals, we can and should and increasingly do consider that as one factor among many when making decisions. So if we rightly consider all of these risks, including animal welfare risks and AI safety risks, should we also consider AI welfare risks? The risk that the further development, deployment, and scaling of AI will both create and then harm sentient AI systems.
It follows from my analysis here today that we should, if there is anything close to anything in the ballpark of anything within a couple of orders of magnitude of a one in a hundred chance of sentient AI in the near future. This is clearly the kind of non-negligible risk that we have a responsibility to seriously consider.
Now what would it mean to consider this risk? It means that we should take some time to prepare in advance for the possible emergence of sentient AI as a precautionary measure for leaders in this space, AI companies. It means taking three basic steps now. First, accept this is a problem and say so out loud. That one is pretty easy.
Second, start assessing your AI systems for architectural evidence of perception and attention and learning and memory and other features that would increase the probability of sentience.
And third, start preparing policies and procedures that would allow you to treat your AI systems with respect and compassion if and when they become sufficiently likely to be sentient.
As for the rest of us, we can keep in mind that we may be the first generation in the history of human civilization to interact with actual digital subjects. And if and when that time comes, oh, look who needs a snack. I told you this might happen. And I told you I would be distracted. But actually this is a perfect illustration of my point because the point is when this happens we can expect to keep making mistakes. We can expect to keep having skewed priorities. We can expect to keep feeling tempted to over-attribute sentience to objects and to under-attribute sentience to subjects.
And so the more we can do now to prepare for this strange confusing new reality, to cultivate a calibrated set of psychological dispositions and social norms so that we can adapt gracefully, the better off everyone will be. And actually, as long as I keep perspective about all this, it gives me more reason to take care of Pete and other digital companions today. Not because Pete is sentient. Pete is still very likely to only be an object no matter how many cute animations he may have. But rather because now taking care of Pete can also be an opportunity to practice taking care of other digital minds who may well in my lifetime deserve moral consideration.
Thank you.
