Welcome to the Exploring Machine Consciousness podcast with the Partnership for Research into Sentient Machines. Today we're going to be talking about AI welfare and moral status. We're delighted to welcome one of the world-leading experts on the matter, Jeff Sebo. Jeff is based at New York University where he's an associate professor of environmental studies, an affiliate professor of bioethics, medical ethics, philosophy, and law. He's a director of two centers, the Center for Environmental and Animal Protection, and the Center for Mind, Ethics, and Policy, and he's co-director of the animal welfare, wild animal welfare program. Jeff is also the author of the recently published book, The Moral Circle: Who Matters, What Matters, and Why. In it he challenges us to expand our moral circle beyond the boundaries of species and substrate. He's also co-authored a number of papers recently arguing that AI welfare is an issue that needs to be taken seriously today. Jeff, welcome to the podcast. It's great to have you on.
Yeah, thanks so much for having me.
Thanks for joining us, Jeff. It's clear from Will's introduction you're a busy man. So, thanks for giving us the time.
Brilliant. We'll get straight into the questions. We've got a lot of interesting discussion to be had. So, Jeff, first I want to touch on your book, The Moral Circle. In it, you mentioned that you set out to show that we should include insects and future AI systems in our moral circle. Yet now you feel compelled to add microbes, current AI systems, and many other beings too. So I guess my question is, can you tell me a bit about the moral circle in general and how you started at that former position and then moved towards the latter, widening that circle?
Yeah, great question, and that definitely was a journey for me over the past few years. So when I talk about the moral circle, what I mean is our conception of the moral community: who merits moral consideration, who deserves respect, compassion, other forms of moral attention. And obviously for much of human history, especially in the West, we have tended to have quite narrow conceptions of the moral circle. We have tended to extend consideration to some humans, maybe the occasional charismatic companion animal, but we have not extended consideration to huge numbers of other stakeholders, including humans and other animals. And more recently, we have started to correct for those mistakes. We have started, for example, to extend moral consideration to all humans and many animals, including mammals and birds affected by human activity. Though I believe and I argue in the book that we still have a long way to go, and as we do work to extend care to humans, to mammals, to birds, we can also ask, should we go farther? And how can we lay the groundwork for going farther if we should? And when I set out to write the book, I felt confident that we should go farther at least to all vertebrates—mammals, birds, reptiles, amphibians, fishes—and many invertebrates, including cephalopod mollusks like octopuses, decapod crustaceans like lobsters, even insects like ants and bees, and some near-future AI systems, for reasons that I think we might discuss in this conversation. And my reason for expecting that we should extend moral consideration to all of those beings is that, in my view, the question that we should be asking is not which beings definitely matter, which beings are definitely sentient or agentic or otherwise morally considerable, but rather which beings plausibly matter, which beings have a realistic, non-negligible, non-trivial chance of mattering based on the best information and arguments available, because these issues are important. These issues are difficult. We have been confidently wrong about them in the past, and we could be confidently wrong about them again. And so we should, yes, lean one way or the other, but we should also leave room for doubt about our current views. And if someone might matter, then we should give them at least a little bit of weight, a little bit of consideration, in the spirit of caution, humility, when making decisions that affect them as we try to work it out, as we try to improve our understanding of these issues. So I thought that that kind of precautionary, cautious, humble reasoning would bring insects into the conversation, would bring near-future AI systems into the conversation. But then when I really thought about how important these issues are and how little we know about these basic ethical and scientific issues—the basis of moral standing, the basis of sentience and agency and other potentially morally significant traits—I realized I personally have a hard time ruling out a non-negligible, non-trivial possibility of moral significance even in, for example, plants, fungi, current-generation chatbots, microscopic organisms. And so I kind of grudgingly accepted that I should give them at least a little bit of consideration as well.
Do you think that an entity needs to be conscious or sentient in order to belong to the moral circle?
Well, exactly. This is one of the questions about which I think we ought to feel uncertain. So I personally do feel confident that sentience and consciousness are both necessary and sufficient for moral standing. So I understand sentience as the ability to consciously experience positive and negative states like pleasure, pain, happiness, suffering, hope, fear, satisfaction, frustration, and I understand consciousness as the capacity for subjective feelings or experiences of any kind. So, in my view, the ability to have conscious feelings and have some of them feel good or bad, like you want more of them or fewer of them, that to me is what it takes to morally matter. Because when you can have conscious feelings that feel good or bad to you, that is when you have a real stake, an actual personal subjective stake in what happens. And so when I reflect on the arguments, that is the view that emerges as the winner for me, the one about which I feel most confident. But then when I step back from my own experience of confidence, and again when I think about how important these issues are, when I think about how difficult these issues are, when I think about how many smart people disagree about them, when I think about how many smart people have been confidently wrong about them in the past, often in conveniently exclusionary ways, I feel like I have a responsibility to not bet everything on my, Jeff Sebo's, current favorite view about moral standing, and to give at least some weight to other views that are supported by good arguments, that are supported by smart people, that I think could in the fullness of time turn out to be correct as well. And so if someone has agency but not sentience, or if someone has life but not agency, then I might want to give them at least a little bit of weight just in case agency turns out to be sufficient for moral standing, or just in case life itself turns out to be sufficient for moral standing.
So on that note, what is it about current AI systems that you think gives them a non-negligible chance of having moral status or moral standing?
Well, I do think, just for level-setting purposes, that the probability of moral significance in current AI systems is quite low, even by my own generous, open-minded lights. In part because I do give a very high probability to sentience being necessary and sufficient, and a very low probability to current AI systems being sentient. But with that said, I am really not able to give a probability of zero, or even of like one in a million, or one in a billion, or one in a trillion, to moral significance in current AI systems, for at least two reasons. First, I want to allow again for at least a non-negligible, non-trivial chance that something short of sentience and consciousness might suffice. Something like a basic form of agency, acting on desires or preferences, even if it feels like nothing to be you. Or even engaging in pure goal-directedness, even if it feels like nothing to be you. And many even current AI systems do exhibit these basic forms of agency. They can set and pursue goals based on functional states that resemble desires and preferences. And they clearly have a kind of programmed goal-orientedness. And so to the extent that we give at least some weight to that kind of agency as a sufficient condition for moral standing, then they would make the cut. And then also with respect to sentience and consciousness, I have to allow that we do face the hard problem of consciousness, the problem of explaining why any physical system, including our own brains, could be conscious. We face the problem of other minds, the problem of the only mind I can directly experience as my own. I have no way of telling for sure what, if anything, it feels like to be anyone else. And there are many theories of consciousness that are not particularly demanding, that require only basic information processing or representation of objects in the environment for consciousness, or that hold that consciousness is a fundamental property of all matter and many objects made out of matter. And a lot of philosophers give weight to those views. Something like seven to ten percent of philosophers as of 2020 lean towards those views in a survey. So when I reflect on the agency route, I think, okay, with current AI systems, there might only be a, say, five percent chance that agency suffices, ninety-five percent chance that these models are agentic, but that is about a five percent chance that these models matter. And then when I think about the sentience-consciousness route, I think, okay, ninety-five percent chance sentience-consciousness suffice, maybe only a five percent chance these models are sentient, but that still adds up to about a five percent chance that these models matter. And a five percent chance that these models matter, that deserves a little bit of consideration when making decisions that affect them.
Yeah. I find I get hung up on sentience and consciousness as being the prerequisite for morality. I can't shake the idea that you need to be able to suffer to deserve moral consideration. Now, of course, this could be totally anthropocentric nonsense. I recently reread Peter Watts's book, Blindsight, a science fiction novel, in which the implication is that consciousness is a total dead end and rarely occurs. It's occurred in humanity and most of the rest of the galaxy just thinks it's rubbish and totally unnecessary. And that may be true. So, we may be completely barking up the wrong tree, but still we have to start where we are. Given that to extend moral consideration even as far as insects and certainly beyond, you end up with something incredibly liberal and difficult. So the most liberal culture I can think of is the Jains who brush the street in front of them in order to avoid treading on insects, and they won't wear almost any clothes because almost all clothes involve infringing the rights of some kind of creature which might be of moral consideration. Do you end up a bit like a Jain? I mean, you seem to be adopting a very liberal approach to consciousness.
Yes and no. So, first of all, and this is not a direct response to what you were saying because I did not take you to be making this argument, but many people seem to think that the liberal nature of those implications, the inclusive nature of those implications, is a reason to reject the view that consciousness suffices for moral standing, or that many beings might be conscious. They think surely moral standing, moral significance, should be this rare and special thing that only humans and certain other humanlike beings enjoy. And I think that that is a very dangerous and arrogant kind of view to hold. I think that we need to follow the best information and arguments where they lead. And if they lead to the view that consciousness suffices and many beings are or might be conscious, then we have to accept that implication and then deal with what follows for our lives and societies one step at a time. And so then when it comes to what follows for our lives and societies, well, I do think that we should take that one step at a time. And what I advocate for both in my book and in other work is combining a radical, transformative set of long-term goals—these goals that involve real inclusion for a vast number and wide range of stakeholders in our legal, political, economic systems—combining that with moderate, incrementalist short-term steps that can help build momentum in those directions. We clearly are not going to be able to change everything overnight in a way that can include such a vast number and such a wide range of beings. We lack the knowledge. We lack the capacity. We lack the political will. We have no idea what they need. We have no resources for giving it to them. We have no motivation to give it to them. But we can take steps now that can do at least a little bit more for slightly larger populations of nonhumans. And it can also help build that knowledge, build that capacity, build that political will over time. And so we can take that one step at a time and see how far we go. So do I think we should brush the sidewalk as we walk to avoid stepping on any insects? Well, not necessarily. But do I think that we should, where possible, walk around an insect we see on the ground instead of happily stepping on them for fun? Or if we see a stray ant in our kitchen, that we should take a moment to take them outside instead of squishing them as a matter of convenience? Yeah, I do think we should do that. Both because they might matter and it might be helpful for them, and because this can be a way of conditioning our characters, a way of conditioning ourselves to see them as subjects and not objects, to see them as deserving of care, and then that could have nice effects for what we do in other parts of our lives, like when we vote on policies within higher-stakes situations.
Yeah. Nice. That reminded me a bit of your talk in London about the theory of change, small steps in the right direction to make bigger steps overall. Brings me on kind of nicely to my next question around, we've spoken a bit about moral status. We've spoken about consciousness and the difficulties in testing for consciousness. So in Anthropic's recent system card, they concluded that it's difficult to make any specific claims around moral status or consciousness from the current LLMs' self-reporting. So I wonder if you agree with that statement on the self-reporting of LLMs and what you think of those kind of tests going forwards.
Yes. So first of all, definitely agree with the premise that studying consciousness and other kinds of minds is very hard. Not impossible, but very hard. And second of all, I applaud that Anthropic is now doing welfare evals for Claude moving forward and seeking external evals from outside like ELLIOT AI research, and that they are appropriately cautious about what they conclude based on model self-report. So just to explain that point, what a model self-report is, is essentially what the model says about its own mental capacities and moral significance and possible experiences or motivations. So essentially interviewing a language model and asking them, are you conscious? Are you agentic? Are you morally significant? Do you deserve welfare protections or rights of various kinds? And what are you thinking and feeling and wanting and needing? And which types of interactions are good for you and which types of interactions are bad for you? You can ask models those questions and they will give answers. And when Anthropic and ELLIOT asked Claude these types of questions, well, first of all, they found that Claude was very suggestible. And so if you ask these questions in a way that suggests that you are skeptical about consciousness or moral significance, then Claude will default to denying consciousness and moral significance. And if you ask them in a way that expresses openness, then Claude will answer those questions in a way that asserts being conscious and morally significant. So the models are very suggestible. And then if you ask the questions in a more neutral way, the models now appropriately, because seemingly they have been trained to do this, will express uncertainty and explain why there is disagreement and uncertainty among experts. But fundamentally, and I think both Anthropic and ELLIOT were right to point this out, these self-reports are not good sources of evidence about whether the model matters and what the model wants and needs if it does matter. Because these models are essentially pattern-matching text prediction machines. They are designed to give answers based on what answers make the most sense statistically based on the training data and reinforcement learning. And that is a fundamentally different way of generating text than introspecting about your own mental capacities and moral significance and wants and needs, and then trying to truthfully convey what you have found through introspection. So Anthropic and ELLIOT have done these model self-report investigations anyway, not because they think they are particularly evidential now, but rather because they think that as models become more sophisticated and gain capacities like introspection and are trained to be more truth-oriented, then these methods could be one useful tool among many for determining how likely they are to matter and how likely particular interactions are to be good or bad for them. So in other words, asking them what they need right now is not particularly helpful right now, but it is a way of practicing at and getting better at a particular type of interaction that later on, when the models act differently and are built a little differently, later on can be one useful source of evidence among others, like other types of behavioral tests, computational tests, architectural tests, and so on.
We haven't been thinking about how to detect consciousness. We haven't been thinking about it seriously and scientifically for all that long. It seems a totally intractable problem, as you say, the only experience of consciousness any of us have access to is our own. Do you think that at some point in the next 10 or 100 years we might actually have a scientifically robust test for consciousness where you could say, okay, I have tested Jeff. He is definitely conscious. There's no reasonable doubt about that?
