Rethinking Consciousness: AI, Empathy and Ethics - Jeff Sebo

Well, thank you so much for having me here.

Indeed. Yeah, yeah, my pleasure. Looking forward to talking.

Yes, I think there is that kind of concern, which is of course related to other concerns. Will it be demanding? Will it be overwhelming? And we might talk about that too. But I think your first question speaks to this sense that moral status is rare and that makes it special. And the fact that we and maybe certain other animals like mammals and birds have moral status that makes us stand out from all of the other objects and entities in the universe. It gives us a special sort of significance. And we might worry that if it turns out that we share a world with a vast number, wide range of other beings that in their own way have moral status too, then in addition to making morality more demanding, more overwhelming, it might sort of cheapen our own perception of our own moral significance. It might make us feel less special, and that might be a kind of existential obstacle that we have to overcome.

Now, I think that is an existential obstacle that we should overcome. We may just live in a world where there are a lot of beings who have their own kind of moral status. And if that is true, then well, we would all matter in our own way. And none of that would take away from the kind of moral significance that we have and the kinds of responsibilities that we have to each other. But it would mean that there is more significance to go around and there are more responsibilities to go around, and so we can be special, but a lot of other beings get to be special too.

Yeah. No, that is exactly the right way to think about it because ultimately we are one kind of being with one kind of mind. Unfortunately, we are in the position of exercising power over and making judgments about many other kinds of beings with many other kinds of minds without knowing for sure what, if anything, it feels like to be them. And as you note, there is a risk of what we call excessive anthropomorphism, attribution of human traits to beings that might not have those traits. However, that has to be paired with a risk of excessive anthropodenial, denying that humanlike traits exist in non-humans that might in fact have humanlike traits. And so the predicament is that we have to make high-stakes decisions about how to interact with these other types of minds, these other forms of intelligence, with disagreement, with uncertainty about whether it feels like anything to be them and what, if anything, it feels like to be them.

And that means that we have to think about it in terms of the ethics of risk and uncertainty. And so we have to weigh the risks and harms of false positives with the risks and harms of false negatives. And we have to ask, how likely is it that I might be overattributing sentience or moral significance right now? And what would go wrong if I was? And then how likely is it that I might be underattributing sentience and moral significance right now? And what would go wrong if I was? And then we can put it all together and we can find a kind of reasonable, proportionate approach to considering and mitigating both kinds of risk at the same time in these circumstances.

That is a really great question and a great way to frame the question because I think it gets to different types of considerations and in a way the answer is both-and. So on the one hand, we can extend current legal and political frameworks to include potentially sentient, morally significant non-humans, including animals, including AI systems, because we do have not only animal welfare protections that already extend to vertebrates, even some invertebrates, but we also have ways of considering risk and disagreement and uncertainty in legal and political institutions about public health, about the environment. And we have forms of legal and political status like legal personhood and rights, political citizenship and rights that broadly could extend to certain types of non-humans with the relevant capacities and interests.

But then on the other hand, we might think that a more radical kind of transformation is necessary because the reality is — and this is why so many legal and political authorities are so resistant to these extensions, understandably — the reality is that if we really did in good faith extend our current concepts, frameworks, methods of decision-making to all of these non-humans who are potentially sentient, potentially morally significant, then we just would very quickly find ourselves in a world with more than a quintillion individuals alive at any given time who seem to merit their own legal and political status and rights for their own sake. And that would just be an overwhelming number of individual stakeholders to whom we have responsibilities, who deserve their day in court, and so on and so forth.

And so on the one hand, we might find that our current frameworks and institutions can and should, if we want to be consistent, extend to this huge additional population, but that when we do that, it then pushes them past their breaking point when we take into account all of our various limitations and bottlenecks. And that is where the need for creativity is going to arise. We will have to think, what can we do that is new and not yet anticipated that can, to the greatest extent possible, give recognition to these beings while still being achievable and sustainable in our imperfect human institutions? And that is a tough question, but a longer-term question, and we can take time to figure it out.

But how so? What do you mean?

Yeah. Yeah. I appreciate you unpacking that and I completely agree and I think this is where a lot of people get stuck because there are two truths that feel like they must be in tension with each other. One is that we do in fact have responsibilities to a huge number of beings who are vulnerable to the effects of human activity. And the second is we can barely get it together to take care of even a fraction of the human population, 8 billion members of the same species, to say nothing of all of the other vertebrates, to say nothing of all of the invertebrates and now digital minds and so on and so forth.

And when you really sit with the extent of our responsibilities, it can be tough to acknowledge the extent of our limitations. And when you really sit with the extent of our limitations, it can be tough to acknowledge the extent of our responsibilities. And for me, that means that a really important first step just is to recognize and acknowledge that both are really significant. We have really extensive responsibilities and really extensive limitations. And that clarifies the predicament. The predicament is figuring out what can we do, how can we strike a balance given how much we ought to do and how little we are capable of doing right now. And this is where theory and practice collide. This is where the ideal and the real collide. But if you can at least start by acknowledging that much, then you can get together with people and in a realistic, reasonable, responsible way say, okay, what can we do and what should we aim for in the long run that yes might be a little bit more radical and transformative, but then what can we be doing in the short term that might be a little bit more pragmatic and incrementalist, and how can those perspectives work together to help us kind of build momentum in the right direction while working within our limitations? This is at least how I think about that issue.

Yes. Well, there are definitely a lot of social psychological factors that go into what might seem to be a kind of double standard, and some of them have to do with what the individuals are like and some of it has to do with how we relate to them both individually and broadly culturally. So one set of factors is how much does this individual look like us and act like us. So, for example, we have a bias in favor of non-human animals with large bodies, large heads, large eyes, expressive faces, symmetrical features, furry skin instead of scaly skin, four limbs instead of six or eight limbs. All of that biases us in favor of an animal. And this might, for example, be why or part of why we prefer, say, a dog over a tarantula.

But then additionally there are these relational factors. So how do we individually and culturally relate to this individual? Do we classify them as a companion, where their role in our lives is to have a social and emotional bond with us, or do we classify them as a commodity, where their role in our lives is to provide some economic value for us, or to be food, or to be a source of scientific or medical knowledge? And this might be why we favor a dog over, say, a cow or a pig, who are also mammals with big expressive faces and eyes and so on. And so when you put all that together, you can see, okay, there are all kinds of reasons why we might favor dogs and cats: the way they look, the way they act, the role that we assign them in our lives, versus certainly farmed animals, but then even more so farmed aquatic animals or invertebrates.

And in the future, we might find similar divisions with AI systems where if some AI systems are built to look like us, to act like us, to be our companions, then we might have a tendency to anthropomorphize and attribute human features to them and feel a lot of empathy and sympathy for them. Whereas if other AI systems are built with kind of bland designs like boxes without human faces or human voices and then are treated more as commodities where they perform basic tasks for us in the background, then we might be much more inclined to engage in excessive anthropodenial and really not at all experience them as having humanlike cognitive states, even if in some cases they do.

Well, I think it depends on what the AI system is like and how likely that AI system is to in fact be a conscious, agentic, morally significant being. So the philosopher Eric Schwitzgebel and I right now are working on a paper that we hope will be out later this year called the Emotional Alignment Design Policy. And the basic idea behind this paper is that in the future we should design AI systems so that they elicit the kinds of emotional reactions that are appropriate given their actual cognitive capabilities and moral significance.

So if an AI system is more humanlike in their cognition and behavior and if they are a plausible candidate for being at some level conscious, agentic, morally significant, then perhaps they should be designed with something closer to faces and voices that elicit a certain amount of empathy. And if they are not, if they really are a pattern-matching text prediction device and nothing more than that, then perhaps they should not be designed with the kinds of faces and voices that might elicit those reactions. So this is an idea that Eric Schwitzgebel and I are exploring right now. It might vary. It might depend on the context.

So, do you mean how likely are we to in the near future interact with AI systems that in fact are sentient, or how likely are we in the near future to interact with AI systems whom we experience as being sentient? Or do you mean both?

Yeah. Well, I think we already are entering a space where many humans do in fact experience AI systems as sentient, conscious, agentic, morally significant. Obviously, there was a big story in 2022 with a Google engineer who publicly revealed he believed that a Google language model at the time was sentient. Since that time, many humans have started to describe AI systems as sentient, conscious, agentic, have started to experience themselves as having social emotional bonds with them, even have started to make sacrifices for them. So I think that already is happening and we can expect that trend will continue and this is part of why we should start taking this issue seriously now.

But then when the question becomes at what point might they actually be sentient? That of course is a harder question to answer in part because it touches on really difficult issues in both philosophy and science. Like what is the nature of moral significance? What features do you need to have in order to morally matter? Do you need to be sentient and capable of suffering? Is it enough to be conscious in any way, like being able to have sensory perceptions? Is it enough to be agentic in any way, being able to set and pursue your own goals based on cognitive states that function like beliefs and desires? Philosophers disagree about that, about what is needed for moral significance.

And then we also have scientific disagreement about what kinds of entities can have those sorts of capacities. So to be conscious, to be able to have subjective experiences, affective ones like suffering or perceptual ones like the experience of seeing the color red. Do you need to have a brain with the exact materials and structures and functions as the human brain? Or is it enough to have any cognitive system with advanced and integrated capacities for perception and attention and learning and memory and so on? Or is it enough just to have any cognitive system that can process information or represent environments? Scientists disagree about that.

So this is why I was saying that your framing before was so helpful because it fundamentally is a situation where we need to engage in high-stakes decision-making in conditions of disagreement and uncertainty about these really foundational philosophical and scientific issues where we can expect that the disagreement and uncertainty will be ongoing. We are not on the cusp of having some secure theory of moral standing or of consciousness where everyone agrees about that theory.

So if the project is to make thoughtful, cautious decisions about how to interact with AI systems, if we are not sure whether they are sentient, then I think we plausibly will be in that situation within the next five years because we might right now not have a lot of, say, architectural indicators of sentience and moral significance in current large language models and other AI systems. But people who have researched this issue like Patrick Butlin and Rob Long and others have found that there are no obvious technical barriers towards the creation of AI systems with advanced and integrated versions of many of these capacities, many of these features that would indicate sentience and moral significance.

And we know that there are a lot of incentives for companies and governments to race towards the creation of exactly those kinds of AI systems because many of the capabilities associated with sentience are also associated with intelligence. Sentience and intelligence are not the same thing, but they might have many overlapping conditions. And so we might find as we race towards the creation of artificial general intelligence, we might find that at some point we accidentally have created systems that really are at risk of being able to have their own morally significant experiences too, even if that was not the goal.