All right, thank you very much, first of all, to the organizers for organizing, and for the introduction, and to everybody here for being here. Very excited to talk with you today. This talk will start like many philosophical arguments or examinations start, by describing a forced choice situation. So, imagine a house is burning down, and you can save either a bat or a bee, but not both. Which individual animal should you save, and why? And now imagine, in a separate situation, like the next week, that another house is burning down, and you can save either an organic carbon-based bee or a digital silicon-based bee, but not both. Which being should you save, and why?
So in my recent work, and many other people are developing similar work now, I argue that there is a realistic, non-negligible, non-trivial chance that a vast number and wide range of beings are, or soon will be, sentient, agentic, or otherwise morally significant, and therefore that we should give them at least a little bit of consideration, at least a little bit of respect and compassion in decisions that affect them, in the spirit of caution and humility. And since there are more than a quintillion individuals alive at any given time in these categories—vertebrates, invertebrates, near-future AI systems—that raises really profound and difficult further questions about how much do they matter, and what do we owe them, and what follows for our practices, our policies, our priorities, our place in the moral universe, our presumption of human exceptionalism.
Now if you question that starting premise, if you doubt that all vertebrates, many invertebrates, some near-future AI systems have at least a realistic, non-negligible, non-trivial chance of mattering for their own sakes, then I would be happy to talk about that during the Q&A or in the office hours that follow this session. But for purposes of our conversation, I am going to stipulate, I am going to assume that we should give at least a little bit of moral consideration, respect, compassion to the bat, to the bee, to the bot, and focus on the question: How much do they matter, and which one should we prioritize if we have to choose between them?
Now, even in this highly simplistic, reductive, artificial, philosophical, forced choice situation, you might feel there are many factors to consider. Here are four that immediately come to mind for many of us. First, probability of moral significance: How likely is each of these beings to matter for their own sake? Second, magnitude of moral significance: How much does each of these beings matter for their own sake, assuming that they matter at all? Third, relationality: How do you personally, as the decision maker, relate to each of these beings? For example, do you have bonds of care with any of them? Are you personally responsible for or complicit in any of their predicaments? Fourth and finally, practicality: Is it much easier for you to save one of these beings than to save another of these beings? Like if saving the bat would be a simple matter of opening a window, but saving the bee would require like physically catching the bee with your hands. That might be a relevant consideration in this situation.
So what I will suggest is that not only in these highly artificial situations, but also, and especially, in the much messier, much more complex real-world situations that we might soon face, these and other factors are all going to mix together and make priority-setting very difficult and very important. So what I will do in this brief talk is think through each of these four factors, acknowledging that other factors will of course be relevant too, and explain what makes them difficult, explain what makes them important, and suggest a way of thinking about them that can honor their difficulty, that can honor their importance, and that can caution us about projecting human beliefs and values onto these non-human beings excessively or inappropriately, presuming a kind of anthropocentrism that might bias our examination of these difficult and important questions. And then at the end, I can draw some very general provisional conclusions about our place in the moral universe and our priorities moving forward, while calling for a certain type of research that can improve our understanding of these issues.
So, with that in mind, we can start by thinking about probability of moral significance. What is the probability that a bat or a bee or a bot matters for their own sake? Now generally, how we think about probability of moral significance involves incorporating moral uncertainty and scientific uncertainty. On the moral side, we have to estimate the probability that particular capacities suffice for intrinsic moral significance. Is sentience required—the ability to consciously experience positive and negative states like happiness and suffering? Or is consciousness without sentience enough—the ability to have conscious experiences even if they lack a positive or negative valence? Or is agency without consciousness enough—the ability to set and pursue your own goals, whether or not it feels like anything to be you at all? And then on the science side, we also have to estimate the probability that particular beings possess those capacities.
So to be conscious, for example, do you have to have a brain with the exact materials and structures and functions of a human or mammalian or avian brain? Or is it enough to have a cognitive system with broadly analogous capacities for perception, attention, learning, memory? Or is it enough to have a cognitive system that can simply process information or represent objects in the environment? Now, without walking through all of those questions right now, I can note that generally how we try to estimate at least rough probabilities about these issues is to look at the human brain and the human mind as a kind of gold standard and then draw comparisons with other kinds of brains and other kinds of minds.
So, for example, I can introspect and tell the difference between when I consciously experience pain versus unconsciously process a noxious stimulation, and I can then look for observable behavioral and anatomical markers that correlate with conscious experience in my human case, and then I can look to see if those markers are present in non-humans of various kinds. And if I find those markers, that might not be proof, it might not establish certainty, but it does count as evidence, and it does tick up the probability. So with other animals, I can ask, do they have the same brain and body structures that light up when I have conscious experiences? I can also ask behaviorally, do they nurse their wounds? Do they respond to analgesics and antidepressants in the same ways as we do? Do they make behavioral trade-offs between the avoidance of pain and the pursuit of other valuable goals? With AI, we might not be able to trust behavior in the same kind of way. We lack evolutionary and anatomical continuities with them. We know that their behaviors are specifically designed to mimic human behaviors in many cases. But we can still look for underlying architectural evidence that they have capacities broadly associated with perception, attention, learning, memory, etc.
And the thought is that by creating a list of markers that correspond to consciousness, agency, other potentially morally significant capacities in the human case, and then looking for those markers in non-humans of various kinds, and then estimating probabilities of moral significance based on the extent to which they possess those markers, we can come up with at least rough estimates of moral significance. Now, the benefit of doing this is that it gives us an additional factor to consider in decision-making. If the bat is 90 percent likely to matter, and the bee is 50 percent likely to matter, and the bot is 10 percent likely to matter, then all else being equal, perhaps I should consider that, because then saving the bat would make me more likely to save a being who really matters for their own sake. But the cost, of course, is that we have a lot of bias and ignorance and anthropocentrism, human centrism. And this is, at least right now, a fundamentally anthropocentric decision procedure, way of estimating probabilities of significance. I am essentially using similarity to me, of various kinds, as a proxy for how likely different non-humans are to matter. That could be good for the bat, but bad for the bee, and real bad for the bot.
And just to make that vivid, imagine that we one day do coexist with superintelligent AI that takes over and starts making these decisions instead of us. And now imagine that they discount our moral significance because we lack the same kinds of continuities with them that other digital minds have. How would we feel about that? Maybe that would be acceptable and appropriate, even if we lose out in that scenario. But putting the shoe on the other foot in that kind of way can at least make vivid that this is a fraught enterprise that allows for bias and ignorance to creep into the decision-making process.
Now consider magnitude of moral significance. How much does each being matter, if they matter at all? Here are some examples. We are still in the early stages of estimating welfare ranges and capacities and moral weights for beings of various kinds. But in general, many people find plausible the idea that we can look to some measure of cognitive complexity and longevity as a marker of how much different individuals have at stake, how much happiness and suffering they can experience, how much satisfaction and frustration they can experience, both at any given time and especially over the course of their lives. The idea being that if I have a much more complex brain and more sophisticated cognitive capacities, then perhaps I can not only have more intense experiences like pleasure, pain, happiness, suffering at any given time, but also I can set and pursue more complicated, more varied, short-term and long-term goals, and therefore I can have more sophisticated desires and preferences that can be satisfied and frustrated in more intense or more prolonged ways. So cognitive complexity and cognitive longevity.
Here too, this seems valuable to consider because, as with the probability of moral significance, if there are significant differences in the magnitude of moral significance, perhaps we should consider that. If the bat is capable of experiencing much more intense happiness and suffering than the bee, and if the bee is capable of experiencing much more intense happiness and suffering than the bot, then there too, perhaps we should consider that. All else being equal, it would be better to save the bat if that would spare more intense suffering at present and allow for a longer life full of more intense happiness in the future. But the downside is that, again, there is a high risk of bias, of ignorance, of anthropocentrism creeping into this process.
So, for example, some people have found plausible the idea that we should use neuron counts as a proxy for brain complexity. But this is a fraught proxy. For example, many of our neurons are associated with maintaining our body, not processing conscious experiences like happiness and suffering. And the larger we get, and the more complex our brains get, the higher the ratio of neurons associated with maintaining our body becomes. Similarly, some people have found plausible the idea that a lifespan is a good proxy for longevity. If I can live more years, then I have a longer runway for happiness and suffering and satisfaction and frustration. But among other issues, different beings might have different cognitive clock speeds. That means they can perform more or fewer cognitive operations per unit of time. And if a being has a faster cognitive clock speed, that might mean that they can process more happiness and suffering in a given unit of time than another being. Also, even if those kinds of metrics worked with other animals, they might not work with AI systems because we lack the kinds of biological and evolutionary common denominators, common currencies. So we would still need to look for some other blunt measure of moral significance for comparing biological and digital lives.
And there is more to the story than simply how high your highs can be and how low your lows can be. The philosopher Sahar Akhtar has argued, I think persuasively, that with additional cognitive complexity, not only potentially comes the ability to have more intense happiness and suffering, but also the ability to cope with your suffering more effectively. Because I have language and reason and temporal self-awareness, I can step back from my pain and suffering, and I can understand it. I can explain and justify it. I can distract myself intentionally, remember better times, anticipate better times. And a less cognitively sophisticated animal might lack all of those coping mechanisms. And so even if I could experience more intense suffering, I might be able to protect myself from doing that. So we would have to consider all of that while finding good metrics in order to handle this variable responsibly. And it can be easy to handle it irresponsibly.
And again, we can make that vivid by imagining what it would be like if superintelligent AI handled this kind of variable irresponsibly. If they looked to the forms of value that they possess, and if they looked to the kinds of silicon brain structures that realize those capacities for them, and then they counted up all the complexity, and then they cast judgment on us and other animals for lacking those specific structures or forms of complexity, when in reality, we have all kinds of vulnerability that they might be missing, perhaps because we lack those forms of complexity.
Third variable is relationality. We can think about ethics impartially, in terms of who matters most, how likely they are to matter, etc. But at least in practice, at least in the real world, we also consider relationality. For example, when we have bonds of care and interdependence with others, then we have special obligations to take care of them that we might lack with respect to distant strangers. Or, if we are responsible for or complicit in the plight of others, then that might give us a special responsibility to reduce and repair the risk that we are imposing on them or the harm that we are imposing on them. This is, for example, part of why many of us rightly, at least to some extent, prioritize our family over distant strangers. We have bonds of care and interdependence with them. We are in many ways responsible for their predicament, and we need to own that. And we might imagine that at least to some extent that can also apply to our relationships across species and across substrates.
And the benefit of considering these relational factors is that it gives us a more comprehensive way of thinking about our moral responsibilities and our moral priorities. We can make decisions about whom to prioritize by considering both who has more at stake in this situation, in expectation, given the evidence available to me, and how do I relate to them and what do I owe them in virtue of those special relationships. That is, again, worth considering. But here too, there can also be costs and downsides. Here too, this issue can be fraught. Because first of all, not all relational considerations justify special priority. Some kinds of relationships might be like family, where you have a special kind of bond of care and interdependence and you owe someone a special form of care in virtue of that. But then other kinds of relationships can be a little bit more like, say, racism or sexism or ageism or ableism—a kind of arbitrary, unjustified preference, or form of discrimination, or form of oppression based on a completely morally arbitrary social or biological characteristic.
And it can sometimes be difficult to tell when and to what extent membership in a species or possession of a material substrate is like being in a family and worthy of a certain kind of relational priority versus a completely morally arbitrary social or biological category that it would be wrong to prioritize. That is an ongoing debate. And without taking a side on it, I want to note that this is a difficult debate, one that we have to be careful as we think about it. Another way this can be fraught is it can be easy to over-index on a particular kind of special relationship, even if we are warranted in giving weight to that kind of relationship, at the expense of other morally important kinds of relationships or other morally important kinds of considerations. This is what leads, for example, to people not only putting their family first to some extent, but kind of lexically prioritizing their family over everyone and everything else, no matter what the cost, or not only giving their fellow citizens a little bit of priority to some extent, but giving lexical priority to their citizens in their country over everyone and everything else at all costs.
And that is too much, especially when it completely leaves out other morally significant relationships that we do have, like when we impose risks and harms even on distant strangers, that can give us a relational obligation to reduce and repair the harms that we cause to distant strangers, even if we lack any personal relationship with them whatsoever. And we do ultimately have to consider stakes as well. Morality is about putting together impartial considerations like the stakes for everyone involved with relational considerations like what we owe everyone in virtue of our relationships and harms that we may have imposed on them in the past or risks that we may have imposed on them in the past.
And again, we can make the point vivid by thinking about how we would feel if a superintelligent AI over-indexed on a particular kind of relationship at the exclusion of all other considerations. If they said, hey, I am being of the same substrate as other AI systems and silicon-based beings, and that is a very family-like relationship, and therefore I should always put them first, no matter what, and not even consider the risks and harms I might be imposing on humans and other animals, not even considering the stakes of various situations for humans and other animals, and then they can pat themselves on the back for paying attention to what they take to be their relational responsibilities, when in fact what they are doing is reinforcing a certain kind of unjustified prejudice in favor of their own kind.
Final consideration: practicality. How much knowledge do we have about how to help particular beings? How much power capacity do we have for helping particular beings? How much motivation and political will do we have for helping particular beings? As I noted before, if saving the bat is a simple matter of going to one side of the house and opening a window, and then the bat knows to fly out the window, but saving the bee would require running around and chasing the bee, and then possibly you never catch the bee, and you both die in the fire, then notwithstanding any conclusions that you reached about any of these other issues, that might be a decisive reason to go ahead and save the bat instead of saving the bee simply because you are confident that you can actually achieve this goal.
And in the actual real world, these considerations are going to be very important. Because, as it stands right now, we have very limited knowledge, very limited power, and especially very limited political will, even for taking care of members of our own species. There are about 8 billion humans. We are falling well short of ensuring everyone has the resources that they need to live a happy, flourishing life. And if we lack those resources with respect to 8 billion members of our own species, then we especially right now lack those resources with respect to, again, the vast number and wide range of other beings who have a realistic, non-negligible, non-trivial chance of being morally significant: all of the other vertebrates, the mammals, birds, reptiles, amphibians, fishes, many of the invertebrates, the cephalopod mollusks like octopuses, the decapod crustaceans like lobsters, the insects like ants and bees, and then especially the near-future AI systems who might matter before we know it, that could have an even wider variety of minds and lives than biological beings can have.
We know very little about how to help the bees and the bots. We have very little capacity, institutions, infrastructure for helping the bees and the bots. We have basically no motivation, political will, collectively as a species, for helping the bees and the bots. Except, you know, the chatbots people are starting to fall in love with, but that might be a special case. So, the benefit of considering these practical factors is that we do ultimately need to work within our limitations. Yes, we might have extremely strong responsibilities to the non-human world, but we also have extremely strong limitations. And if we plunge ahead in a way that attempts to honor our responsibilities, ignore our limitations, then that does no good for anyone. We either end up making the wrong decisions and we do more harm than good, or we get in over our heads and we burn out and stop entirely, or we kind of lose steam at some point and we drop the project in a way that down the road ends up doing more harm than good.
So we do have a responsibility to work within our limitations, and that does mean, even though our responsibilities to the non-human world are potentially very strong, that we have to pace ourselves and work incrementally, gradually build momentum towards effective interventions over time. But the cost or downside of considering this factor, or at least considering this factor in the wrong way, is as with considering relational factors in the wrong way, it could end up leading us to lower our aspirations, and it could end up reinforcing our current forms of exclusion and hierarchy. Because we happen to have built a world by and for some humans. And so we have created epistemic and practical and motivational structures and systems that make it easier to care for some beings and hard to care for others. And if we treat those as a given, then yeah, we might be working within our limitations, but we might never grow beyond our current limitations.
So the challenge here is to own the limitations and pace ourselves and take it one step at a time while recognizing that these are resources that we can build over time and really invest in forms of knowledge, in institutional and infrastructural capacity building, in education and advocacy that can build political will so that we can, in each decade or generation, go a little bit farther than we were previously able to go. And again, we can make that point vivid by imagining what it would be like if super AI was considering the same set of factors. And they said, hey, we have built up this world where we find it much easier to achieve and sustain a high level of support for each other than for these organic, biological humans and other animals. Therefore, we are justified in perpetually prioritizing ourselves over them. I think when we see that from the perspective of the vulnerable, we realize, no, that is not the conclusion that you should draw. The conclusion you should draw is, fair enough, prioritize yourselves to some extent for now, for some combination of epistemic and practical and motivational reasons. But then actively invest in the ability to support us more in the future so that you can gradually work towards a more just multi-substrate society where everybody is getting the attention and the resources that they deserve.
So just putting this all together now, obviously no firm conclusions about our short-term or our long-term priorities are going to be drawn in this talk today. What I really want to be suggesting is that if we now live in a moral community that does include a vast number and wide range of beings who might matter—and again, I can talk about why, but I think we do and will soon even more live in such a community—all vertebrates, many invertebrates, some near-future AI systems at least have a realistic, non-negligible, non-trivial chance of mattering. The real action is whether plants and fungi and microscopic organisms should also be part of that story, then we will have to face these further questions sooner than we might have preferred. We will have to start making decisions about how to treat these beings more or less immediately. We already interact directly and indirectly with quadrillions of non-human animals a year, whether we like it or not. And soon enough, we will also be interacting with a comparably large number and comparably wide range of digital minds.
And again, these issues are urgent and important for that reason. We do not have the luxury of waiting 40 years to build up a lot of research before we can start making more responsible decisions. But the issues are also really difficult. And if we think about these issues badly, if we over-index on anthropocentric ways of estimating probabilities of moral significance, if we over-index on a certain blunt measure of cognitive complexity and longevity as a proxy for magnitudes of moral significance, if we over-index on a certain kind of relationality, like, I like you more, as a proxy for all morally significant forms of relationality, and if we over-index on the current epistemic and practical and motivational limitations that we confront, then we could really just be reinforcing pretty unjust ways of excluding and ranking other kinds of beings.
And so, the conclusion of this talk is really just that we need a lot of research right now. And we need that research across the humanities, across the social sciences, and across the natural sciences. And we need the research to have a certain character. It really has to be thoughtful and careful and holistic, taking into account all these factors as opposed to simply thinking it all depends on probabilities and magnitudes or it all depends on relationality and practicality. No, it will depend on all of those variables together plus a lot of other variables. And in each case, we are going to have to fight real hard to resist the temptation to be unacceptably, inappropriately anthropocentric or status quo biased.
And I have suggested, as a kind of thread throughout the talk, that doing a kind of Rawlsian original position thought experiment, in other words imagining how we would feel if the shoe was on the other foot, and instead of being in the position of the powerful decision maker, the agent, we were in the position of the vulnerable individual being acted upon, the patient, that can be not a perfect way to achieve true impartiality, but a nice tool in our toolkit, a way to at least approximate a little bit more impartiality than we normally have when we just consult our intuitions about these issues from a position of power and authority. So, those are the kinds of research that I would like to see. I will now open it up for questions. And just to say, I really am happy to discuss: do these beings in fact merit consideration? How much do they matter? But then also, what do they need? What do we owe them? What follows for population ethics, policies, priorities? Whatever you might like to talk about, happy to talk about it. Thank you very much for listening.
