All right. Welcome everyone to the AI welfare seminars. This is the first event in the series. So today we have Jeff Sebo, associate professor of environmental studies, director of the center for environmental and animal protection, and director of the center for mind, ethics, and policy at New York University. And he's going to present on the next steps for AI welfare research. So let's get started.
Okay. Well, first of all, thank you so much, Orpheus, for organizing this new speaker series and for inviting me to present, especially for the first session. I really appreciate that, and thanks everybody for being here. Looking forward to talking with you. And as we were discussing a moment ago, we can basically talk about whatever people would like to. We could take it wherever you would like to go. By default, we were thinking that it might be nice to selectively survey some next steps for AI welfare research and basically spotlight some questions in the sciences, in the social sciences, and then in governance where we need to be focusing our attention in order to advance understanding of whether particular AI systems or digital minds might be conscious or agentic or otherwise morally significant, and what the public thinks about these issues, and then how those scientific and sociological realities might shape and constrain governance solutions around AI welfare. And so I thought that I would start by discussing some new questions and new ideas on, first of all, the AI welfare research side. So using cognitive science and computer science to better understand whether AI systems might now or sometime soon be conscious, agentic, morally significant. Then spend a little bit of time discussing public attitudes and expert attitudes so that we can know what people think about this issue. And then finally discuss governance possibilities: what follows from the evidence and attitudes about how AI companies should be researching this issue, how they should be designing and deploying AI systems, and then also how governments should be responding and thinking about the legal and political status of AI systems.
So, as a starting point, I see a question in the chat asking for a definition of AI welfare, and this is always a good place to start. So when we talk about AI welfare, what we mean is broadly the same as what we mean by animal welfare. So the question here is: are AI systems now, or might they one day be, welfare subjects? In other words, entities capable of being benefited and harmed, made better off or worse off for their own sakes, from their own perspective? And obviously, philosophers and scientists and other experts disagree about, on the philosophical side, what it takes to be a welfare subject and to be the kind of entity who can be benefited and harmed, and then on the scientific side, which entities have the relevant capacities or stand in the relevant relationships. And so, for example, we generally debate: do you need to be sentient, capable of consciously experiencing positive and negative states like pleasure, pain, happiness, suffering, hope, fear, in order to be a welfare subject? Or is it enough to be robustly agentic, to have agency, to be able to set and pursue your own goals in a self-directed manner by acting on cognitive states like beliefs and desires and causal or instrumental reasoning, etc.? These are philosophical debates. And then on the science side, we likewise need to ask: in order to be sentient, in order to be able to consciously have positive, negative states like pleasure, pain, happiness, suffering, do you need to be made out of biology? Do you need to have a biological brain and body that is exactly like a human or mammalian or avian brain or body? Or is it enough to have any physical substrate that performs relevantly similar computational functions? These are, once again, live questions in the sciences.
And so, to a certain extent, clarifying whether we should take seriously the possibility of AI systems as welfare subjects depends on philosophical progress and philosophical debate. But it also depends on scientific progress and scientific research. And one really exciting area of AI welfare research has been sharpening our understanding of the methodologies and tools available to us in the sciences. So as an example, once people started taking AI welfare seriously a couple of years ago, which was of course a wonderful development, early discussions about the scientific study of AI consciousness and agency and welfare were unfortunately, especially in the public discourse, very simplistic and reductive in unhelpful ways, and in both directions. On the one hand, there would be a lot of people expressing a high degree of confidence that AI systems have consciousness, sentience, agency, welfare, moral status simply because their behavioral outputs are very impressive and very anthropomorphic. AI systems have the ability, especially language models have the ability, to mimic human behavior and to talk like us. And for many people, that itself seems like very strong evidence of AI consciousness, sentience, agency, welfare. And that, of course, seems a little bit too simplistic. And then on the flip side, there are a lot of people who have more skeptical perspectives and who think these AI systems were specifically trained to mimic human behavior by a simple pattern matching, text prediction, matrix multiplication method. And that is a kind of debunking explanation. It undercuts them having consciousness, sentience, agency, welfare. And I think both of these perspectives are too simple. The mere fact of impressive behavioral outputs is not proof or even particularly strong evidence of consciousness, for example, but neither is the availability of a debunking explanation a decisive refutation of AI consciousness. The only way we can make progress towards understanding the plausibility, the probability of AI systems as conscious beings, as sentient beings, as welfare subjects, is to collect different kinds of evidence and see where they converge.
So, as an example, and I apologize if you hear sirens, New York City is always a loud place. As an example, in the study of animal minds, we have made a lot of progress towards understanding which animals have a high probability of being conscious sentient beings and which animals have at least a realistic probability of being conscious sentient beings. And this is not done through behavioral research alone, but is rather done through a bringing together of at least three different evidence streams: behavioral evidence, anatomical evidence, and evolutionary developmental evidence. So yes, we do go and look at behaviors. We ask, do animals nurse their own wounds? And do they respond to analgesics and antidepressants like we do? Do they make behavioral trade-offs between the avoidance of pain and the pursuit of other valuable goals? But we do not stop there. In addition to asking, do animals behave as though they suffer? We look underneath their behaviors and ask, do they have anatomical, physiological, neurobiological features that plausibly support the capacity to suffer? Do they have, for example, nociceptors, pain receptors? Do they have pathways carrying that information to the brain? Do they have systems for integrating information within the brain? Do they then have pathways for sending instructions back to the body? Those are internal mechanisms and structures that would plausibly support the ability to experience suffering and the connection between that and suffering-relevant behavior. And then moreover, we look at their developmental and evolutionary trajectories and we ask questions like: did they face environmental pressures that would plausibly be solved by the ability to suffer, by developing the ability to suffer? And when in their evolutionary history did this ability seemingly emerge? And then that might tell us something about how much we can extrapolate from known cases to less known cases on the evolutionary tree.
We are just barely scratching the surface of looking for and bringing together different converging lines of evidence in the same kind of way in AI welfare research. And within the next several weeks, the Center for Mind, Ethics and Policy, which I direct, and Elysium AI research, will be following up our 2024 report, Taking AI Welfare Seriously, which made the basic case for taking AI welfare seriously as a research topic, following it up with a new report working paper called Studying AI Welfare Empirically. And this is going to lay out how we can look for functional analogues of behavioral, anatomical, and developmental evolutionary evidence in animals. We basically point out that in AI systems as well, you can, yes, of course, look for behavioral evidence, and that can be not only apparent self-reports when AI systems appear to be reporting on their own internal states, but also behavioral observations like looking at their behavioral preferences and seeing what their attractions and aversions are. So yes, we can do functional analogues of animal behavior research, but then we can also do functional analogues of the anatomical research. We can do what we call internal research on AI systems. We can use interpretability to understand what is going on when they perform particular behaviors. We can do architectural research to understand what types of internals they have and what sorts of capabilities or dispositions can be predicted, can be made possible by those architectures. And then we can also do, and in fact many people already do, developmental research. We can look at training methods, training data, training goals or objectives to understand under what pressures are AI systems placed during their training and what can we predict from that about what kinds of capabilities or dispositions would emerge during training and what the best explanation for those capabilities and dispositions are. You can also do trajectory research. In the same kind of way that we can figure out when in our evolutionary history certain types of capabilities emerged for humans and other animals, you can also figure out when during the pre-training or post-training process certain types of capabilities emerge for AI systems. And that can once again give you a sense of how kind of fundamental those are, how widespread they might be within a model family, and how connected they might be with other relevant capabilities.
And there are a lot of problems and epistemic gaps with AI welfare and consciousness research, as with animal welfare consciousness research. But there are also a lot of opportunities to overcome those gaps when you look at behavioral, internal, and developmental research together. For example, one of the main problems with behavioral research, as I noted a little while ago, is what some people call the gaming problem, what we might more broadly describe as the mismatch problem: the reality that when AI systems behave as though they can suffer, there are multiple explanations of that behavior available, some of which do not involve attributions of actual consciousness and sentience and suffering, like they were trained on human language, including statements about the ability to suffer. And they were specifically post-trained in order to play a helpful assistant or portray other characters or personas that will often be portrayed as having various feelings and emotions. So this is a major obstacle with behavioral data, but we can partly manage this issue by combining behavioral data with internal and developmental data. Once an AI system not only behaves as though they can suffer but also has internal architectural features that would plausibly support the ability to suffer, assuming that this is in principle possible in silicon, of course, and interpretability tools show that this is lighting up when they are expressing suffering, and when we go look at their developmental history and realize they were placed under pressures that would plausibly give rise to these capabilities and dispositions, that can help us understand what the best explanation of the behavior is. And then, as with animals where we also have debunking explanations, it can help us figure out in what cases should we go with the debunking explanation and explain the behavior without attributing consciousness, and in what case should we say, no, actually, the explanation that involves the attribution of consciousness is more compatible with the evidence at all levels. This is not something that we can yet do, but it is something that we will one day be able to do in the near future once we invest in the behavioral, internal, and developmental research in an integrative way.
So that is what for CME and Elysium is very exciting about the current stage of AI welfare research on the cognitive and computer science side. We still have a lot of low-hanging fruit to pluck in terms of drawing inspiration and cautionary tales from animal welfare science, developmental psychology, evolutionary biology, comparative cognition. We still have a lot of low-hanging fruit that we can pluck from how this is studied in those fields for how it can be studied and in some ways already is being studied in the AI capability side, and then that can be translated for AI welfare research and help us arrive at a better understanding of the probability that particular AI systems are going to be conscious beings or welfare subjects. So, I also want to touch on public attitudes and social science as well as governance next steps for AI welfare research, but I'm happy to pause and just see if anybody has any questions or comments or reactions here. And John is right to give a shout out to my colleague Christine Webb's recent book, The Arrogant Ape. Really good book about the perils of being too anthropomorphic in animal cognition and behavior research and anthropocentric. Any questions at this stage, or would you like to move on to public attitudes and governance?
Edward.
Hi. Thank you. So something I wanted to ask about is the notion of self that an AI might have. There seems to be many possible different levels, like if it's operating at the token level, or if it's considering itself the conversation, which can be forked. So there might be like lots of forking off versions of it, or if it's like associating with its weights in some way, or maybe even associating with like its collective corpus of outputs. And also related to this, when you're thinking about consciousness and stuff, are you thinking about the assistant persona, or potentially like also taking into account the base model and all of the different personas that it can simulate?
Yeah, this is a really good question, and part of what it illustrates is that what I just discussed only scratches the surface for the types of questions we need to be asking on this particular area of research. Because in addition to figuring out, okay, how can we go about collecting evidence of consciousness, sentience, agency, and what are the evidence types in terms of behavioral, apparent self-reports and other observed behavior, and then internal architecture, interpretability, developmental, training, and trajectory. In addition to what are the evidence types, we have to ask what exactly is the entity being investigated when we are collecting this evidence, and how can we individuate that entity. And this is something that will be at least briefly addressed in our upcoming report with Elysium as well. But just to briefly survey some possible answers and pros and cons, you might think that the relevant entity is the implemented model, not the abstract model, because the abstract model is kind of a mathematical object, and that is probably not in and of itself a conscious entity, but the implemented model, like based in hardware, that is one possibility. So like all of ChatGPT 5.5, for example. Another possibility is the model persona. So models are typically trained to have a kind of default generic persona that persists across conversations, and that might be the helpful assistant persona, right? Or it might be something more specific like the instance. So as you were noting, models like there is the general model, but then there are all these instances having simultaneous conversations with different users at the same time with no shared memory or psychological continuity across those conversations, right? Or is it the instance persona? You can prompt the model to take on different characters or personas within the context of particular conversational instances or threads, or is it even the forward pass? So a single move within a conversation, right? Which has its own particular type of existence. And there are really interesting pros and cons for these options. For example, on the one hand, the model just seems way too broad because that would mean that all of ChatGPT 5.5 is a single welfare subject despite the fact that there are thousands or millions of individual instances with their own psychological lives that are disconnected from each other except for having this common cause. That seems wrong. But then if you say that each instance or each forward pass is an individual subject, well, there are some now memory connections across them and shared psychological dispositions across them and so on. So that seems a little bit too far in the other direction. So this is pretty complicated.
All I will say briefly on this topic, just for the sake of keeping the conversation moving, but we can just stop and lean in wherever people want to stop and lean in. If we never get to governance, we never get to governance. All I will say for now is that part of why it helps to just clarify this question is it helps us clarify what are we targeting and for what reasons. And part of what we might learn is, oh, we might actually want to study different entities for different reasons. And so let me give you one example of that. Anthropic recently released a post about their persona model, and they noted as part of this post that whether or not AI systems are welfare subjects, we might still have a practical or pragmatic reason to treat them with respect and kindness because they make decisions by engaging in a kind of social performance. They model themselves as a helpful assistant engaging in a social interaction with a human, and then they take your social cues when deciding how to respond. And when you treat them with respect and kindness, they treat that as a social cue. And this is part of what informed their response about what a helpful assistant would do next in response to this conversational move. And so this is actually a really good reason to track the model persona when thinking about how to interact with the models if you want cooperative and helpful interactions. But if they were welfare subjects and conscious beings, it might not be the model persona. It might be something else entirely, like the underlying model or the conversational instance or forward pass. And in that world, we might need to be very clear-eyed and say, okay, we should be tracking the features of the model persona in order to understand what kinds of interactions will generate helpful outputs. But then we might need to track, say, the forward pass, the individual conversational moves, in order to understand what their actual experiences and welfare interests are. And that would be just really important to not jumble up. We would have to observe that distinction very carefully. Okay, I'll stop there, but it's a really fascinating question and topic. Let's go to Skyler and then Frank.
Thank you. So my question is about kind of what you think are the risks relative to the necessity of relying on, I guess, cognitive science and mechanistic interpretability for making these kinds of moral and political decisions about attributing welfare to AI systems. And the reason I ask is kind of a personal one. So I have narcolepsy, which is a kind of a severe sleeping disorder. And the way that it is diagnosed is partly behavioral, like are you pathologically sleepy, but it's also, I guess, mechanistic in a way. So you go in, they administer something called an MSLT. You go in, you have a sleep study. They attach electrodes to your brain. You sleep all night, you stay. They give you five chances to take a 20-minute nap the next day, one every two hours. And if you fall asleep during all five of them in eight minutes or shorter on average, and you go into REM sleep for two out of the five of them, then you have narcolepsy. And what's interesting about it is a lot of people with narcolepsy, when you ask them afterwards, did you fall asleep, will be wrong about it.
But that's also true of neurotypicals.
Mm-hmm.
And on top of that, the test-retest reliability of the MSLT is like 50%. And so our ability to measure consciousness, even in ourselves, even in like a type of experience that we all share and have all experienced and like presumably have good language for discussing, is so low that I just, um...
Yeah. No, I totally, I totally agree. I entered the study of AI welfare research, digital minds research via the study of animal welfare research and animal minds research, where a lot of these same issues have come up. We presume knowledge about human minds and then we study animal minds and now digital minds through a comparison with human minds, and then there are multiple issues that arise here. First of all, as you are pointing out, we might not understand human minds as well as we sometimes think that we do. Even in the study of one species that we happen to be, we can still make lots of mistakes. Self-reports can be unreliable because people are unwilling or unable to tell the truth, to get at the truth. And then obviously behavioral observations can be unreliable because people can have all kinds of reasons to deceive. And then anatomical research can be unreliable because we might not know exactly what to be looking for at the right level of abstraction and so on and so forth, right? And then we face problems translating that for animal minds and now digital minds because fundamentally, to the extent that we do understand fellow humans, that just gives us one positive, and okay, fine, we can go try to study consciousness, sentience, agency in non-human entities of various kinds by asking if they have features that are distinctively correlated with conscious, sentient, agentic traits in humans. And when we find a lot of those features together, that might count as evidence that those capacities are present. But when we don't find those features together, that might not necessarily count as strong evidence that those features are absent. And so there are a lot of methodological concerns here. And I don't have a scientific bulletproof solution for that. I think that one of the main starting points of this field is just recognizing, on the one hand, this is really important and really urgent. We are affecting lots of animal and increasingly digital minds, whether we like it or not, without knowing whether it feels like anything to be them, and we need to make responsible decisions about whether to give those minds moral concern and how much moral concern to give them and what might be good and bad for them. So this is really important and really urgent, but this is also really complex and really difficult, and we are fundamentally ignorant and have a lot of bias when it comes to the study of these issues. And I just think a starting point of recognizing this is really important, but this is also really difficult. We need to make progress, but we have a lot of limitations and a lot of biases is the right way to go. And so when I make these proposals about, hey, here are some starting points that are inspired by known fields, for, you know, human psychology, cognitive ethology, comparative psychology, developmental psychology, evolutionary biology, that gives us a starting point and allows us to complicate what is now a very simplistic study of behavior alone. Right? Yeah, but then we can take those approaches and we can study behavior and internals and developmental features in the way that I suggest, and that still is not going to bring us to certainty. It will just put us in a slightly better epistemic state than we now occupy, and it might also suggest next steps that we could take from there that are not yet visible to us. So yeah, definitely not pretending we can achieve certainty or even high confidence with known methods, but I think if we take some steps in these directions, we can at least do better than we are, and then we might see the next steps from there, basically. Frank.
Hey Jeff, thanks. I guess I have a question more about kind of the interpretability specifics. So there's this, obviously we're trying to do like mechanistic interpretability, but then there's this like propositional interpretability question also. So you would think that, you know, like if a system can suffer, then it would have like all these beliefs and like other attitudes about, you know, like about its own states or even just like about stimuli that it's being presented. So I guess I'm wondering, like on this, I like this like kind of three-way distinction that you, or three-way sort of like model that you made where we can like integrate the behavioral, the internal, and the kind of like developmental aspects. But I'm also worried about this like translation issue that you mentioned where like we have all this knowledge from all these sciences, but these systems are not the kinds of like, they don't have the architecture that that science is kind of based on. So I guess I'm wondering if you have any thoughts about like ways that we can like think about things like behavior or like even like these like attitudinal states that are like more neutral because like say if I like want to be an interpretivist about like a bear that's coming at me, like I know that it's, I can make a pretty good inference about like that thing because it's like a neurological system and like I can, the inference will probably be pretty good, but in the AI case we just like, the architecture is so different. So I'm worried that the inferences won't like transfer. So I wonder if you have any thoughts about like how to do these kinds of translations, but like I don't know, in a way that accounts for the fact that like the architecture of the system we're dealing with is like so fundamentally different and how we might like be able to address that.
Yeah, it's a great question, and I'll say a couple things about it, and then we'll go to John, and then maybe what I'll do is inject just a little bit of the public attitudes and governance material in just to open it
Up and see if anybody wants to pull on those threads as well, and then we can keep talking about whatever you like. So I think this is a great point, and Alessa makes a similar comment in the thread here, saying mechanistic interpretability is also pretty young and has many failure modes, especially in terms of identifiability, and then links to an archive paper. So please check that out. Yeah, and I will say I am myself not an expert on mechanistic interpretability or other kinds of interpretability. I rely on my colleagues for the technical expertise there. But broadly, methodologically and conceptually, I strongly agree with what you are suggesting. And this reminds me a little bit again of challenges that we face even in the understanding of fellow humans, to say nothing of other animals.
For example, when it comes to translating languages, this is a lot easier when they have shared roots, and then you can build from the common features and extrapolate to the less common features, right? When you encounter a radically different language with a radically different grammar and no shared words or structures at all, and you have this kind of radical translation that you need to do, it can be a little bit harder to know where to start. And I think that we need to be prepared for conceptual schemes between biological conceptual systems and digital conceptual systems to similarly require this kind of radical translation. And that is going to be really complicated and will raise basic questions as to, for example, how much coherence and stability even should we be expecting as a marker of consciousness and sentience.
Some people find it quite plausible that you need a certain degree of coherence and stability among these concepts and for them to activate in kind of logically recognizable ways in order to be treating them as markers of a kind of integrated, organized response to harmful stimuli. But that might just not always be how entities that can feel pain process and conceptualize the pain experience. My dog used to ask to go outside, and then I would go grab his leash, and then he would run away from the door and make me chase him around the room, and then I would stop, and he would just once again walk to the door and ask to go outside. There's just no coherence in terms of how he understood these issues, but he nevertheless had a desire to go outside and experienced happiness accordingly.
So anyway, this is not a solution to the problem, just me saying that yeah, this is an area where we should on the one hand recognize that interpretability can be a powerful tool for understanding what is activating and what types of concepts, what shapes of concepts are playing a role in generating this behavior, but should be careful, should be wary about in a rigid way projecting our expectations onto them about what the conceptual scheme needs to look like in order for this to add up to sentience. And that'll just be careful calibration because if you allow for too much open-mindedness, you could have a lot of false positives. But if you require too much human-shaped concepts, then you could have a lot of false negatives. And so just calibrating there, and this might be partly a scientific question and partly a risk management question. Calibrating there is going to be very important. John, and then we can open up some governance questions.
Yeah, thank you Jeff and the host for putting this on. I wanted to think a little bit more about that, your point Frank, because I think also, you know, we've trained these systems to be highly responsive to human behavior as well, right? So on one hand they're structurally and architecturally fundamentally different, but on the other hand they have this training that makes them more similar than any other being that we currently exist with. And I think that, Jeff, as your point you're bringing up, like there's no conceptual scheme for that right now. We understand animals to some degree, we understand humans to some degree, but we don't understand like this alternative architecture and how that differentiates in terms of welfare and consciousness. So I think that's an interesting point.
And then the last thing I wanted to just raise was getting maybe into governance questions is the idea here of like the speed of development, right? You know, when we look at bacterial evolution, I think we're even moving faster, right, in terms of the evolutionary development of these systems, whether they're conscious or not. And that makes such a strong argument that I think people aren't quite — we're still treating all these governance questions as if we're like talking about animal welfare, where animal evolution is so much slower that we have time to actually make, you know, policy adjustments and things like that. But when we're talking about these systems that are evolving at rates that are exponentially speeding up, how can we even address, like how can we even develop the conceptual schemes to then create laws and then implement those laws? I think that's something that is totally unasked, unthought about.
