Okay, it's time to check in on a story we've covered periodically on this show, which is the state of the social media backlash and these social media bans that have been going into effect in countries around the world. Yes, Kevin, you may remember that at the end of last year, I made a prediction that by the end of 2026, 16 plus would become the new norm for getting a social media account worldwide. But I think I have one thing that I'm going to feel confident about with high confidence in 2026, and it's this: I think that 16 plus becomes the new norm for social media accounts worldwide.
And as we enter July, Australia, Brazil, Indonesia, Malaysia, France, the United Kingdom, Denmark, and Slovenia have either enacted laws or are preparing measures that would limit children's use of social platforms. So, typically barring kids under 15 or 16 from TikTok, Instagram, Facebook, YouTube, and X. That's a big change.
That is a big change. And I remember you making that prediction, but I also remember that you didn't include Slovenia in your list of countries that would apply one of these bans. So, I'm only going to grant you partial credit on that one.
Here's what I've always said about Slovenia. They could do anything there. It's a very dynamic place, is what I'll say about Slovenia. So that was the prediction. But over the last few weeks, we have gotten several big updates that make me feel like, you know, number one, this prediction is absolutely going to happen. But more importantly, I just think the internet is going to start feeling like a very different place sometime soon.
Okay, make your case. Maybe start close to home. What is happening in the US with social media bans? So on Monday, the US Supreme Court declined to block a Texas law that requires app stores and developers to enforce age verification and make kids get parental consent to download apps. So this is one of the stricter forms of age assurance, as it's called, that you will find out there. Texas passed a law in 2025 called the App Store Accountability Act. And if you live in Texas and you're a kid, you have to link your account to a parent or guardian. And then your parent or guardian has to approve any app before your minor can download it. So the Supreme Court did not take up this case, which basically means that what this is now legal to do. You can, if you are a state, you can pass a law saying that you have to age verify to get onto social media.
Yeah, there was a legal fight over it. Industry groups tried to fight it in court and a judge at one point blocked the law on the grounds that it likely violated the First Amendment. But in June, the Fifth US Circuit Court of Appeals put that judge's order on hold. And so when the Supreme Court says, 'Hey, we're not even going to look at this,' that means that yes, it goes into effect in Texas and it will likely give other states who want to do the same thing carte blanche to do so. And in fact, there are four other states who have passed laws very similar to the one that Texas has passed.
Which one? Name them from memory. Don't look at your laptop.
I would love to name them from memory. They are, of course, Utah, Louisiana, and Alabama. So, I guess it was really three states plus Texas.
Okay, very good. But California, interestingly, has passed a similar law that's less restrictive. It takes effect on January 1st of next year and it requires operating system providers to collect age or birth date information when you're setting up a new device. So, you know, what is interesting about that group of states is obviously now you have red and blue. This is becoming a bipartisan thing across the country. And are all these states sort of defining social media similarly? Like, are they all including the same websites? I know YouTube has sort of been on the cusp for some advocates of these bills, like because obviously there's educational stuff on YouTube, kids are using it in school, but it's also sort of social media, at least the way that some of these attempts to sort of age gate it have portrayed it.
Yeah, for a minute it did look like YouTube was going to be able to wriggle out of these restrictions, but increasingly it is being lumped in with these other apps. And I think it's important to say that that doesn't mean that a kid will be denied from using YouTube because of course you can still open up a website and type in youtube.com and play videos, but it will prevent you from using an account. And advocates would say this does have safety benefits. So for example, nobody can send a message to your kid if they don't have an account. Maybe the recommendations won't be perfectly tuned to, you know, their particular brain rot if, you know, they are logging in every time they watch YouTube.
Right? So those are the state level bills. Is there any movement on a federal ban on social media? Like that was something that you and I had talked about earlier, that the US government actually step in and do something like this nationwide. But are we seeing any movement on that?
Not really. The Congress has been debating a raft of other child safety measures and as usual it seems like it's getting very close to the finish line and then falling apart at the last second. We've seen that several times over the past few weeks. I think what is interesting in the context of will we ever see a federal ban is that Pew published new research last week which found that nearly six in 10 US adults do support banning social media for under 16. So this is now the majority position in the United States and I would guess that if eventually you see let's say like half to three quarters of states implement a ban like this then that's probably when Congress will feel, you know, confident enough to pass it at the federal level. So the first kind of ban like this that we ever talked about was the one that went into effect in Australia, which I think has been the template or the model for a lot of these other bans going on around the world. And my understanding is that we actually now are starting to get some data back about how the Australian social media ban is going. There was a very good story by John Herrman in New York Magazine recently about how hard it's been to ban teens from social media in Australia. But like what are some of the details there? What do we know about how that experiment is playing out?
Yeah, so there was a new study examining the early effects of the ban that came out two weeks ago and basically these researchers at the University of Newcastle in Australia did a 90-day check-in and the big headline was it found that more than 85% of kids reported using social media roughly three months after the ban took effect. And so that led to a lot of people sort of saying, 'Aha, I told you so, these sorts of bans are impossible.' Or they said, 'Aha, look at these terrible social media companies. They're doing nothing to kick kids off.' Australia's government in response is now introducing legislation that would double the fines against social media companies that do not take this more seriously. But I have to say, Kevin, as I dug into it, I became a lot less convinced that this is a failure.
Well, when you look at what Australia's law actually requires, the requirement is that platforms take reasonable steps to try to keep kids off of these platforms. It does not dictate a particular method. And so what we're seeing the platforms do is adopt a range of strategies. One of which is what's called age inference, which is looking for signals that you're underage. So, for example, if you, you know, create a new Snapchat account and everyone else that you message, you know, seems to be like a young teenager, this may give Snapchat a signal that you're underage or...
Yeah. If you're using the word lowkey a lot, that might be a sign that you're a teenager.
No, the people who are saying low-key are now solidly in college. Kevin, you need to update your...
That's the youngest slang I know. What are the teens saying now?
I don't actually know and I'm proud of that. Teens, tell us what you're saying. Yeah. To get past the filters.
But the point of the story is it takes a while to develop these signals, right? And so if this is going to be one of the main things that you're relying on, you're probably not going to have banned 90% of the teens on your platform within those 90 days. It also just seems incredibly easy for teens to fool these age verification systems. There was a detail in John Herrman's story about how people on Reddit are talking about submitting black and white photos of Thomas Edison to these age verification systems to fool them. So it seems like the systems themselves are not perfect. Well, so when you implement one of these bans, you have to decide how much of a hard ass you're going to be. It's possible to be a real hard ass. You know who's a real hardass? China. You know what they require to create an account online? Like an official government document...
I mean, getting close, right? And in these democracies, we're not seeing that yet. They're trying a more gentle approach in part because they don't want it to be an enormous pain in the ass every time that you and I as adults want to do anything online, right? We don't want to have to upload our driver's license to create a YouTube account, right? But that is what they have to do in China. So, so far they've taken this gentle approach and I think that they're hoping that a combination of methods will eventually start to filter these teens out. And I actually think that there's an argument that this is the right thing to do. That sort of swinging the pendulum all the way over to like upload official government ID to do everything would be like terrible overreach and these somewhat leakier methods while less effective in the short term are probably easier to handle. But like if the net result is that all the teens in Australia are still using social media even after they're technically banned from doing it, like why are we doing any of this? Well, we're doing this because we assume that over time teens are going to use social media less. That like by increasing the amount of friction over time, you're just eventually going to make alternatives look more appealing. It's kind of like, you know, once they got rid of, you know, Napster, teens didn't immediately stop downloading music illegally. They were on LimeWire and a bunch of alternative services. But over time, that got harder and there was more malware in LimeWire and it seemed riskier to use. And then one day Spotify shows up and people think, well, I'm just going to buy a subscription. So, I think we're going to sort of see something here that is similar where it's going to become more and more of a pain for these teens to be on these networks and so they're going to start to find alternatives.
Interesting. Yeah. Now, Casey, I want to talk about something you wrote which was about this Candace Odgers character. She has sort of like emerged as the foremost critic of kind of the Jonathan Haidt school of let's ban the phones in schools, let's ban social media for under 16. She and Haidt have kind of been at each other over this issue of whether this is even a good idea or not. And you seem to be taking the side of Haidt and saying, 'Well, I'm not sure about this research or these arguments that Candace Odgers is making.' So maybe like run down her arguments, how they contrast with what you believe is going on.
Yeah. And you know, let me first say I think Candace Odgers is a really thoughtful critic and great researcher and I have enjoyed reading her work on this subject. Why I...
That's a prelude to dragging her ass. Okay, go for it.
Well, I saw her give a TED talk that was posted online recently. The actual talk took place in April and I did take exception to some of the ideas in it. You know, I think what Odgers says in the TED talk is that these bans don't work and that they let tech companies off the hook and so we shouldn't do them. And among my criticisms of that idea is that we just don't have enough data to be able to say that they don't work. You know, her argument in the TED talk is I've talked to teens, they tell me they don't work. My argument against it would be like let's give it more than 90 days. You know, like let... and also it's not really a principled argument to say we shouldn't do bans because bans don't work. If you think that the bans could work, you would just sort of make the ban more effective.
I mean, one of the arguments that the folks like Jonathan Haidt made, which we asked him about when he was on the show most recently, is like not just that these bans would work and that they would get teens off social media, but that they would also improve the mental health or well-being of teens. So, I know it's still early, but like do we have any kind of evidence about whether it is actually improving teen mental health?
No, we don't. But I think it's important to point out that on the whole subject of social media bans, teens on social media, there have sort of been two eras of argument. And the first era was, hm, it seems like there's a mental health crisis involving teens. Let's like try to go out and like study teen mental health at the population level and try to see to what extent we think social media is playing a role in affecting it one way or the other. And typically when people have gone out to do this, they've either found no effect or the effects are very small. And this is the Odgers argument is you're pointing the weapon at the wrong thing. Okay. The innovation that Haidt and his collaborators make is they come along and they start looking at all of the teens that have been groomed and sexed and lured into danger and scammed. And they're realizing that millions of teens are making these reports every single year. Those are direct harms being experienced by an enormous group of people. You don't need a sociologist to go study at the population level effect sizes to be able to confidently say that millions of teenagers are being harmed. And so what Haidt says is if you got the 13 and 14-year-olds off of social media, you would spare millions of them from having these terrible experiences. So that is the thing that the sort of Odgers side of the argument doesn't reckon with at all. And I just think we can because I think it's terrible that that's happening to all those kids.
Is her argument and the argument of the folks in her camp more like there are offsetting good things that come from having teens on social media that outweigh the harms? Or is it that the reported harms are just wrong or they're being misread, that these platforms aren't actually as bad for the teens as we're saying?
Some people argue what you just argued and Candace does not make that in her talk. Where she makes the argument is in saying like you're looking at the wrong thing. Like take a look at adult mental health care. Take a look at the suicide rate of parents like over the past 10 or 15 years. Look at how high that went. If we really want to help kids, what we need is to open up more counseling centers, hire more high school counselors, that sort of thing. Which, by the way, like those sound like pretty great ideas. I would be completely in favor of that. But if you want to bring the discussion back to like what do we do about the fact that so many millions of teenagers seem to be having bad experiences on social media? I just feel like we know at this point the social media companies are not going to do anything to help. And it doesn't seem like legislators anywhere have been able to design effective regulations that improve the experience of those millions of teenagers. And so I just look at that state of affairs and I say, what else do we have left but to actually just get the 13 and 14-year-olds off of Instagram?
Yeah, I find that argument reasonable. I think I want to see more data about some of the mental health effects before I weigh in on whether these bans are working. It seems like on the narrow question of like are they getting teens off social media? The answer is like basically no, not yet. Because the teens are still getting their fix. They're figuring out they're holding up photos of Thomas Edison. They're getting, you know, their parents to, you know, let them on. It does not seem to be having the sort of enforcement action. It doesn't seem to be having the sort of deterrent effect that I think the proponents of the bans hoped. But you're saying it might give it time. And then there's this other to me separate question of like, well, does actually getting them off, if you could actually do that, would it improve their lives? Here's what I know. Very early after the passage of a law requiring seat belts in cars in this country, Kevin, adherence to seat belts was about 15%. Pretty bad. That was not in and of itself an argument that seat belts did not work. It was just an argument that you needed more time, right? So, I think this is going to be similar. I'll give you another example. Think about how easy it is to get around paywalls, particularly at the beginning, right? You're surfing around the internet. Oh, you know, they're asking me for an email address. Well, screw it. I'm just going to open it up in an incognito window.
I would never do that because I respect intellectual property, but...
For journalism, you love paying for journalism as well. So, you know, fast forward to today, paywalls are harder to get around and like, you know, many news organizations have built businesses around subscriptions. This stuff just takes time and sometimes what you do is you introduce a little bit of friction and then you gradually ratchet the friction up and then eventually you sort of arrive at the state of affairs that you're trying to design. So, I just think we're at the dawn of this new era.
Yeah. I continue to have several like worries about this whole pursuit of social media bans. Not that I don't think it's like an experiment worth running, but you know, I've talked before about like my whole sort of feeling that this is kind of fighting the last war because like the real action is in AI right now and like we should be trying to regulate that and social media just feels kind of obsolete. But I actually don't know if social media is even a useful category anymore because it seems to me like the social media apps that I use have sort of two main functions. They are your address book and they are an algorithmic feed for short form video and those things are kind of stapled together. And I don't know that it makes sense to have, you know, a set of laws or set of bans or restrictions that just targets both of those things equally. And it seems to me like a lot of the things that people are identifying as problems with social media are actually problems with like unlimited short form video just on your phone at all times. So, I don't know. Do you think about that as like a reason to be more skeptical of these bans that like the social media apps that are being targeted here are evolving so quickly that maybe the old kind of tactics don't really make sense anymore?
I don't know. Again, if you want to just sort of use the direct harm frame and say, well, when we look at the kids who've been groomed, sexed, scammed, have developed eating disorders, have developed other, you know, sort of mental health challenges. What is the set of apps that they have developed those conditions on? It's a limited number of apps. It's generally not calculators. It's not Microsoft Word. We can actually identify the little squares on the phone that seem to be leading teens into harm. So, I don't think it's unreasonable to just, you know, try to draw a circle around those and say that's where we're going to focus.
Yeah. Yeah. You know, I mean, I think it's worth running the experiment. I just I guess I'm just I'm not heartened by the news coming out of Australia and I want there to be some long-term thing that kicks in as you're suggesting. I mean, one possibility is that it just kind of, you know, this is a temporary sort of growing pain that, you know, these children who have had social media accounts and have had them taken away are sort of especially desperate, but that if you're like eight now and you've never had a social media account and you're growing up in a world where you're not allowed to have a social media account until you're 16, like maybe that changes something for you. You don't have that same sort of lust for the algorithm and the dopamine hit. This is another Jonathan Haidt argument which is that one of the things that a ban does is to solve the collective action problem. Right? There are many teens you can read interviews with them where they say look I would love to not think about my phone at school but every time I look up from my desk all of my classmates are on their phone right so I have this like terrible fear of missing out if I'm not on my phone. So, one of the things that the ban can do is just sort of take that off the table and you know a year from now you're entering middle school in Australia or you know maybe one of these states that's passed a similar law and none of your peers are on social media because you know it is just the sort of world has moved on and it feels a little bit less urgent. Now, don't get me wrong. I'm sure that kids are still going to be using apps. And one of the points that Odgers makes that I think is absolutely true and worth noting, which is that once you sort of draw a circle around the apps that you want to be banned, kids will just sort of migrate to a less regulated space. Like there will always be a website with more lax security that sort of lets you do the thing.
Maybe the teens will be on LinkedIn and that would be a bad thing.
You joke, but there was I remember a great Taylor Lorenz story I believe from a few years ago about how kids would sort of message each other in the comments of Google Docs like during class. So, to that extent, yes, kids are always going to find a way to communicate, but that is not my concern. I think it is great for you to communicate with your classmates in a classroom. What I want to prevent is a bunch of like creepy strangers from contacting your kid or for your kid from developing some sort of mental health challenge by like falling down an infinite rabbit hole of video.
Well, Kevin, let me turn this on you. You know, another thing that we have talked about on the show over the past couple of years are concerns about the way that young people are using AI. They're getting into these sort of very serious relationships with AI companions. AI I think looks like social media in some ways and looks completely different in other ways. So do you have a favorite approach here as regulators think about what kinds of access they want 13, 14-year-olds to have to AI systems? What should they be thinking about?
I mean, one thing that I feel very confident in saying is that I want in addition to whatever these governments are going to do about banning social media, I want there to be much better parental controls for all like child-based forms of internet activity. Like there is no standard approach for AI services and products. It is even like worse. There is effectively no good parental controls on any of the major chatbots where parents can say, 'Oh, I want my kid to be able to like vibe code or do a research project for school, but not to talk about like personal topics with a chatbot.' Like that basically doesn't exist on any of the platforms that I'm aware of. So, I just think like yes, we should focus on, you know, regulators should be focused on what are the right interventions at the policy level, but I worry about the kind of blanket nature of these bans too. You know, I was we were hanging out with some friends yesterday. They brought over their kids. Their 9-year-old was like sitting there during dinner like vibe coding an app. And I was thinking to myself like a that's very cool and I'm very old. But also b like if you ban these apps from being used by under 16s, like the curious kid who wants to sit there vibe coding an app is not going to be able to do that without their parents like permission, right? So we want to find a middle path there.
Now, Kevin, I know a lot of listeners are going to be wondering, what kind of app was a nine-year-old vibe coding?
I'm glad you asked. It was a star chart, like an interactive game. You know how some, you might not know this, but like as a parent, you're always looking for little ways to like bribe and motivate your child to like do the right thing. Okay. And what a lot of families do are like star charts where like, you know, you clean up your room, you get a star, you get 10 stars, you get to like go to the candy store or whatever. So, this nine-year-old had basically turned the concept of a star chart into like an interactive video game where he was, you know, making it so that, you know, if you do something good, you get like 100 points and you can redeem those like you would in like a, you know, in an app for, you know, a trip to the, I don't know, the mini golf place or whatever the sort of treat was. And then the parents could get points, too, and you could deduct points. It was all very cool. And this is a nine-year-old. Like, and I'm thinking to myself, like, if you ban this stuff for nine-year-olds, you're probably saving some of them from horrible experiences, but you're also preventing some of them from having these what I would consider like very enriching experiences.
Absolutely. Let me ask, is this 9-year-old, are they interested in taking investments? Because we would love to take this thing to the next level.
Yeah. Yeah. We would love to do just kind of a seed round in this, be kind of fun.
Don't give VCs any ideas.
Well, Casey, there was something that caught my attention this week, and it was something called JSpace. And at first, I thought, another dating app for Jewish singles.
Now, that's interesting, Kevin, because I've been pronouncing it J-space in the French style. But as it turns out, JSpace is a new discovery from interpretability researchers at Anthropic about what they describe as an internal workspace in Claude that is analogous to the way that humans internally think and process unconsciously. And it was named after a mathematical concept called the Jacobian, which you probably already know, but just in case we have any kindergarteners listening, is a way to describe how a multivariable function changes locally. You can think of it as the multivariable analog of the derivative.
Oh, now it makes sense. I got it. Yeah.
So, this was fascinating. I stared at this paper kind of blankly for a while, just sort of trying to wrap my head around what it meant. There's still a lot I want to understand about this research and what it means, but I thought that this would be a great week to have a discussion about AI consciousness because this is a topic that I think just a few years ago was very fringe, very taboo. You kind of couldn't find many credible people talking about it. The people who were talking about it were largely ostracized or sort of seen as kind of out there. But now this is becoming a real topic that labs including Anthropic are starting to study. They made very clear in their JSpace research publications that they were not saying that this proves that Claude is conscious or sentient or anything else but it is kind of similar in certain ways to some of the research that consciousness researchers have been doing on humans for many years now.
Yeah. And if nothing else, I think that this paper speaks to the increasing sophistication of these models, right? Like we are a long way from pure next token prediction. These models now have these, you know, internal elements that appear to play a huge part in the way that they reason and communicate. And we are just now beginning to understand them and to have discussions about the potential implications of that sophistication continuing to grow.
Yes. So if you want to learn a lot more about the Anthropic JSpace research, you can go read their papers and blog posts about it on their website. But today we're going to have a conversation with someone who's been thinking about the topic of AI consciousness and welfare for longer than almost anyone. Jeff Sebo is an associate professor of environmental studies at NYU. He is also the director of the Center for Mind, Ethics, and Policy and he is one of the foremost researchers in the world on this topic of AI consciousness. He's been writing about it for years and he and a group of colleagues have a new report that just came out last week called Studying AI Welfare Empirically where they dig into some of these thorny hard to nail down issues around AI sentience and consciousness. Whether these models might become conscious someday, whether they might be deserving of some kind of moral consideration and how we would even start to answer those questions using a more scientific approach.
Yeah. So, if you're in the mood for a sort of heady conversation about the big questions of life and the universe, I would say buckle up.
Yes, very heady stuff. And before we get into it with Jeff, we should make our AI disclosures. I work for the New York Times, which is suing OpenAI, Microsoft, and Perplexity. And my fiance works for Anthropic. Jeff Sebo, welcome to Hard Fork.
Thanks so much for having me.
So, I learned about your work first with a previous study that you and some of your colleagues did. This was a paper that came out in 2024 called Taking AI Welfare Seriously. And now you and your colleagues are back with a new report called Studying AI Welfare Empirically. We'll get to the report in a little bit, but just to kind of give some background and context for people who haven't been following these discussions about AI consciousness. When people in your field talk about AI consciousness, what exactly are you talking about?
Yeah, that is a great first question because consciousness is a word that means many things to many people even in science and philosophy. So, people often use it to mean being awake instead of being asleep or being self-aware instead of being not self-aware. And even when you zoom in on the specific concepts that are most important in science and philosophy, there are still different ones. So two that might be especially important for our conversation today are what philosophers call access consciousness and phenomenal consciousness.
Now I of course know what those things are, but define them for Casey.
Sure. Casey, please listen carefully. All right. Access consciousness is a functional concept. It refers to states that can be accessed by the system and used for reporting, for reasoning, for control. Whereas phenomenal consciousness is more about feeling. And so normally when people are talking about welfare and moral status and ethical responsibilities, they have phenomenal consciousness in mind. The question is, does it feel like something to be this system? And in particular, might the system be capable of having feelings and emotions like pleasure, pain, happiness, suffering, satisfaction, frustration that feel good or bad?
And at a high level, I think it's great that this is being studied, but when I've talked about it with some people, some people want to know like why even bother studying that? Like it seems sort of so ridiculous to people on its face that anyone would imagine that an LLM could have consciousness. Tell us a little bit about why this is such an area of interest for you and your colleagues.
Well, one reason is this is very important. Figuring out what entities in the world can have feelings and emotions like pleasure, pain, happiness, suffering, that makes a really big difference for how we ought to treat them, how we ought to interact with them. And it could be really bad to make mistakes in either direction. As many people have pointed out, it can be really bad to overattribute consciousness to non-humans, to see them as being conscious when they are in fact not because that could lead to inappropriate social and emotional bonds with them, misallocating concern to them, even interacting with them in ways that increase risks involving with misuse and loss of control in the case of AI. But then it can also be really bad to underattribute consciousness to non-humans, to treat them as lacking consciousness when in fact they have it. That can lead to abuse and neglect of vulnerable populations as has often been the case with non-human animals. We presumed they lacked consciousness. We scaled up industries like factory farming and then later we realized they do in fact have sophisticated feelings and emotions, but now we are entrenched in these industries and it will take a long time to transition away from them.
Just a few years ago, it was taboo even among the very AI-pilled sort of researchers at the big labs to discuss consciousness at all. Like it made you sound like a crank. There was this guy Blake Lemoine who got, you know, fired from Google after making some claims about their language model becoming sentient. Like when my Sydney story came out in 2023, I got like months of, you know, people emailing me saying, 'You idiot, you're saying this thing is,' and I wasn't even saying it was sentient or conscious. Because I was just saying like I had this crazy experience and it was like you are reading, you know, human traits into this inert language model, you absolute [__]. And now like just a couple years later the major labs are all sort of starting to study these issues of consciousness and there are conferences where people with fancy degrees come and talk about how we can test the models for consciousness and people like you Jeff are, you know, putting out papers with, you know, credentialed co-authors talking about these subjects. Has it been as surprising for you as it has been for me that this issue is now being taken seriously and what do you attribute that to?
Yeah, it was really surprising when we started working on this and publishing frequently on it several years ago. We thought it would take longer for especially AI companies to start taking this issue seriously. We were pleasantly surprised when Anthropic in particular hired Kyle Fish as their first full-time AI welfare researcher and then subsequently the following year in 2025 started a model welfare program, started doing evaluation, started doing interventions, seeking external guidance. Now Google has hired philosophers. OpenAI is at least looking into user perceptions of model consciousness. And those are of course minimum necessary first steps, not nearly enough to actually address the issue in any meaningful way. But even those minimum necessary first steps, I thought they might have taken longer than they did.
I think Kevin and I both experience the progress of large language models over the past year in particular to be really dramatic and I'm wondering if you have felt that as well in this particular discipline. If the advancement in the model capabilities have made these questions feel more urgent to you.
Yeah, they do feel more urgent and for a couple of distinct reasons. One is that with these advances in the technology, there will be higher probabilities of consciousness over time. And this is part of why we want to be assessing models for welfare relevant features now and preparing policy responses now so we can be ready if and when the time comes to start showing models some proportionate level of moral concern. But then the second reason is that as the models become more advanced, more widespread, more integrated into our lives and societies, more utilized as companions and assistance, people are going to start wondering about this whether or not the models are conscious. And as people start wondering about it and as they start disagreeing about it, it would be really helpful to have a multidisciplinary research field that can actually offer evidence and analysis to ground the discussions. Otherwise, people are going to get polarized based on their perceptions, intuitions, assumptions, and that is not going to be helpful for society.
I talked to people at some of the AI labs, including Anthropic, about this sort of research agenda about consciousness and model welfare and whether AIs deserve rights or not or might in the future. And there are some objections from them that I hear pretty consistently. So, I just want to run them by you to sort of get your gut check or temperature check on them. The first is like just because an AI model can talk about consciousness doesn't mean it's conscious, right? You can't really trust a model's own claims about its experience because in some way it is just like repeating what it has seen in the training data or inhabiting a persona that it thinks you want to hear from. And these aren't actually, you know, verifiable claims about anything that's going on underneath the hood. What do you make of that?
I think that is a good point as far as it goes. The question is what does it show? I think what it shows is that we need more not less AI consciousness science and philosophy because that is going to tell us when we should take behavior as a sign of feelings and emotions as opposed to mere pattern matching, text prediction, matrix multiplication. Think about animals for a second. When they behave as though they suffer, why do we attribute suffering to them? Not only because we look at their behavior, but also because we know about their internal anatomies and their evolutionary and developmental histories. And so when they behave as though they suffer and they have no receptors that collect information about noxious stimuli and pathways for carrying information to the brain and systems for integrating information within the brain and they evolved where they faced pressures where if they developed the ability to suffer then they could be more likely to survive and reproduce. And that is part of what helps us understand this behavior is actually a sign of suffering as opposed to something more basic and mechanistic. And this is what we now need to do with AI. Look beyond the surface level behavior and look towards the internal structures and mechanisms, the developmental training history and trajectory so that we can similarly distinguish when we should explain behavior in terms of feelings and emotions if ever versus when we should explain behavior in terms of mere pattern matching, text prediction, matrix multiplication.
Another objection that I sometimes hear is like you can't be conscious if you don't have a body. It is something that requires contact with the physical world and physical experience and emotions that come from hormones and things like that. So what do you make of that objection?
I think that is also a very reasonable plausible point. There are a couple of points to keep in mind though. One is that that is one perspective about a requirement for consciousness and there are other perspectives about requirements for consciousness and we are unlikely to arrive at a secure settled theory of consciousness about...
which we can have anything approaching consensus or certainty in the next one, two, five, maybe even ten years. And so we will fundamentally need to be making decisions about how to treat AI systems without knowing for sure which theory of consciousness is correct and which requirements for consciousness are indeed requirements. And so I would give some weight to the idea that you need a biological body navigating a physical environment. But I would also give some weight to other types of theories. Some of them are called computational functionalist theories that say as long as you can perform the relevant computational functions, then you can have feelings and emotions of a certain sort, whether or not you have a biological body. Now, you might need a biological body to have my kinds of feelings and emotions, to have human or animal-like pleasure, pain, happiness, suffering, satisfaction, frustration, hope, fear. But you might not need it in order to have very different types of feelings and emotions that might be barely comprehensible by humans or other animals.
So what I'm hearing from that is that like an Nvidia H100 could be a body for the purposes of this discussion.
Yeah.
And Jeff, the third objection that I sometimes hear and that I often share myself is like it's just a question of like resources and like what we are worried about as a society. Like right now I am way more worried about the things that AI models could do to humans and human society even if they are nowhere near conscious, right? Like an AI system doesn't have to be conscious to produce a weapon or conduct an autonomous cyber attack. Those things are really scary to me not because of whether the entity performing those things is conscious or not, but because it sucks for humans to be on the other end of that. So like something that I'll often hear from people, and I've heard this from people at Anthropic too, is like, well, you know, the AI model welfare people, like they're off doing their thing, but like the real work, the work that we're going to devote most of our resources to is trying to make sure that these models aren't doing harmful things to humans. Do you have a take on that?
Yeah, I definitely want us to be doing more safety and alignment work and not less. In general, there are a lot of issues that matter all at the same time and we need to be working on all of them at the same time. We really need to keep working on ordinary issues like global health and development and animal welfare. And then within AI, we need to be working on algorithmic bias and economic disruptions as well as more future-oriented risks involving misuse and loss of control, as well as, I would argue, more future-oriented risks involving the possibility that models could eventually develop their own welfare capacities and their own very different types of pleasure and pain. This is not a situation where we should be picking as an entire community or society one issue to focus exclusively on. We should see all of the issues that matter. We should have a division of labor where different people are working on different issues and then we should work together so that we can try to find co-beneficial ways forward. Ways forward that can be good for humans, for animals, and potentially eventually AI systems all at the same time.
So tell us about this new paper that you co-authored studying AI welfare empirically. What message were you guys trying to get across?
So this is a new working paper from the Center for Mind, Ethics, and Policy and IELIOS AI Research, and it follows our 2024 report, Taking AI Welfare Seriously. In that report we argued for taking some of those minimum necessary first steps: acknowledge this is a serious issue, start assessing models for welfare-relevant features, and prepare policies and procedures for treating them with an appropriate level of moral concern. In the time since then, as you noted, a lot of people have started working on this topic, including at companies, and a lot of people have also been making very confident arguments that AI systems either have or lack consciousness based on one type of evidence. People might be looking at behavior alone and saying, 'Wow, that behavior is so impressive. They must be conscious.' Or they might be looking at design alone and saying, 'Well, they were designed for prediction and therefore that must be the only reason why they behave the way they do.' And part of what we argue in this report is that if we truly want to understand how plausible it is that AI systems might be developing welfare-relevant properties like consciousness or sentience, the ability to experience pleasure and pain, agency, the ability to act on desires and preferences, part of how we tell that is by systematically collecting all of the different types of evidence that matter and putting them together: behavioral evidence, how the models behave; internal evidence, how the models work; and developmental evidence, how they came to be. And if we look at all of that together, then we can see what the best explanation of their behavior is and how plausible it might be to attribute something like feelings and emotions to them.
