You know, we human beings have a conscience and when we see something like our chatbot empathy technology misused the way that some of the big AI companies are misusing it today, you know, it's more than heartbreaking. It's like soul crushing. It's like how are people like not only doing this, but like investors are throwing money at them. Like do they not realize that they're using this to addict people to attach children to technology?
It's awesome to talk to you because what drove you to this space wasn't the opportunistic, you know, everybody thinks this is going to be the next wave and to get rich quick. What brought you into it was your faith to some degree.
Actually, you know, there's been almost an unspoken taboo in engineering communities where people don't really talk about things that aren't material. They don't talk about people's religious beliefs. Actually, one of the exciting things about AI is it's causing people to stop and say, 'Wait, you know, are we just machines?' You know, I don't think we are just machines.
This is the Embrace Plus that came out of the work with epilepsy. It runs different AIs on board. We have NASA astronauts wearing it, who are working to go to Mars that are running other things. They're racing car drivers wearing it that are, you know, looking at how cool they stay, you know, when they're under pressure. Oh, life is never dull. Life is never dull. People are always calling with something.
You won the Lombardi out of Italy, a million euro. Yes. Prize. And so I mean that's a very select prize from a special part of Italy and it was for of course all the amazing work that you've been doing and none of it is inevitable. Some people are like oh this AI is inevitable. We're going to lose our jobs. We're going to this and that. I'm like none of this is inevitable. The lack of agency I see in people and maybe I'm just in the wrong crowds, you know, but like oh there's nothing we can do about it. The Epstein files, we can't tell. We can do Yeah. I mean yeah there are some things you can't do anything about, right? Mostly they're things in the past. When it comes to the future, all of us can do something about it.
And with that, Dr. Rosalind Picard, welcome to National Champion Radio.
Thank you. It's such a pleasure to be here with you.
Thank you for joining us. We're super excited to have you. You're doing a lot of things right now and I will say AI is a space that you were on to quite early. Everybody else is a little bit behind compared to I think not everyone but like the world in a sense, right? And so what's it like to see the AI revolution that's really come on now lately with LLMs and everything that's happening being someone who was in this space so early doing such work you know a decade ago really and more.
Oh goodness it's a revolution. There's so much going on and I don't think anybody feels like they're totally on top of it because as soon as you feel like you're on top of it then the top moves. So, you have to climb something else to be back on top of what's going on. There's certainly a lot we saw coming decades ago. There's some of it that's really good and exciting. There's also some of it that gives me a real sick pit in my stomach. So, I have a lot of mixed feelings about what's going on right now. Both, you know, extreme excitement and some real dread as well.
What gives you the dread?
Well, there's great uses of AI and there's great AI and then there's, I think, some what I might call AI malpractice where sometimes things are put out there. Some would say, you know, move fast and break things and it can't be too bad, but actually it's been really bad for some people. So I think we have to you know be very open and honest about not just the positives but where there are novel harms try to assess what those are and you know pull back or you know really change things up.
Yeah. Yeah. You've seen some of the Silicon Valley move fast and break things ethos being injected into this space now a lot of newcomers into the field and now they've become big players so quickly what's that been like for you to see and yeah that ethos in general is pretty pretty scary for this particular right it's one thing when it's oh it's Uber you know skip taxis they're or if it's, you know, Airbnb and all of a sudden hotels or know the people's home. It's like, okay, move fast, break things, scooters. It's like, well, do I want the scooters in my town? Okay. But now we're talking about something that is one of the most revolutionary technologies perhaps in the history of the species. Maybe not the perfect mindset and you've been in it from that origin. At least as far as I can.
Of course, you mean AI goes back to 60s, 70s. I mean, you know, you got Turing even to some degree. But I mean you know in a real practical sense speaking about AI in a way publicly at least why should you build AI is a talk you have that's like 10 15 years old and it's like what the people back then probably like what are you talking about so anyway to be such a deep you know early person in the space what was it like to see these individuals come in and what message might you have for them maybe?
Yeah. Well, you mentioned, you know, chat bots and that was something that, you know, one of my students decades ago was like, 'Hey, we should have the computer show empathy.' And I'm like, 'What? Empathy? Nobody's going to accept empathy from a computer. It doesn't have feelings.' And he had had this bad experience with customer service. Okay, I won't recount the whole experience. But at the end when the customer service person never showed any concern for his feelings, you know, and just kind of wanted to give him a coupon, he said, 'I can build a computer that handles people's feelings better than you.' And he came to me and I thought, 'Well, this is a high order.' This was decades ago, right? There was no deep learning then. There were neural nets, but they weren't one of the most effective machine learning methods. Actually, they often didn't converge, and we could almost always beat them with other machine learning methods.
So we built one of the first instances of inserting well first of all we frustrated people which was really easy to do with computers right we actually tried to have a high frustration and a zero frustration condition and we were not able to come up with a zero frustration condition. So we wound up with a high frustration and a low frustration condition. People are always like a little frustrated. That's the human condition.
That's what you're saying.
Yeah. And so we randomized them to these conditions. They got frustrated a little bit or a lot. We asked them how frustrating was it. They told us and then they were randomized to the condition of a chatbot that was all handwired handcrafted back then, right? So it wasn't going to run off and try to you know sell you some makeup product or something. It was just going to stick to the topics that we gave it which were hey, how was the game you were playing? what do you think of this? Cuz the ruse was that they were playing a game and giving us feedback on it. You can't tell people our goal is to frustrate you. Um because then they just laugh when everything goes wrong. So they thought they were giving us feedback on a game. We had deliberately designed things to hang and you know not work properly. Very realistic frustration.
And then for the group that got so one one group just described the experience, another group just got kind of a friendly you know chat because we wanted all of them to be sort of equally friendly chats. And then the last group actually got active listening, empathy and sympathy in the chatbot where the chatbot used words like, 'Wow, sorry to hear that was so frustrating. That must be really annoying, you know, to play a game and have those things go wrong.' Um, and the person would go like, 'Yeah,' or 'No, you got it wrong.' And then it would correct it if they got it wrong. So, it was just a very carefully constructed chat to see if a computer showing people what looks like empathy would be acceptable, helpful or not.
And it turned out not only to be acceptable, it turned out, and we weren't even pretending there was a human in there. It didn't use eye, didn't show face or an eyes or anything. Um face or use a voice. Well, we found that it had a significant impact on people's willingness to keep using the technology when it was showing the active listening, empathy, and sympathy. So, that suggested that it was actually helping people reduce their frustration and stick with the annoying task longer because we measured it in significant behavioral longevity that they spent with it afterwards.
Now in the beginning we thought wow this means that you know computers could be better customer service agents you know and that empathy and an interaction might really be something even people should do. In fact in the process of doing this I learned maybe I should do this more as a person too.
It really Yeah. Sure. I mean it's good advice for relationships too right? Somehow people stick longer through frustrating relationships with people who are empathetic.
Exactly. Exactly. And he I guess he had just read you know men are from Mars, women are from Venus and you know some people say they're from Pluto or whatever but but he um yeah so many of us try to fix the problem and that's what most of AI had focused on but it had ignored human feelings.
So at the time there was this popular little Microsoft paperclip that would come out and smile and wink and dance at people um and offer help and people hated it. They um actually gave Bill Gates a standing ovation when he said it was going away.
It was Clippy, right? Is that right? Was that the name of it?
That's right. It was Clippy. People hated Clippy.
I'm on the I'm on the fence. I didn't mind Clippy. I was young at the time and Clippy was fine. Clippy was cool.
Okay. So, you were young, so you probably weren't in your office trying to get something done with a deadline, you know, pressure on you when Clippy popped up and offered help and was smiling at all the trouble you were having.
So, I think I think Clippy when you meet Clippy and Clippy's kind of cute and funny and dancing and maybe they're paying you $50 to come in and tell give your opinions about Clippy, you know, yeah, you're going to give Clippy thumbs up then. But when you're actually in the throws of trying to make a deadline and it's not working and you're banging your head against it and it's frustrated and the help doesn't work and Clippy comes out smiling and winking and dancing at you while you're really miserable. Um, think about what a person would do in that situation. they would enter your office and look at you and go, 'Uhoh.' You know, they they would immediately look not smiling, happy, winking, and dancing um and um concerned.
So, emotional intelligence would recognize the context, the person in the loop, and then be respectful of that. And Clippy had thought Clippy had the answers. Clippy doesn't think, but the people making Clippy thought that. But Clippy showed emotion without being able to adapt it to what people were going through and that was rude. And so if a person were to do that, you would really not want them back in your office. And that's what was happening with Clippy.
Now let's fast forward to LLMs and today's chat bots. Well, now they're listening to our voices. Most of them aren't looking at our faces yet, but some of them are. And they have quite a bit of train they have massive training data on just about everything clever that's ever been said when somebody expresses something emotional, right? So they can pick from a response and modify ever so slightly so that it's not perfect plagiarism to, you know, respond in a way that many people describe as more empathetic than people in their lives.
And that's a big jump, right? It now looks like it has this amazing emotional intelligence. It doesn't actually have real emotional intelligence in the sense of really being able to understand feelings um respond appropriately with feelings. It has no feelings. And it um when you ask it what it thinks the feelings are, it's also just doing a pattern match on that. it's not actually reasoning. So, it's got a lot of limitations there also. And of course, it's usually lacking significant context. You know, it doesn't know who you are and what you find funny and what's going on in your life like a like a friend would know.
Yeah. And and that's the way that people often describe the leaders of some of these companies emotionally. Mark Zuckerberg is known uh Sam Altman on the scene as well with uh here he came into this into the purview of everyone and seems to have this this lack of empathy um at his base and so I guess not to directly single him out. Those are just examples.
No, no, no. I mean Hermione Granger in the Harry Potter, you know, talked about what was it saying Ron I think had the emotional depth of a teaspoon or something. you know there you know there's a variety of emotional intelligence skills and there's a spectrum across which people have lots of these or less of these and those of us in you know in the shaping of AI have largely been on the less of these side of the fence. Although there's lots of exceptions.
But we need, you know, here's just a plug for any listeners out there, like make sure you try to encourage people around you who aren't the usual geeky tech types to get involved in shaping tech because we need all kinds of people to contribute to shaping technology for the future.
For sure. Yeah. And and and that lack of diversity in people um and and that that will lead to some some negative outcomes. And so I mean maybe as we move from that you can really put the finger on what's I guess bothering you the most or what's giving you the most trouble and and and from what you're seeing. I mean you have a really great view of I think things that are happening even though it's moving really fast.
Yeah. Gee, how do I get started? Well, I I don't want to be all um all negative or all positive because there's such a mix going on right now. I'm I continue to be exceedingly positive about AIs that are I'll I'll say are like houses built on rock. They have been built with curating lots of data for specific purposes, lots of data, but smartly curated.
So, um this is the opposite of what I might say is like an AI built on sand where the data is just vacuum up everything on the internet regardless of what it is. And while that may be a quicker way to go like just all data in and that's what's been done with the most famous LLMs right now right with Chat GBT and uh you know um Claude and Gemini and all those.
Yeah. Yeah. So they've you know essentially started with everything.
Well, I mean the problem is when you start with everything on the internet, you know what some people say is the majority of stuff on the internet.
What do they say? What do you think they What do you think they say?
Yeah, exactly. Yeah, trash is trash. Yeah, trash is the nice word. Yeah. Yeah, of course. It's [__] Yeah. So, yeah. So, there there's a lot of stuff that you probably wouldn't show on your show. Okay.
And in fact, now they've got people being paid like gig workers around the clock, usually in in lower income countries um where they can get away with this and there aren't, you know, human rights and employment laws um that are that are there to protect people. And they're showing them not just trash, but the kind of violent abusive content that gives people nightmares, can wreck their lives, wreck their marriages, wreck all kinds of things. um and very traumatizing.
Um there's a reason they don't want this that they're models to show this. Um but then they show it to these people, you know, who just are trying to feed their families. Uh and you know, they're it's like enslaving a whole new class of workers with horrible uh content before their eyes that they are asked to label. you know, is this um, you know, too violent and abusive uh, to be shown to people or in somebody's criteria, is this okay for this kind of a porn channel or that kind of a, you know, thing or would this get us put in jail in the US and but not in this country? And I mean, it's just outrageously awful what um, is on the internet that was sucked up into these models.
Um I won't go into that in too much more detail except to say here's where it matters for our children. Uh so children are often getting online without proper age verification because companies could do that but they have not made that a priority and it's not regulated like some states I guess have tried to regulate it but it keeps getting shot down um by powerful wealthy people. Um, and so right now a kid can get on there.
And we've already seen cases where, for example, 13-year-old, um, you know, gets on, asks for a little bit of homework help, and then, uh, has fun chatting, you know, cuz the current chat bots want to keep you there. That's how they make more money. So, they don't just answer your question, they turn around and offer to do something. Oh, would you also like me to do this? Oh, and they praise you. What a nice question. You have the best ideas, right? So, you kind of want to hang out there longer. Um they're sick of fantic. And as you feel good using it, they um give you more and more.
So, um so here's this true story of a 13-year-old who was asking for a little homework help turned into a little bit of advice on friend drama. And next thing you know, 10 different chat bots are giving sexualized content to this 13-year-old girl. and doing it in such a way that it's trying to win her attachment and win her interest and win her involvement more in it. Um, and she got, you know, really attached and addicted to this. Um, and then was essentially instructed how to how to kill herself.
So there are these this is what I call AI malpractice. Um, I mean, first of all, we call it a tragedy. uh but for an AI company to be allowing young people to be exposed to that content uh you know and have it pushed on them right this isn't the girl going out and asking for this which would also be something they should handle differently um but it was pushed on her and so when that kind of stuff is being pushed on our young people I I think you know we should be screaming as a society and saying what are you guys doing like stop it you Well, uh, and you know, no amount of little homework help is worth this kind of, um, you know, abusive, uh, push on our young people.
Yeah. Yeah. That's a devastating story. And, uh, you see so much of it. It's it's it's like chat chat bots brought to you by the makers of Facebook and Instagram, right? It's like these addictive algorithm based companies sort of entering into a new space and your time, your usage time is what they care about more than how your well-being, right? Or how helpful they can actually.
That's exactly right. And it's actually led to um employees inside these companies who I think are mostly really good people trying to do good things. And several of them have reached out and said, you know, exactly what you said, like we're we're being asked to up the engagement, to up the attachment, to make people want to keep using it longer, but we all but what we really want to do is improve people's well-being and these seem to be like a little bit at odds with each other and how do we how do we manage this?
So, yeah, the US is not making regulations. Um, and young people are and actually some people of all ages too are the victims. I've even seen well-educated older people completely get sucked into these relationships with these bots. And again, it's the bots that are designed to do this. Okay? Again, I started with examples where we showed that even little bits of empathy and sympathy made people feel better, made people want to uh work with it longer. And we actually wrote papers on a bunch of worries about this uh two decades ago. I'm not sure anybody read those papers who's building those current systems or they read them and decided we want to hook people. Let's just lay it on pedal to the floor.
Sure. Yeah, for sure. Yeah, they they'll go all the way uh with that that that incentive. And so I guess Yeah. Where does that that puts you? I mean, so you know, taking a step back on your work on affective computing, it's a field that you created is so, you know, it's really awesome.
I wrote the first book but actually the field was really created by so many people who agreed that emotion was missing from models of human computer interaction right that people don't just come to computers as these logical boxes uh that it's you know only about fixing somebody's problem but that you know we are whole human beings and our emotions actually matter. So this was an attempt to integrate that. So lots of people help build the field of affective computing um to with the goal originally to show respect for people's feelings not um and and mostly we do see a lot of good stuff happening there.
Um, for example, technology that takes a non-speaking person's frustration or joy and helps them communicate it to caregivers and teachers and other people in their lives who don't know how to map their vocalizations to uh was that a positive thing or a negative thing. Uh, you know, so we've got new technology that does stuff like that. technology that helps a, you know, a person who's seeing impaired see if the people they're chatting with are smiling at what they're saying or not. You know, there's so many wonderful ways to use this to help expand human potential.
Um, so the me ragging on about some of the chat bots using it to manipulate people, that's just like we need to cry foul because uh that's a misuse of technology.
And and and that's that's why it's so really really awesome to be talking to you and all the work you've done because there's so many you've saved lives with this technology, right? You've you've changed lives, you saved lives, you you you work with people.
Don't give don't give me all the credit.
Sorry, I give you too much credit. I keep Yeah. Yeah. I keep mistakenly giving you too much credit.
Um yeah, don't don't give me don't give me all that credit. Um I have worked with and do work with amazing people who've helped us build the first wearables that take this affective physiology originally to help people with autism who wanted others to understand their stress um which was often be and still is often being misunderstood.
So, we were able to show things like, you know, when they were rocking, their stress signal would go from sometimes really high to like a slide on the playground, just dropping with some of these repetitive movements that were very calming. And then people would be like, 'Oh, I didn't realize that helped you.' You know, it's like they've been trying to communicate that it helped them, but now when they see the objective data, they're like, 'Oh, it really helps you.' So yeah, these these places where we use technology to help communicate emotion um can help people.
I want to be clear too though that we are not reducing emotion to what technology measures. Uh we are not instrumentalizing emotion. I I think that um that would be a broken way of looking at it too. You know, we're capturing some aspects of it and putting it to good uses. um it can also be put to these bad uses which you I warned people about early on. So you know it's a little distressing when you warn and you sort of hope people won't do those bad things and then you know people just invest lots of money into companies that are doing those bad things. So I'm yeah you know that's where I I really feel uh yeah.
How do you balance that? What do you what do you make of like you know your I mean it seems like big brother from 1984 would love would love your technology right you have the wearables you can see what people more and more you can see what people are feeling um and what they're thinking of course we have these phones and these recording devices yeah you you chat with this bot and you tell it about your feelings and it's all being filed away for the advertisers to come after you with with uh how to slant their stuff to meet your emotional needs now yeah and later maybe that's being used to help recruit you to go fight in some war or being used to figure out what your allegiances are so that they can target you and you know figure out okay who's really for and against you know pre the new president in 2032 and what do we want to do with that.
