Rosalind Picard: Venturing into the Unknown

I do think that the value of thinking differently is one that we need to keep encouraging. It's not just about this diversity terminology everywhere. I mean, there's just a real value to inviting in people who see things differently than you do and really listening to them.

Thank you for having me here. It's a pleasure.

Oh, goodness. There's a lot of crazy adventures I've been on. Maybe the short story is I started as an engineer working to build smarter machines and to build machines that worked like the human brain. I started learning a lot about the human brain and that got me interested in parts of the brain that I really did not want to be associated with. For example, as a woman trying to be taken seriously and being very mathematical in science, I did not want to be associated with emotion. And I stumbled into the role of emotion in perception and attention and engagement and other areas of computational AI that we were working really hard on. So I decided I should try to get some guys interested in working on that because I wasn't about to touch it and ruin my career, I thought. So I went around tried to get them to work on it, nobody would take it. I was oddly at this point I had graduated from Georgia Tech, gone to MIT, done bachelor's, master's, doctorates in electrical engineering and computer science, and I had joined the faculty in a place called MIT Media Lab, which is this amazing place that actually values the crazier the idea, the more they want to hear about it. And in fact, Jerry Weisner, the former president of MIT, and science advisor to Kennedy and others, had given me this one piece of advice as junior faculty. I said, 'You know, Jerry, what's the most important thing I should do?' and he said, 'You should take risks.'

So with his words in my head and with these findings of these parts of the brain that were involved in emotion that I did not want to be associated with, I wrote a book called Affective Computing to get people to take a serious look at the evidence for how affect intersecting with computation and AI and things we could start to measure objectively might improve not only AI and more intelligent machines but how it might improve humans' lives. And that led me on a journey that turned out to actually be quite good for my career, not what I had expected, but I was just convinced by the science that this was important to start working on and getting it out. And then I had to start getting objective data to measure it, so we started building wearables. People like Steve Mann and Thad Starner were very instrumental in the early days in our lab helping us build wearable technologies. And the Media Lab is an amazing place where people with every skill set are willing to collaborate and build things that no individual could build on their own. So we built some of the first wearables to monitor the continuously changing physiology that we now know is quite coupled to what's going on in the brain and to understand those patterns in daily life. And that led to a lot of things I would have never ever expected to find.

Well, I do have to confess, in the last year I found out who my birth mom was. I always grew up as a happy adoptee and I still am a happy adoptee, but I also learned that my 17-year-old birth mom was a skydiver and I used to be a skydiver, so maybe there is a little bit more risk taking in my genes than just comes from talking to Jerry Weisner. But I'll have to say that as a woman in engineering at the time I was getting started, there weren't very many of us, and to be taken seriously meant a lot. We kind of had to, you know, people used to have these signs like you had to work twice as hard to be taken half as seriously. And it meant that I really didn't want to throw it all away with some piece of work that people would say was embarrassing, which did actually happen. I was at a conference, one of the major conferences in computer vision pattern recognition, and I heard a guy whispering about, 'Have you heard what she's doing lately? No, she used to do respectable work.' And I was horrified because that was exactly my fear if I worked on emotion, what people would say. But five years later that same person came up to me and asked me for my data. He had started working on the same kinds of problems, trying to model emotional changes in human beings with objective data, and that's what we had shown you could start to make some real nice progress on. And then that has now connected up to lots of different medical conditions, mental health and physical health, and it's so exciting to see the revolution taking off now.

Well, I do. I use a lot of different technologies. I don't want to be in any one person's walled garden. But I do think that the value of thinking differently is one that we need to keep encouraging. It's not just about this diversity terminology everywhere. I mean, there's just a real value to inviting in people who see things differently than you do and really listening to them.

Yeah, society. Let me tear this apart a little bit too because I imagine your listeners have a lot of different kinds of understanding. I know it's a super smart group of listeners. And yet when I talk about emotion with machines, I break it down into a lot of different kinds of abilities. So for example, the stuff we worked on hardest in the beginning was for a machine to be able to objectively gather data related to human emotion and maybe better interpret it. Okay, machines don't truly understand or know anything, right? And they certainly don't feel anything. But they can take, watch the way things are moving on our face, listen to how things are changing in our voice, and then associate that with labels like, 'Oh, that's a smile, that's an eyebrow raise, that's a pitch go up.' And in a particular context, say where you just showed a joke, maybe the eyebrows raising, the smile, the head bouncing, the laughter sound, maybe those things mean that the joke was funny to that person, right? It doesn't tell you what somebody's actually feeling, nor does it enable the machine to really feel anything because machines don't have a self or emotions in any of those senses. It does, however, give them some symbolic ways to handle data from people. And if you're getting ready to drop $6 million on a Super Bowl ad and you want to know if it's causing people to laugh or not, sometimes what people tell you doesn't actually match what they do. So the ability to say to people like, 'Hey, would you be willing to let this camera watch you while you watch some of our test content?' that actually does give very informative information that the machine can process about human emotion. So that kind of emotion processing, like listening to whether a caller is upset in a call center, taking people with their fully informed consent and instead of them just filling out a survey 1 to 10 how funny was that, looking at their response, that kind of stuff is already in use by more than a third of the Fortune 500 companies right now. And you know, decades ago this was just a crazy idea.

Yeah, like healthcare, lots of ways. So in addition to every healthcare company hopefully caring about their customer experience, there are lots of other unique ways that affective data is important for healthcare. I'm involved a lot with wearable data through both my first startup, Affectiva, later acquired by SmartEye, and now through Empatica, with an 'e' that's the Italian word for empathic. And Empatica focuses on wearables that, it's not so much about selling the device, although their devices have won a lot of design prizes, it's about collecting and understanding the data, really selling data to people who need clinical quality data but in a package that looks good enough that you bought it from a fancy wearable design firm. And the data is telling us things about not just the usual stuff that consumer wearables do like sleep and steps, although we get all that too, but it's telling about continuous autonomic changes, sympathetic nervous system response, parasympathetic, continuous interbeat intervals and electrodermal activity, as well as pulse oximetry, FDA cleared by the way, real commitment to the quality of the data and giving people the raw data so that different researchers can process it the way they want, or we can provide Empatica different kinds of processing. And that allows us to see things like what is the role of different flavors of stress in the healing process or in the pain perception process or in the customer experience process. We're also looking at the way stress interacts with lots of different health conditions.

And this was really cool before the pandemic started. I had this, we've probably all experienced this late at night when you're really tired and you're pushing yourself to stay up late and you're working on something and you're wondering if you should go to hang it up and go to bed early because you're tired and you don't know if you're getting sick or if you're just tired because you've been working hard lately and you should stay up and push through it. And I wish I could look at my watch and see, is this like fighting an immune response or is this just fatigue? Well, we got some early data related to the night before people got sick and it looked like we might actually be able to tell that you were having a response before you got sick. Later this was augmented with gold standard challenge study data, two kinds of influenza, H1N1 and rhinovirus, put up people's noses, not the kind of study I really want to be in. But the US government and Duke and others who worked hard on this got this data using our wearables, and together we were able to show that in fact we could tell the night before your first flu symptoms that you were actually having an early response to getting the flu.

Later, as you can imagine, when SARS-CoV-2 hit, we repurposed the grant for that. We actually had to stop one of our big IRB approved studies with Columbia at that point unfortunately because the pandemic shut down a lot of studies, but we were able to restart things with COVID and build an early COVID warning system. So that now is a medically cleared product in the EU, not yet through FDA in the US, but that's called Aria, A-R-I-A, and it works with our wearable. So again, one of these crazy dreams that comes from late night watching your own data, seeing weird patterns, wondering if there's something real there, and then taking what really turns into years of proper studies to make sure that we're not just getting noise or having too high of a false alarm rate. So that now is sensitive and useful enough to be out there actually helping people make better decisions about going to bed early tonight or changing their plans for tomorrow so that they can get a proper PCR or avoid people who are at high risk.