Developing Intelligent Machines for a Human-Centric Future w/ Toyota Research Institute's Gill Pratt

Yeah, well that's absolutely right, Ed. And first of all, I want to thank you for those wonderfully kind words. But when Toyota began, things in Japan were really tough and there was a tremendous shortage of labor, really hard economic times, and so there was this great need to find a way for people to be more productive. And they invented an idea called jidoka, which means automation with a human touch. And it was actually when Toyota was not an automobile manufacturer but a manufacturer of textile machinery, so looms. In the past, there would be a human minder for every piece of textile machinery. And when a thread broke, that person would stop the machine, fix the thread, and things would continue. The innovation, it doesn't seem like a big deal today, but it was back then, was let's have the machine watch for all the threads breaking. When something goes wrong, it raises a flag. And that way one human minder could mind many machines. So this was very primitive automation but the goal was for human well-being, to make people more productive. And this started a tradition in the company which is still with us now, which is the purpose of automation is not to replace human labor, it's actually to amplify human labor. And that's been our philosophy ever since.

It's a very interesting thing. It really is about keeping that perspective. Everyone loves machinery. Any technologist loves bringing things to life and seeing things that were in our minds suddenly start to operate. And I think we have to keep in mind what the purpose of the autonomy actually is, and it's to improve the human condition. And not just for the person who's asking for a ride. I think the most important thing in this field is to remember that human beings are the goal of the technology and it's not the other way around. And whether it's AI that's helping us to write or to create, or whether it's autonomy that's helping us to be safe when we drive or autonomy that's helping us not have to work so hard when we drive, the goal is always about human beings.

Right. So again in this mindset that we're building technology to amplify people and not to replace them. One of the truths about human beings is that we make mistakes all of the time and can the technology be used to help us to improve. So let's take driving as an example. Most people drive pretty well. And it's actually a myth that we need autonomy in order to help with terrible human driving. Human driving is incredibly good. It's 100 million miles between fatalities for human driving. So that's an incredibly good number. That's in the United States. And of course, a lot of it is because the technology helps in terms of passive safety and some active safety now, too. But let's remember, people are really very, very good. How can we help them be even better? And that's the idea. So Kaizen means continuous improvement and we do that both inside the company and we're looking for ways to do it with our customers also.

So what was so wonderful and this was nine years ago that this really all started is that we were just at the peak of the hype cycle for autonomy. And you guys all know about the hype cycle, right? It goes way over to what's called the peak of inflated expectations and then there's this my favorite phrase the trough of disillusionment which happens after and then it comes back up and then there's this plateau and actually things keep getting better and better and better. And it's actually that curve, the hype cycle curve, is an example of something called Amara's law. And what Amara's law says is we tend to overestimate the rate of technological change in the short term but underestimate it in the long term. And we have to remember that this is just an innate part of human perception. We always tend to do this thing. We see some technological opportunity and we think this is it. Everything's going to change overnight. But then it doesn't. But that also means that we shouldn't be pessimistic and say things are never going to change because in the long term actually we underestimate the impact that things are going to have. And this is true in autonomy especially. And when we came into the field 9 years ago this top of inflated expectations was happening and people were saying it's just around the corner. Well it wasn't because there were a whole lot of things that had to be developed and the most important part of that was actually how to predict what other actors are going to do, what a pedestrian is going to do, what a bicyclist is going to do or what another driver of another car is going to do. That's something that human beings are very good at. We have empathy. We can see other people and we can predict what they're going to do remarkably well. And it's taken a long time for automation to learn that art of, oh, I think this driver is going to go this way. I think this pedestrian is going to do that. And I think though now we're actually starting to see this happening. And in this age of generative AI and LLMs, we're now much more near this plateau that happens after the hype cycle where things are actually continuing to get better.

Sure. I want to make very clear that I believe really strongly in startup culture also and Toyota itself we have an early stage startup fund and I'm not trying to give an ad for it here, but we believe strongly in startup culture. We have it internally as well. We have many many teams that in the best way I expect that a large number of their efforts are going to fail. But if they're trying for something that's extraordinarily difficult, it means that some of them will succeed. And if you look at the kind of work that we've done in automated drifting as an example or in the robotics field with what's called diffusion policy and generative AI, it's incredible how some of those successes happen. So, I think what's fundamentally wrong is this image that car companies are dinosaurs. That's just not true. Inside car companies, in every single one that I know, there are some incredibly innovative groups. And we have some of them, too. And what's remarkably effective is making sure that that innovation machinery is constantly running and that the best of what comes out of there is actually given an opportunity to make a difference. And that's when after that hype cycle is done, that's when you see the results of that investment. So, you mentioned an example. I think we have a little video that maybe we can show really quick of this autonomous drifting, which again, I don't think is necessarily the first thing that people think of as some of the work you're doing. I want you to explain it, but let's take a look at it first.

So that's lots and lots of fun, isn't it? So first of all, the person who's the leader of the group doing this is Avan Balachandran. He's here today so you can speak to him after. But why are we doing this? Well, of course it's a heck of a lot of fun. But the goal is actually to show that we can make cars safer by allowing them to do maneuvers that normally takes incredible human talent in order to do. So imagine that you are in a situation and either a pedestrian suddenly jumps into traffic or you're on ice and you're trying to avoid a crash because you have a low coefficient of friction with the ground. This technology can directly go into that in an active safety system to avoid having a crash because it can actually control the car's motion even if the wheels are skidding. And so that's the primary reason for developing all this. There's a secondary reason also which is that we don't believe that safety and fun are opposite each other. We think actually there are two different dimensions and that you can have both and we imagine a future where people essentially if they want to learn how to drift, if they want to learn how to have more fun with their car than just driving to work, can have the car teach them. So imagine that the car actually knows how to drift better than you do as the driver and you take it to the racetrack and you learn how to do high performance driving with the car as your mentor. So it can be both a safety guardian and also a teacher at the same time. So the future is just incredibly bright. And when we think about autonomy and we think about real human needs and human desires, I think there's incredible matches that can go way beyond if we just take a technologist view and say autonomy for its own sake.

Well, I know from the introduction that you're going to be hearing a whole lot about LLMs and AI and things like this. We have in addition to the work that we do on cars, we do a lot of work on robotics and it's not only for robotics in the car factory, but for future products. So a robot that would be in the home for instance for aging society. And our goal with this again is the same as our goal is with cars. Cars are tremendous amplifiers of human ability. When you are in a car, you have many more horsepower than the fraction of a horsepower that your muscles have. You have the ability to turn. And the reason driving is so much fun, excuse me, is that it amplifies you. Well, imagine robots now. Robots not only in the factory, but in the home, on the farm, all kinds of places where they can work. How can we amplify the human experience with robotics? The goal is not to sit on the couch and watch television and have the robot do all of our work. The goal is to have the robot help us to do our work so that we can be even more productive than we are before. So we're doing tremendous work on that. And the innovation that's happening now which is just so exciting is machine learning. And so the robots instead of having to be programmed the way that we used to have to program autonomy and cars as well are now given examples. And it's remarkable how few examples we need in order for the robots to learn the kind of behavior that human beings do and then to replicate that behavior in the kind of work that they are doing. And so this whole idea of sometimes it's called behavior cloning, all kinds of different names are coming out for it. But it's this remarkable new innovation. And I think that the future for this is incredibly bright. The key is we have to remember the purpose of it all. And that's why I keep coming back to this. We have this idea in the company and it sounds very naive but it actually comes from the very top from the founding family that the purpose of our business is happiness for all. And again in the west it's sort of hard to understand this but it's really about human well-being and improving quality of life. And in all of the work we do in the automotive industry and beyond if we stay focused on that just think about how much better life could be.

Sure. So, we showed a video at CES that many of you, I think, saw of a robot doing t-shirt folding, and we were actually the first ones to do things like that. And the way that we taught the robot to do that was by writing zero lines of code. Instead, the robot watched a person doing it several times, each in a different way, and then it learned what the behavior was. It didn't just replicate the trajectory. It learned how the person responded to motion of the different things in order to do and it learned how the person recovered from errors and then it cloned that behavior. And so lots of people are doing this now. We think it applies to driving as well. But what is remarkable is just how impactful that is. Now I want to take you to another place where people typically have thought that the work is somewhat dull, somewhat dangerous, that sort of thing. And that's factory work. We actually have a tremendous shortage of factory workers not only in the US but throughout the world and many people love doing that kind of work but if it's repetitive and it requires a lot of stress and stuff like that it can actually be very hard on human beings both for their spirit and also for their body. Well now imagine changing the nature of factory work of saying that human beings in factories should actually be teachers and factories actually should be much more flexible than they are right now. We don't have to mass produce everything that looks the same. Imagine a factory that could make a wide variety of different things. And human beings are teaching the AI systems how to do it. And then the AI systems are doing all of the dull parts of it. That's what the goal is. Having this technology amplify people rather than replace them. And amazingly, it harkens back all the way to those textile machines and jidoka. This idea that the purpose of the automation is actually to help human beings have a better life. And so it's really the same philosophy.

That is absolutely true. And you know there's this old saying since I used to be a professor, you don't understand something until you teach it. Well imagine if everybody had the opportunity to do that kind of teaching not only to other people but to the very machines that help them to live better lives. And what it's like if I teach my machine to do something we have this idea called fleet learning where that machine then can teach every machine on Earth a little bit of that wisdom that each one of us gave. And it's kind of like contributing to Wikipedia where you see that the efforts that you make no matter who you are can now have an impact on everybody else's knowledge and understanding. It's a very beautiful, very utopian view of the future with AI and in that way I'm very much a techno optimist but we have to remember that it's not technology for its own sake. It's technology to help people to lead better lives, more meaningful lives, and safer lives. And I think that we can absolutely do it.

Well, thank you very much.