NEURA Robotics Executive Talk | Nakul Duggal (Qualcomm) on Physical AI

They're making physical AI really reactive, being able to understand the environment, but actually being able to react to different kinds of situations. And I think this is also the necessity of a new level of physical AI. So let me, the way that I've started to think about robotics is when you are trying to do something that has not been done before, you have to break it down into how will you learn something that has not been done before. And I think as AI has made it evident, there is a data loop, there is the accuracy and the quantum of data, how much data do you need to where you can actually train AI to understand what the rewards are, how the reward function works. But with physical AI, there is also the aspect of the sensor itself because in physical AI, you don't have any historical data. You've never really dealt with what it means to touch something, hold something, move something, assess what it is, and how do you deal with it? And that's a part I feel beyond the brain that actually has to do with the nervous system. It has to do with connecting things that typically we haven't ever had to go deal with. So I would love to get your thoughts on it because you've been working in this space for such a long time. That's the core, like it's not just to do things or react on things you see but actually beyond that. So it means when I'm taking this glass or this cup to figure out the shape of it, like how does it look beyond the cup which you don't see, and for that you do need to have a reactive nervous system to be able to really interact and react. For the autonomous and even more, I mean we are right now talking about mainly solving industrial challenges because we are living in a world of having the same labor shortage. And in that case, we saw that vision language action models was just simply not enough to do a real task a human can do, simply because there was some information missing and the other part is it was way like you needed too much data also. So it means we right now lack data. So if you're just using one approach like just the vision, you actually need much more data on video stream than you would need if you also use the other sensors we have. We want to solve that you can actually train a robot a new task in less than 3 hours because if not, it will not be scalable. And for that, I think the biggest requirement was how to gain the right data, at the same time how to train it in much faster way, and the third actually even how do we enable the world to be trainers instead of us doing the training for everyone else. I think in addition, I feel like this is something that I picked up in one of our previous conversations: when you're trying to automate something, the tendency is to think about how would a human do it, but a machine could do something quite differently than a human. So when you think about degrees of freedom...

Exactly. So I feel like this is one of these phases where there is a new level of learning which is very reflexive as opposed to something that is pre-programmed or something that has a set of rules that are predefined that you have to deal with. And of course there have to be constraints because you have to design some kind of an actor where you know this is the maximum that I can get out of what the physical sensor is that I have, but then you also have to be able to think about what is the level of coarseness or precision that is required to complete the task. So if the task just has to be completed but best effort is good enough, what does that look like? Or if it is a very precise machine welding approach that has to be done exactly a certain way, what does that look like? So I feel like there is a very interesting kind of correlation between the automation path that we are on and how are we going to get there, what are going to be the steps. And I would love to get your thoughts because I'm sure you spend a lot of time thinking about what is a core skill and there are lots of core skills versus what are fine skills and what is the order in which you have to solve them and what are the dependencies, what are the interdependencies between the actor and the training and the environment, and perhaps we'll talk about portability of those skills.

So it's a super good point because how do we make physical AI be a global intelligence without actually being forced to have a certain embodiment? So it means, and it's very simple also to explain: if you're working only with vision, then it matters that you know exactly the kinematics of yourself because you do need to calculate everything to it. If you're actually working on the senses of also feeling, then it doesn't matter if my finger is thicker, you're still able to press it the same way and you just basically trained on force feedback or the sensor input instead of the kinematics itself. And this is also the necessity, I think in the future to build a physical AI system which is not depending just on the form factor of a humanoid or just on a certain humanoid form factor, but you can actually think physical AI over every physical device which has somehow sensor input. If we really believe that one company itself can solve all world's issues themselves, then I think the media would be right to say it will take too many years until this gets real. I think the possibility of having a perception language action model is enabling basically the whole world to contribute to make this real in very short time frames. First because you're shortening the training time, second because you're not caring a lot about the embodiment itself and the form factors of it. I really like the concept of Neura where your model is, of course we'll build our models but we don't have a problem if our customers and our partners also build their models because it really is about having access to the data, access to the embodiment, trying to figure out how to go solve these problems, and there is not necessarily just one single way to go solve these. You know, when you came up with the concept of the central nervous system, it makes a lot of sense because you are essentially trying to figure out how does the human body operate? How does it react? But when you think about converting that into machine intelligence, the nervous system intelligence is quite different from the brain intelligence.