Nakul Duggal12:40
You know, as you describe that, if I try to think about it in terms of what would it mean as an architecture? I have to think that it has to be like an AI control system because an AI brain we understand because that has more to do with reasoning. It has more to do with I saw something. I understand what it is. I go back into my memory bank. I now know what it is. Therefore, I need to figure out what to do next. But this whole concept of how do I grip? How do I move? How do I feel and then make a decision? You can't equate it to a brain because there isn't anything that you are reasoning about. You are actually delegating it to a control system. But the models for this have traditionally been very linear. They've always been rules based. They've always been, you know, I think like you it's a very good example. Hey, I hear the click and I know that the joint is in the position it needs to be. Now I can actually take action. I have a rule that I have programmed and therefore whenever those things are heard I can now take the action. If you start to move that away to something that is more generalized where rather than actually making it rules based you're basically saying here is the actor that I have, what are all of the things that I can do with this actor? What are your thoughts on the coarseness or the precision that is needed? Because in my mind for example, if you want to do something in a very precise way, the question is do I have to look? So do my eyes have to get involved as I'm trying to go do something versus if I'm doing something coarse, I'm pouring water to you. I can look at you and I can pour water because I know physically where things are. What is your thought process around what is the sophistication of the actor relative to the coarseness or the precision of the action that you have to take? It's mostly not the precision, it's basically the reactiveness and the reflex. Exactly. Like when you're even like pouring water inside another, why it works because you even hear and you feel like even in this moment how you do it and this is also how you react on it. Because if you wouldn't have the feel, you would maybe just over pour, it will just everything pour, you would even not be able to pour it in a very careful way because you might not see what's inside this carafe or whatever. So it is always the speed of the reflex and this is also why we are so much into this smart limb concept, simply because this is something you can train perfectly and while giving the input from the brain towards the smart limb which is right now responsible for the reflexes basically by doing a task, you can actually have the speed managed in a way which you need to react on this real physical world. And in the same time, the interesting thing is that you don't have as a user or somebody who really takes advantage of this concept, you don't have to think about it. And that's actually the most important topic. I think it's not getting more complicated now because simply what you do is by gaining the data out of data suits or out of the robot or even out of simulation, you're also having the same concept again that you're gaining there only the data which are required to do this fast reaction. So you only care about this little nuances basically of the nervous system to be able to fulfill the task properly.
When you introduce the concept of the kinematic model for the specific embodiment. So if there is a physical model that I know the embodiment is designed around, that provides a lot of rules in terms of okay I know what it is, the constraints of the hardware are based upon this, I can have an implementation that allows me to operate with a certain degrees of freedom. So depending upon what the task might be, I can use different types of modalities, I can have a different type of capability set, I can keep evolving. That concept to me was very interesting because I've heard about that concept in the context of locomotion. You know, we've seen videos where if you have a robot dog and you chop off a leg, it will still keep walking because it is essentially designed to reconfigure its kinematic model with one less degree of freedom. Now it knows what to do. But if you start to apply that to the next set, right, where you say, 'Hey, I can pick this cup with all five fingers. I can also pick it with two. One is probably not possible. I can pick a different shape.' That gives you those degrees of freedom. And I feel like that is probably something that is really interesting to dive deep into as far as what will PLA models be designed for, because you can add a lot of complexity. You can also make them very simple such that there are tasks that they must be able to do that are highly generalized and then you build capability on top.
How does that interplay with the types of problems you want to solve? Because there are so many different problems, so many different environments that require automation. How have you thought about how to segment? The smart limb concept is like human. You have a nervous system, you have a brain. But I think it's even going beyond human because the good thing is, and it's also where Qualcomm partnerships is very important, is interconnectivity actually with all the other surrounding information. So it means that in this world, being a robot, you don't have to go to your fridge and open it to see what's inside. You can actually interconnect with the eyes of the fridge if there is a camera inside. The same also with autonomous driving vehicles. You can interconnect with any other sensor around to actually give information and react on them which you're coming close to. And I think this is actually the next big step for us, is like going beyond human capabilities and saying, okay, you actually have also another angle to work on. So you have two robots interacting with each other. You could actually use robot number one can use the eyes of robot number two if this is a better angle to see beyond, to be able to react and solve the task. And this is also the next big step we have to do to bring artificial intelligent physical AI actually into the broader space, and in the same time also to bring them into industrial space, because it's all about interconnecting, it's all about talking to devices without maybe pressing a button or without just sending commands and having a response basically on that.
It's very interesting you say that because when you were talking about Neura AI and the ability to see what another robot sees or another, you know, say an appliance in the home, what is the oven scene, etc. We are working with a team inside Qualcomm and it's a camera AI team, and the technology that they've built is the ability to download a model into the camera and you can design the model to do whatever you wish it to do. And then it will have a bunch of triggers that allow you to then ask the camera to stream if you want more information. And then you can define what is the level of streaming capability you need: low definition, high definition, you want to zoom in on something. So you have this, it's almost like I call it a close caption stream, and I have early information on what the camera is seeing. And if I'm interested, I can start to say, I would like to know more. Please start to send me live data because I don't want to trust your model. I want to just see the stream and send it to a more capable model where I will process it myself. And when I was listening to you talk about robots connected into an intelligent environment, it is very similar because it's like I want to understand what is the physical state of something that I have to ultimately plan to engage with, and that creates this network of robots and the environment that the robots are in that is essentially like a vision language network because you are converting it to a language that the robot can understand.
Ultimately, it will potentially even go one step further. So, I feel like the deeper you go into automation, the more of these capabilities you start to think about in terms of how and the other piece that is fascinating to me is that almost all of this has to be deployed at the edge. Yes. Because this isn't something that you're going to go to the cloud for. The latencies don't permit it. So, you will either have something that is an on-prem deployment. In many cases, almost all of the inference will happen at the edge. Maybe there is a collaborative network that works that robots and local infrastructure works with each other. So to me it's really interesting space that we are starting to get into.
I think this is also the core thing is why we talk always about three level architecture. So the first is like nervous system, second is the brain and third is the extended brain. And the way of how we do also the extended brain is not just all the time interconnecting basically with the arch device and be extended by that but more because the good thing is about if you're actually able to train the network the way to make more let's say environmental and situational depending. Then you can also have the external brain mainly giving you information by the situation you're in. Like you're coming into a kitchen you don't expect a CNC machine so you're expecting to like a fridge and microwave and whatever and dishwasher, and if there is a CNC machine inside you're allowed to think a minute about it why and download the information you need, but normally even it gets faster than a minute today. So this is exactly how we see the future. So that's also why the neurogyms we building right now, there's training centers for the brain, physically AI brain basically, we mostly get it into context. So you mostly have even the environment look like a kitchen or like an industrial environment because I think this is what the robot or the artificial intelligence mainly cares for, to give you the right information in the right time and as fast as possible so you can actually use the whole capabilities you have.
Yeah. What I find really interesting about your concept of the neura gyms and the neura suits is you are trying to replicate an environment in industrial cell something that you want to automate and then extract as much of the data as possible so that you can start to think about how to build not just newer models but even newer sensors. Yes. Because you have to be able to kind of get as close as possible towards the most efficient way do this. In the training centers you're figuring also out that's I think also the advantage like you're training you don't need much data so less than two hours maybe and then you're training and the good thing in the gym is that you can right away deploy it and check how good it is, does it work, does it not work, what's the limitation to make it work. And this is also right now the main task Neura is focusing on: building the pipeline the way that you don't need too much data, that you basically can train without being an expert, that is making huge progress by also being efficient with data. This is the core like how do we build a pipeline to be as efficient that you don't need tons of data for a small task. I want to be able to as a developer get access to a lot of data to keep improving my model and I want to keep validating how close am I getting to where I would say the model is not working good enough. So it's a full loop. Synthetic data, real data, stream it, store it, process it, train it, deploy it, look for how far, how close it is to where you want to be, and repeat the process.
And I feel like we have a lot of different capabilities now in terms of being very surgical about what problems to go solve. So this specific idea of building a perception language action model where you have specific modalities that you need to use like for example we use edge impulse with autoencoders, you want to be able to take time series data and you want to be able to look for patterns on that something you can train for. So I feel like this controller for neural networks for reflexes requires a very different type of infrastructure in terms of what you need to be able to go train these things and it will be very interesting how we take our combined assets and keep evolving this capability. I think it will help us a lot because this is exactly the core thing we are working right on is exactly to build the pipeline where everyone without experience can click on and get the right information out to train the model. So this is I think one of the toughest thing to shorten the training time by a lot.
Let's talk about ecosystem. Yeah. You know, I've most of my career I've spent developing ecosystems, being part of them, evolving them, transforming them, building new ones, and it's a very satisfying experience because you can see the before and the after state. We did that with automotive, that's been a great success for us, and we are starting to focus on that with industrial and embedded, and then suddenly now robotics is here. Our approach with the ecosystem has been you really have to focus on what is the problem statement that if you unlock then there is tremendous value for everybody, for society, and if you pick some goal like that and then you are true to that goal, then people want to come work with you. So one thing that we did last year was to acquire Arduino because we felt that in the age of AI, in the age of how quickly technology is evolving, you have to be able to connect to appeal to developers who want to be able to have access to the best technology and be left alone to develop. We just announced here at Embedded World the Ventono Q which is our second product that we've announced jointly with Arduino which is based upon one of our IQ8 chips that is a highly capable chip. It has safety, it has lot of camera support. It has a lot of AI, a lot of processing, multimedia. It's a multimodel chip and I feel like what we need to do at this point in time is to put a lot of capabilities in the hands of the ecosystem, learn from the ecosystem and then evolve. And I know we're going to work with Arduino to figure out how can we also get a new Arduino partnership going. Well, what's your learning been in working with ecosystems? I think that's exactly also again the difference from us to others is that we are the only one not claiming we can train the model to solve every task in the world. And I think here is very important to mention especially in the industrial space, there are all these companies having this know-how, they know how to do it and that's also why they're existing and this isn't something they want just to give everyone out because it would not make sense, you will have only one company then. And we started to build the Neura platform we call, and this Neura platform is there to build the way that in a developer environment where everyone can train the models and everyone can actually add their information to it, in the same time they can also keep their privacy by not publishing everything. And I think this is where today when I watched Arduino basically talking about what they work on, what's their network or how do you say the community look like with more than 30 million developers, this amazed me the most I would say because I really see this is the way we want to go. And I think the combination of the approaches we have and also the way like what we can probably learn from here, I think it's amazing. It's exactly what we were looking for because I believe that the future will be created by all companies in the world working together on solving that instead of one company making everything somehow and owning everything themselves. I think even in a data game, you can have companies like Palantir but in the same time you can actually just let everyone build their own, be their own Palantir. And this is the approach we are going right now: having everyone give everyone the capability to connect a sensor with an AI model on the platform Neuraverse in their secured environment and deploying the models there by themselves and also taking use of everything which is already developed. And I think this is the great vision of Neura and what we want to do. So here I really looking forward to learn a lot with you building ecosystems. I mean we are just on it. And looking forward also to see what we can do together on that.
Thank you very much David. It's been fantastic in the very short period of time we've known each other. I feel like this was meant to be. So yes, really looking forward to the core Neura partnership also.