Welcome. Thanks so much for joining us and coming early this morning. You heard the title of this, which is 'Building AI Systems When Failure Is Not an Option.' And I do want to get into, you know, best practices. But first I want to start with where we're at in terms of a pretty broad industry of physical AI, but nonetheless. Every other day it seems like we hear of, on the LLM frontier labs piece of AI industry, rogue agents like breaking through, and I don't hear that as much in the physical AI world, whether it's robotics, whether it's autonomous vehicles, whether it's in defense. And Nathan, I want to kick it off with you: is it just that we're not hearing it, or is the physical AI industry, generally speaking, just more responsible, possible, and tackling safety very early on?
Yeah, absolutely. I think it is that case that we are really thinking through how to create these systems, build these systems, deploy them, and do so with the types of assurances and guarantees that are required by our customers and our partners and allies and similar. Within Shield itself, just to set the stage, we have a part of the company that focuses on autonomous systems. So think of these as platforms that operate in the real world, as well as a framework and ecosystem that we call Hivebind that really focuses on both building the intelligence as well as operationalizing physical AI. We've deployed in environments in the air, at sea, even in space. We've deployed intelligence in operations on everything from maritime vessels going on the surface of a littoral environment to satellites in space or uncrewed jets and similar. And the key takeaway to answer your question is: from the very beginning, we focused on the infrastructure, the tooling, and everything required to really ensure that what we were building is ready to not only be operationalized but to really work in a trustworthy and reliable manner. So from the very beginning, it was very intentional, and we invested more on those capabilities than we actually did on, say, the development of the edge-level intelligence itself.
So with Shield AI in mind here, I mean, you said what your company is doing, but what about other players or even competitors in this space? Do you feel like they're taking that same level at like a very base foundational level of safety, or are you just the standout here?
So I think it really varies, and it depends based on the country and the urgency and the need as well. If you're talking about other companies in the space within the US itself, I think there is an intentionality and a thoughtfulness, and also you have to keep in mind that when you're talking about the, let's say, DoD or similar, there are really well understood ways of taking a technology and maturing it across technology readiness levels to really deploy from an experimental status to a mature and stabilized framework. And so we have that insight over many decades of building advanced technologies in the defense domain within the US. Our partners and allies have similar frameworks. But the reality is there are many operating in this physical AI space, in this realm, where the threshold of urgency and need does translate into a level of risk-taking that doesn't necessarily meet that standard. The big question that we had to ask ourselves was: how do we hold these high standards while being able to move really, really quickly with agility? Because for us, our mission sets need very dramatically, the platforms very dramatically, and the urgency of need could be something like, I need this to be able to, you know, I need this platform X to be able to run this very unique mission Y, and I have a few days to execute it. So really thinking through how to make that possible while holding high degrees of performance assurance.
Raquel, I'm wondering in the autonomous vehicle industry if you're seeing that same level. I mean, we certainly have seen safety incidents occur, but it feels different than sort of what we're seeing in the frontier labs like LLM world. It's more how autonomous vehicle technology might be working, like operationally, when there's a blackout or something like that. How do you feel like that industry is tackling safety, generally speaking?
Yeah, I will say that it's clearly very different table stakes when you think about, you know, for us, an 80,000-pound truck, you know, driving at highway speeds. You want to make sure that it's always going to behave in the right way, right? So I would say that the playbook that has worked for LLMs is not a playbook that works for physical AI, right? And the LLM playbook has been: you can deploy a model that is not necessarily super performant, but it's going to be very useful for humans if you just sprint enough. And it's going to still provide you, you know, all the value that you want, or much better than, you know, in the absence of that, right? And then alignment kind of came afterwards in terms of, you know, more of an afterthought. In the physical world, you can't do that. You know, the consequences can be tremendous. So you need to prove safety before you actually deploy. So that has, you know, fully transformed in terms of, you know, how we go around the development process and how we go around the deployment process. Now, when you think about proving safety for self-driving vehicles, it's actually not that easy. Why? Because, you know, if you think of how often do fatalities happen in the real world, it's actually not that often, which is a good thing, right? But if you take highways, for example, every 100 million miles there is a fatality. And when you think about how are you going to improve the safety of the systems without, you know, you can't have a fleet and drive, you know, as much, right? So there's been a lot of technology developments that, you know, Waabi has been at the forefront of: how can you bring AI systems and simulation in order to give you, you know, that kind of unlock in terms of really being able to understand safety before the deployment. Now, in the industry today, not everybody is going around this in the same way. And I will say that there is a shortcoming that we are starting to see, particularly on the trucking domain, that comes from a lot of, I will say, pressure from showing investors that you are actually advancing, and a consequence of people not having enough money on the balance sheet to actually have, you know, large runways that will allow us to just focus on doing the right thing. And what we are seeing is deploying driverless on public roads with platforms that haven't been fully validated, and that's actually pretty dangerous. So that's something that we absolutely will never do. You know, we believe that both the hardware, the software has to be fully validated, and it has to be, you know, ironed out before you actually put that on public roads, and that's different from others.
Yeah, Michael, I'm wondering how that relates to what Raquel was saying, and specifically around like the validation and simulation piece, how you apply that to robotics. Robotics, obviously, incredibly general, like you can apply it in so many different ways. So let's narrow it. Let's talk about humanoids, which I know is a favorite subject of yours. Is that where you're starting when you think about safety and applying it? Is it how am I going to validate this, or are you starting at a different point?
Well, in industrial robotics, so I've worked there for about 10 years, there are really well-defined regulations and test methodologies to show that you have a safe thing, a safe piece of equipment to put into a factory, a warehouse, or anything like that. And so under those regulations, to date, you aren't allowed to use any non-deterministic algorithm for safety purposes. So already in the industry, when you're working on any kind of robot, you are architecturally making very strategic decisions about what is a safety function and has to be implemented a certain way, versus what is not a safety function that may be able to leverage even machine learning, let alone some of the new AI technologies. And so that's the way the industry has been going about it. That means that most of your motion controls typically are deterministic algorithms that are not driven by any kind of model. What's changing, especially with humanoids, is that companies that are using models to drive like the full-body kinematic control, well, now your control is driven by something that is non-deterministic. And what do you do about that? And with two legs that are dynamically balancing, you turn off power, that thing falls over. If the power is on and you push it, it does what a human wants to do and it flails, which if you're nearby that thing, you might get hit and that might hurt. And so we're having to think about really different kinds of potential hazards in industrial environments with the advent of bipedal humanoids. And I think there's a really interesting kind of pushback coming from customers in that industry to say, listen, like if we are going to need to figure out how to mitigate all of these risks introduced by this form factor for safety, there needs to be real clear benefit to it. And that's why you're seeing a lot of the humanoid companies now introduce wheeled versions of their robots, because you take away those legs, you just solved a bunch of safety problems. And they're starting to respond to that, too. So it's been really interesting seeing it go from, oh, look, I can control an entire humanoid form, to, oh, you know what? Let's just focus on those hands and make everything else simple.
Yeah, it's interesting, like this dialing back of, like the dialing back of the technology so it can be in a space and scaled and deployed as a first step. And I'm wondering if you're seeing that in defense as well, or if it's the opposite, right? Like, how much can we do and how fast can we do it?
I mean, it's very much what was just described. There's a high degree of caution and consideration. We have to think about everything that was just detailed there in terms of meeting certain requirements and compliance and certifications, and we have to have strategies for being able to navigate the use of non-deterministic cognitive capabilities on top of what are largely systems that have to be assured and certified. So you have runtime assurance that is interfacing with the system, you have guarantees and certifications on the underlying software that's running it, you have ways that you have to bound what different techniques you use, and your choice of methodology is really focused on making sure that you're able to deliver the right levels of performance with the right levels of guarantees. So either that's a deterministic optimal outcome or some kind of bounded performance guarantees, but you're able to do that in a timely manner. And so these are all the types of considerations that we have to think through and be able to leverage and understand as we deploy on these systems. I think going back to your earlier question of why are we seeing maybe fewer rogue scenarios like what you were describing there, I think what it comes down to is that the cost of failure for everything that's been described here is very clear. I mean, the cost of failure of a large truck hitting people, very clear. The cost of failure of poor decision-making in a wartime condition, also very clear. Or even if you're just testing and experimenting on a hundred-million-dollar-plus platform, the consequences are very real, they're physical, and so we have to be extremely cautious and meticulous in how we bring those capabilities to bear. So then the fundamental question that we have to ask is: how can we meet those expectations, standards, and requirements, but move through that process blazingly fast so that we can continue to lead and be at the edge of the state-of-the-art?
So obviously there's very specific mechanics that you will put in place, but I want to step back and talk about what that feels like to be part of companies in which that risk is, you know, potentially life or death. It's very real. And Raquel, I mean, you're leading a company. You've started Waabi a few years ago. You're on the road. And as a founder and CEO, how are you managing that? Dare I say, emotionally, as you also want to move quickly? And I think I ask that question because there are a number of founders, I think, who want to know how to do that: is how do you pace and scale and also manage understanding what those stakes are?
Yeah. So I will maybe give also for people that don't know Waabi, where we are at today, to frame a little bit your question. So we are building a, you know, leading physical AI platform that is composed of, you know, general intelligence that drives the vehicles as well as the simulator that enables us to really test the safety of the system. Today we have, you know, massive customer traction. We just announced a couple of our customers, including today, Volvo is a partner, and we have a massive partnership with Uber where the Waabi Driver is also going to drive our robotaxi program. So where we're going to deploy more than 25,000 robotaxis. So let me say that again: more than 25,000 robotaxis. So as you can imagine, you know, table stakes here are very, very high, right, when you think of the sizes of the operations and things that are coming. So definitely for me, it's top of mind. And actually, it's so top of mind that the reason why I'm still in self-driving in the first place, and it's been, I think, 18 years now, is because, you know, unfortunately, you know, I have lived through my life accidents of people close to me that were pretty, you know, pretty severe that have changed lives of people around me, right? So it's very clear that this technology is going to save many, many lives, and I feel that is for us, you know, we have a responsibility to bring this technology so that, you know, 95% of errors today on the road are ones that actually could be avoided, right? So we can be superhuman, right? And every day that we don't deploy the technology, there are many lives that could have been saved, right? So that's also the framing that oftentimes is not talked about this way. But obviously, you know, you need to do this in an extremely responsible manner, right? And there are actually ways that you can go very, very fast but also minimize risk. But in order to do this, you need to really innovate in the type of technology, right, and build from day one, you know, technology that can scale and technology that can solve also all the corner cases and things that you're going to see on the road, right? These truly intelligent systems, so that indeed when you deploy that in the real world, right, you are not trying to chase the tail of, you know, small corner cases and things that you didn't think about before. So to me, it's about investing in the right technology. Make sure that you build that from the ground up for everything that can scale. Make sure that you have the partnerships for scalable, safe hardware as well, and then bring that all together to market and build something that the customers really want to utilize. And if you do it this way, you know, you will not put just a small prototype on the road immediately, but you're actually going to build the right technology that, you know, you can scale and basically leapfrog everybody else in the industry super fast, right? And that was how we built Waabi, and it's been amazing to see that you can go fast, you can go safer, but you need to really innovate and build this new AI kind of next-generation technology to get you there.
If I can actually build on something you talked about, the scale, right? I think that's a really important point. And I worked previously at Amazon Robotics where, you know, we were all kind of reminding ourselves regularly, right, a one-in-a-million thing happens daily at Amazon, right? If you think about the number of packages shipped and the number of robotic interactions with those packages, a one-in-a-million happens daily. And so thinking as a developer, you have to set this culture of that when you are building something that you want to scale, to say, you know, you can't dismiss, oh, that'll never happen, that'll never happen, because if you are successful in what you are building, it will happen a lot, and you need to have already been thinking about that, already been architecting for it, and already making the right decisions up front to make sure that by the time you reach that scaled success, you have built something safe.
Yeah. And I will say that as a founder, as a CEO, the culture starts at the top, but you want to make sure that everybody really embraces that culture. And that's super, super important when you think about what your employees really care about. Why are they here? What are they trying to do in the first place? Right. And, you know, for us, it's never compromise safety no matter what. Right. And that's something that should be, you know, frontal and clear for everything that we do.
Yeah. I mean, oftentimes we think of safety in terms of an engineering problem, right, and like how to tackle that. But there's many incidents where it's company culture that has created an environment in which the engineer doesn't speak up or doesn't say the thing. And so it's not an engineering problem at all. It's a human problem. And, you know, Nathan, how are you thinking about that in a very, you know, high-stakes company like Shield AI?
Yeah. And it absolutely is sort of a high-stakes situation. And for us, because we both build hardware as well as develop software and really look toward deploying that in the field, there are different ways that you can talk about safety. You can talk about safety in the manufacturing line and how we produce these platforms. And there's a way to engage there with that type of teammate to make sure that they're comfortable in always communicating concerns around safety. When it comes to the software or the capabilities that we're bringing into the field, we have to have those conversations around rigor and precision and understanding from an observability and a telemetry perspective, so that we have highly granular insight into exactly how these systems are operating and what they're thinking and what decisions they're making, and being able to really track that carefully and then have the mechanisms that exist to communicate that. So that's a different way of thinking about safety in terms of not just how we work with these systems, but how these systems themselves operate and that they operate as expected. And then finally, we have to think about those teammates who are out in the field working side by side with our partners, with our allies, with the US DoD, who are there facilitating and enabling and supporting them to be successful in the use of what we've created. And so then you have these high-stakes conditions where you have teammates having to make very thoughtful decisions with high complexity in high-stakes conditions. And so it really varies across the environment. So we internally, as a leadership team, spend a lot of time thinking about how can we approach this both in a more narrow manner and specific to those different parts of what we do, but holistically in terms of establishing a culture of clear communication, clear expectations, a willingness to stand down, a willingness to change what we're doing, a willingness to hit pause on the release of software, what it takes to make sure that we are maintaining a threshold of performance and assurance that meets our standards and the standards of our partners, our allies in the US itself.
Michael, you've interacted with, you know, so many different companies in robotics. You're at GM right now. You mentioned Amazon. And what is, in terms of like the foundational safety mistakes, maybe unintentional even, that you see big and small companies make when it comes to, you know, robotics or applying AI into the physical world?
I think there are two common mistakes. So one is that, technically speaking, in the US, industrial safety regulations are optional. If you have an incident, OSHA will show up, investigate, and if they find that you did not comply with an available standard, you will get fined. And so large companies generally obviously don't want the liability, they will always comply with the standards. But a lot of small and medium manufacturers, third-party logistics providers, technically it is optional. And there are companies that decide, I'm only going to sell to the ones who aren't going to mandate those regulatory compliance things, and we'll build from there. And so I think that there is a temptation to make, like, it is hard to design a robotic system that complies with safety regulations for industrial use. It is hard. I do not at all resent that difficulty because it is so critically important, and I think it creates really interesting constraints to drive innovation. So you can really differentiate and be innovative by doing safety well. But there are companies who decide they want an easier way out, and they'll just target these customers, and so they don't bother at all. So I think that's one thing, right? It becomes a self-limiting issue. The other issue that can compound that is it is really hard to make a system safe after you have already built it. It is down to the choices of hardware. The embedded computer that I had to use on one project had two cores at 90 degrees so that if it was hit by a cosmic ray, only one would fail. Like, the level of redundancy in the hardware leading all the way through to the software that you implement is really, really an early design and architectural choice that has to be made. You can't slap it on later. You can't just buy a safety-rated LIDAR, put it on, and say, ta-da. That's not how it works. So I think it has to be from day one. It is a culture thing. It has to be set. We have to agree to take on those constraints and that difficulty, and it has to be done from the ground up.
