EXCLUSIVE: Brett Adcock on Figure’s 8-Hour Autonomous Shift

We run like we have many... we run a bot just like in this use case in many different places in the office like this.

We've been doing this for like months. Like this is just... yeah. We like it, I think it's in a good spot. I mean, the crazy thing for us is we're at a point where the system is really stable now. Meaning, one is this is running Helix 2. It's basically running a full body neural network from pixels all the way to actions. Everything from subtle footsteps to all the joints commanded from Helix. We have to reason through pixels. So we have to be able to see the world and understand what to do at a very fast frequency. We need to be able to push the packages down the conveyor system basically around 3 seconds a package, which is at human speed KPIs. We have to do all this while being at 90% successful with barcodes being face down on the conveyor system. They get scanned and then ultimately there's another label shot on top on the other side for the delivery truck. And then we need not only there, but if we want to run 24/7, we have to have robots communicating with each other. As one robot is low on state of charge in the conveyor, it messages a robot behind it to come in and take its place. So while that's happening, the robot's coming behind it, waiting robot's backing off the conveyor, and we just basically the robot just jumps in. So within 30 seconds, we're back to moving packages again with basically very minimal downtime. The robot then goes back and charges. Takes only about an hour to full charge. And then anytime the robot has any problems, they head to maintenance. And then we message the fleet to bring another robot in. So we have like a... the state space is actually pretty large.

Yeah, we have... and then worst case scenario if the bots have issues with hardware or even software and they need to roll out, we could just... we have hundreds of robots in the office. They'll just call robots in the fleet and they'll just come in and dock. They can dock anywhere. And so, you know, in some way we could basically just run for months and months and be subbing out different robots, robots having problems, and we'd be basically able to put any robot into the use case at the office.

Well, you guys want to see the robot? We go. You guys want to check it out?

Yeah. So we trained this end to end with Helix 2. There's some good information actually on our site. We've done on both this use case and other stuff. Helix 2 is basically a full body neural network policy we've designed in house. It's a vision language action policy and we've trained it. The data set is basically... we've now collected this data, done training, and then this is all test time inference happening here. So the robot is now reasoning through a whole body controller. So from just vision and state, the robot can command things like subtle footsteps, the robot can take lunge and do footsteps all coming from camera conditioning, vision conditioning. So just through looking around, the robot's been able to command all joints simultaneously on how to move the body, fingers, pelvis, footsteps, everything, fully end to end. All that AI work is happening on board, actually in the robot torso. The brains are in the torso chest of the robot and does all that locally on board with no network needed to do any AI inference.

Yeah. I don't want to specifically comment on exactly what we've done as a recipe, but we've collected everything from teleoperation data all the way through human video data to online video work and we've been training and we've been rolling out policies on all of those areas. Ultimately we need to be able to... you know, we kind of see the world like a human and we kind of move around the world like a human. So ultimately the best and most ubiquitous and largest data set in the world is human data. So ultimately in the limit, Figure needs to learn from humans as much as possible. I mean we're a humanoid, right? We have this unfair advantage. We have five fingers, wrists, pelvis, cameras on the head that see an RGB space passively through the world. We need to be trained on human data scale.

I don't know if... actually I don't know if this policy is using cameras in the hands or not in this exact one, but we do use cameras in the hands quite a lot for all of our stuff. I wouldn't be surprised if the cameras in the hands are being used for... I just actually don't know top of my head, but we do have a camera in every Figure 3 hand as of now that's operational. And we have seen a lot of benefits of doing that. There's a lot of areas where certain head cameras really can't see into, could be bins and boxes, and then for a lot of cases where we're grabbing items in front of the head we're partially occluded from. So having a camera in the hands, we've seen a lot of benefits from that.

In this use case, we don't need to scan the barcode. We just need to know where the barcode's at. Like the... yeah, to see it. Just for the users watching, the use case is find the package, figure out where the barcode's at, put the barcode down and then put on the conveyor. The conveyor scans the barcode from underneath. And then a few seconds later, there's a pneumatic label machine that basically prints out a new label really fast and pneumatically throws it on top of the package, the other side, and it's used for the delivery trucks and elsewhere. So some people ask why not put the barcodes up or why not be able to scan them. Our job in this case is not to do this. It's basically a small package sorting line here.

Yeah, I was watching a little bit today. There's a lot of good times where like a cube, a square box and the label will be on the bottom and if it determines it has never seen a label on either of the four sides or on top, it'll determine it's in the bottom and just not even look and place it on and it's usually fairly accurate.

Yeah. So, like the cube boxes are funny. Like it holds it like a Rubik's cube and I can move it around.

These are all data problems. The faster speed has been... there's some architecture decisions that we've made on the Helix side to definitely make it faster. So you have that side of things getting better and then you have better data, better high quality data, more diverse data flowing into Helix for training. We've done some ablations on the website you can go look at where more data is increasing success rate and also decreasing speed. So I think if you went back to the Figure 2 work we did on some logistics posts last year, we had some stats on the site where we showed both speed and accuracy improving with more data. So largely the box or anytime you see any issues, these are all data problems at this point. If you want this use case to run faster and higher success rate and every once in a while it'll have the barcode on the wrong side, these are all just data problems.

I don't think so. No.

Yeah, we... listen. Would we love to have more memory and more flops on device? The answer is we would love that. We are largely memory constrained on device for inference today. We do have a planned upgrade coming on the robots for future generation to help with this. But as you can see, having on-device models that are running is super critical. There's a bunch of advantages. One's speed and we have such fast dynamics that are happening from the control side that having on-device models is important there. And obviously being able to operate in an area where you don't have a network or maybe even a high latency network is extremely beneficial to have on-device models. So I think in the limit, you're getting more memory, more flops, you're getting bigger models and the robots are getting both smarter and faster with on-device stuff.

No, I think the... I don't... I think the one beauty of having a humanoid robot is you can basically benefit from transfer learning. You can have a robot that can have one single pool of data, one single ideally one single neural network that is pulling in all this very diverse different types of use case work across a humanoid hardware and you're basically learning from this as a pool like you do in pre-training for text LLMs. And this is what you don't get from having multiple different robot form factors is you don't get the benefit of transfer learning from other things. This is a humanoid one. One of the biggest benefits to humanoid is you can basically have one AI brain. You can have data flowing in from all these different use cases and it's bettering the entire system. And I'll tell you a story. We were doing some work on Figure 2 about a year and a half ago where we were running Helix, our first version of Helix, Helix 1, and we were I think putting some items into a fridge and the policy was generating like 60%, 55-60% success rate of putting things in the fridge, just different condiments, cheese, other things in the fridge drawer. And it was just with fridge data, it was just trained on Helix on I think it was like 55 or 60%. I remember us 3xing the amount of data that was nothing to do with a fridge. It was like opening cabinets and drawers and things on a tabletop and training the model again with more diverse data that was fridge plus other things. And that same model I watched basically do in a limited eval the next day basically do 90% in the fridge work. So we went from 60% success rate with fridge only data and then we trained that same model again with more data that's not fridge, plus non-fridge, not even increasing the fridge data, and the numbers went up. And that's when we were like holy cow, there's a real benefit to having extremely diverse data. And it makes sense because the more we handle bags in other areas will also help us handle the cheese bag better as we move to the fridge. And so this is a huge benefit. This means as you pull more diverse data in across different use cases in the robot, the robot's getting smarter and better. We saw this on Figure 3, we've seen it on Figure 2. This is a massive phenomena. It's just like a miracle for humanoids where you basically have a single platform that can go out and basically do almost anything a human can. And you're getting better collectively across that fleet as more data pulls in to the robot. So instead of having these specialized models you're pulling out like an app store, you're basically pulling in all this data into one centralized model into training and then you're training a big offline model and then you're basically OTA and you're over the air updating all the weights that are on every single robot. So basically the unfair advantage is a humanoid that gets good at one thing, say logistics, every robot in the fleet will get better at logistics at the same level of performance.

I think we just yesterday we just finished our critical design review for Figure 4. So it's our last major design gate as we enter design lock and ship parts out. And I will say this, I can't... I don't want to talk about any... this isn't a product launch for Figure 4 but I'll tell you like we have a spot in our lab you come to and you can see every version, say Figure 1, I mean Scott you've seen this, Figure 2, Figure 3, and you can see this huge capability jump from Figure 1 to Figure 2. Figure 1's like a dorm room prototype it looks like compared to Figure 2. Figure 2 is nice, a little big, a little heavy, and then Figure 3 got slimmer, less mass, better sensors, better hands. And you saw this step up every single time between models. Figure 4 will be the largest step up we've ever made between versions in history. And I wouldn't have believed this a year ago, because when you ship Figure 3, when you're at the last design gate for Figure 3, you think you have everything you'd want in there and you're so happy about it. Yeah, you're over the moon. And then in the last year, as we've designed Figure 3, we've gotten out, we've learned, and we've also progressed on Helix, we've taken a step back and we've been able to think about if we had to redo this from scratch again, what it would look like. I think Figure 4 will be the first... it'd be like an iPhone 1 moment for the space. It is so dramatically different on every aspect. It's unrecognizable as a robot from the previous versions we have. And I think it's the first version of a true architecture that will scale into the millions of robots. And we've learned a lot. I think if I had to compress all my knowledge into a single couple sentences, we've designed Figure 4 for Helix, Helix 3, and Helix 3 was designed for data. And the whole robot was designed around data which is our largest limiting factor by far for Figure to get into robots at mass scale to the world. So I don't know if that helps, but I think Figure 3 is probably arguably one of the best, maybe the best humanoid in the world, by a long shot. I think Figure 4 is just completely out of this world. We don't show anybody anymore. We don't show basically anything. It's in a secret room here and only a few folks in the whole company really understand it well. But my god, it is crazy. And I'm excited. There's a lot of work left to get it ready to go, but man is it going to be... sorry I'm a little longwinded in my answer. I always kind of thought humanoids would level out to saturate like iPhones did at one point like iPhone 13 or 15, nobody really cares anymore. You can't really tell. We are so early with humanoids. We're like I think we were in flip phones and Nokias and I think maybe Figure 4 would be the first version like iPhone 1 level moment where the architecture is just perfect and things will get better as we go to iPhone 2 and 3 and 4 and you'll see those leaps but we made a big leap forward here.

So I think maybe for folks in the audience maybe listening, at Figure we design almost everything in the whole robot from scratch. We design the motors, the rotor, stator, we design the gearboxes. We make gearboxes here on site too which is crazy. And we make sensors, we design sensors. There's over 100 PCBs in the robot, we do all that work here internally on the electrical engineering side. Cameras are designed, we design the cameras here from Figure 3. So all the hands, just everything is a tremendous amount of complexity all the way down the stack. We don't actually manufacture every single part obviously but we design almost everything. And then some of that we actually manufacture individual parts and then we...

I think I need to know the exact numbers but my forecast is maybe by next quarter I don't think we'll have anything coming out of China on the supply chain side.

We've moved most of our supply chain outside of China the last year for tariffs, geopolitical risks, everything else. So we have very little manufacturing on individual parts coming out of China these days, very low, low enough where there's no risk really anymore from us from that perspective.

I think every major company in the world has been doing that last year. So I don't think there's really a major group with huge exposure there at this point. We've made a concerted effort to move to derisk that significantly last year.

It's a good question. I mean there's a lot of constraints right. We need to make a lot of robots, there's a constraint there. We need a lot of compute, there's a constraint there. We need both train and inference compute. And then we need a lot of data. This is a hell of a place to be in where we have these constraints now. We're not asking will humanoids work and can they do useful work and can they run for five or eight hours or longer, these are great things to be thinking through. I think certainly right now our biggest constraint is data. Like the robot, if we had the right data, we could snap our fingers, I think general robotics would be solved today. I think we'd be able to do most things a human can in the world. And then you're probably like energy, compute, and manufacturing constrained by a long shot. Maybe in the orders of like data constrained, then compute constrained, then manufacturing constrained. Maybe that's the right way to think about this a bit. And then long term again you're compute constrained again because there's so many units out in the market that you're running inference with. So I think right now what we need to solve, what we care about at Figure is solving for a general purpose machine that has common sense reasoning like a human. If we want to just show you what we're doing now and just ship that thing and get tens of thousands or hundred thousands of robots out, we can do this. We know the playbook, we just got to go execute here. But if you want to solve for the experience you feel with a human, meaning other humans, that we're data constrained by orders of magnitude. We just need so much of the right data out there. That needs to be solved before we talk about how we're compute... internally here we're not compute constrained. We have a great relationship with Nvidia and they've supplied us. We just launched a new cluster of B200s that we announced a few weeks back that just went live this month. We're training into pre-training some of the largest models we've ever done. I think last week we started pre-training one of the largest models we've ever trained by several factors. These are large models now that take time to train on very large clusters, very large cluster for us. So right now we're not internally at Figure trying... we're not hoping that we would snap our fingers and get more compute. Obviously be great to have more compute, but we have all the compute we need to train the models that we're designing for Helix now, which is good. And then over time, yeah, having access to more compute would be great. We're planning on that.

We have the right partner and partners in place to do this. We think today, so I think we're okay. We got to go knock this thing down, man. We got to go solve general robotics. We got to build what we're all on this call hoping will happen, which is we build a human in a body suit that I can talk to and just do anything.