The New Pace of Space & Defense: Speed, Scrutiny, and Early Validation

I prefer going before Bob because Bob's an astronaut, so it's hard.

Yeah, no, I appreciate that. So, Matt McElvogue, VP of Design at Teague. So Teague, like Mo said, we're a human-centered design firm. We work across aviation, space, defense, very relevant for here, but also robotics and autonomy, and we find that those conversations are increasingly relevant in other sectors as well, of course. But we're hardware, we're software, we're engineers, prototypers, researchers, kind of all under one house. And, you know, we've helped design human interfaces for LEO, for lunar, for launch. And recently, like one of our sort of recent things is we helped design and build a full-scale, we call it like a living twin of Starlab station that's now set up at JSC for like training, equipment validation, operations testing, and all that kind of stuff. And we've had a number of things in JSC over the last sort of decade or so. So we're very excited for that. I mean, it's exciting, I think, for us to see NASA get like their first permanent commercial space station mockup on site. And, you know, I think that signals kind of an exciting era for commercial and private. And I think that'll be a part of our chat today. So that's me. It's nice to be here. Appreciate it.

All right. Well, thank you, Matt. As Mo mentioned, I'm Bob Behnken. I'll talk a little bit about kind of my background because while there is some familiarity with Lockheed Martin Space, the kind of ways I'm going to answer questions today, Mo, is really a reflection on kind of that history that I've had. And so, you know, when I finished graduate school, I came on active duty in the Air Force, spent some time in the Air Force Research Lab before going off into the flight test community working the F-22 program back in the early days. Was ultimately selected by NASA to be in the astronaut office. Got to fly a couple of shuttle missions but then participate in something truly unique, the development of a new spacecraft or actually a couple of new spacecraft. Orion was under development during my career as well as the Boeing spacecraft, the Starliner, and the Dragon space capsule as well. So we had the three spacecraft that were all underway kind of during my career and it was really interesting to take that Air Force Research Laboratory, that flight test community experience and try to reflect it on the development activities that were required to bring those systems forward. And fast forward when I landed at Lockheed Martin Space, you know, that's what's been on my plate is the development of new technologies, applying them to challenging environments like the exploration of deep space or human exploration portfolios that we have. You know, and one of the things that I hope we get a chance to emphasize today is really how important it is as we try to move forward with speed that we move forward as well with kind of a methodical approach that includes kind of the right kind of testing, the early testing that ensures that we don't have discoveries kind of late in the development cycle. I think it's the most expensive time to have a discovery, definitely something that you have to fix. You know, the most expensive time is if you don't discover that till the end of the program. And so being able to make sure that we have those right checkpoints, those right demonstration activities, whether it's what you described, Matt, in terms of having a kind of a test facility that's in front of the users kind of early and often, you know, that's a recipe for success. And I think it's really important that we ensure that those things still find their place as we try to compress and move forward. And so there are opportunities in today's environment with digital simulations, with the environments that we can put in place from a rapid prototyping perspective that really lend themselves to having a place and being able to still do some of the things that are important from a rigor perspective but to do them at speed and at pace so that, you know, we don't have those late discoveries, those expensive discoveries.

I guess I'll jump in and start, you know, and I think I kind of gave that answer and I'll reflect a little bit further back from my test pilot school experience and the combined test force that the governments would try to put together bringing contractors. When I worked the early flight test, we brought the contractor community. I flew with contractor test pilots when we were performing evaluations and we set up those rigorous test points kind of early in the process so that we would have that discovery. If you don't create those opportunities, if you don't kind of push those teams together, you get into too formal of a review cycle and you don't create situations where you can facilitate that discovery and facilitate the users on both sides, you know, the engineers and the operational users that are on the government side as well as on the contractor side, just the understanding of the problem in general. If you don't put them together in terms of executing the test, you won't necessarily be in agreement that there's a problem that's worth fixing or that it has potential kind of complexities that if they're not addressed early, you know, could potentially be a problem. And I will say that having both sides of the fence see those challenges kind of early in the process is really critical. Otherwise you just burn time as you move forward. And so we were able to do that for most of the programs that I was a part of and when we didn't, you know, they were definitely challenges. I know there is continuous pressure to kind of drive out all of the unnecessary elements. But, you know, I firmly believe that those early integration points and forcing them to be maintained in a way that creates structure for kind of feedback, if you will, between the quality of the design and the ultimate user set is really important. The one thing I would add to that is relative to requirements, right? If you can bring the user community in and get expertise both on the, I'll say the customer side because that can be investors, that could be ultimately users, but if you can bring them together and recognize that, hey, this may meet requirements but it doesn't meet the mission need, you know, we can have discussions about changing requirements to focus on ultimately delivering on mission, which is what we all want to happen, right? And we use requirements as a tool to try to make that happen, but sometimes they're not the right tools to guarantee mission success or guarantee that we're meeting the mission the way that we intend.

Yeah, just to add a couple, fully agree with Bob. I think bringing people in as early as possible to challenge requirements is critical. You know, we joke at Teague that like we get the call when they drop the ball, right, as an outside firm that can kind of help with things. And sometimes we come in in a scenario where that hasn't happened and it's very hard to challenge those once things have started to get baked and teams have been challenged with things. And I think bringing lots of different perspectives in early can help challenge more technical requirements to make sure that they do accommodate for people and in the commercial future for customers, right? Not only operators doing things out there and mission control folks, but also the customers that are worried about their IP or wanting to see results or whatever it is, right? So I think that's critical. And then the other thing we kind of go in with is a bit of a mindset shift. It's like it's very normal to kind of talk about milestones and gates and PDR, CDR, and all this kind of stuff. And we try to structure those gates, if you will, around like learning cycles. It's like what are all the things we're trying to learn in the run-up to those? And it's subtle, but that creates kind of like an experiment, like set a hypothesis, try something, learn, iterate, evolve, move on, versus trying to only land kind of a buttoned-up plane right before those milestones, which can be one of the mindsets that we kind of run into. And then the last thing I'd maybe add is put people in the room that are very good at identifying the sort of minimal viable product around that testing and learning. So what's the right fidelity? What's the right medium? You know, is it a foam mockup that gives us like a low fidelity understanding of a space? Is it something higher fidelity? Is it something in virtual reality? Right? You need people that understand those mediums so that you can deploy the right one at the right time. But we don't always run into that.

Yeah, I can talk to earlier startups in there. It can depend, is what I would say. And it's not a hard rule that one is like, you know, sort of neo players and the other one's like paleo kind of players, right? It can depend. I'd say the rule of thumb, like startups are going to work with us because they need to extend their capability and move very fast. And we, by dint of kind of how we work, which is typically fixed fee, fixed timeline, has to get done no matter what, we're very good at scrappy problem solving and moving very quickly. And I don't mean that to be an ad. I'm more saying like we come in with those folks to extend their capability oftentimes, right? Something that they can't do. They really don't want to grow into that space. And we're very like MVP focused. It's like, okay, it's exactly what needs to get done right now. Let's make it happen together. We may also bring like outside perspective that helps kind of round them out. But, you know, normally you find pretty hard-driving groups in the startup space, space and defense. They have a pretty clear idea of where they're going, right? So we're coming in to sort of extend their capability. I think with more like incumbent players, if you will, we're coming in to provide sort of outside perspective, additional ways to think about problems. And I think when it works best, we're all kind of like leaning into shaking up how things have been done to defend against, you know, new players or a lack of speed or something like that. That can depend wildly inside of these companies, right? Like Lockheed, very large company, right? There are all kinds of brilliant thinkers there and we kind of come in though, I'd say, to bring some nomadic wisdom and maybe still bring some outside skills that they don't have internally as well. So, maybe a pretty plain answer, but that's kind of like how we get involved with some of these different groups.

I mean, I think for me it's the fastest programs nowadays aren't the ones that like nail everything every time, right? Like that can be the perception. It's really programs that like learn the fastest. So we have a philosophy here of thinking through making. So it's like you're always building what you're designing and the fidelity generally sort of the curve goes up, like you end up at a higher fidelity. But there are times where you spike because it's the right method for that moment, right? But I think mockups let you validate things that are on the page or on a computer screen in a way that you can't quite see when they're in those, like when you're just looking at it in CAD or on a piece of paper, you can't feel it and like understand it. And I don't mean that to sound super mushy, but when you put it all together and you actually test like, okay, how are we going to get all this equipment in here right through that door? How is that going to come in and get set up in microgravity? And once you're doing that, I think the physical space at a very low fidelity can teach you a lot of lessons a lot more quickly than trying to do that all on paper, right? And I think Bob and I like vehemently agree on that. I think the biggest risk on some of these programs is this stuff gets, like you said, Mo, it's like it's a nice-to-have or it gets cut very quickly. And Bob, maybe you can talk to some of that. I know I don't want to steal that line from you. When we had our prep call, we talked about this getting cut too quickly, right?

No, absolutely. And I think kind of an important thing, and I think Matt, you kind of referenced this, when we use the term mockup, probably that at times can be too disparaging in terms of what the utility actually is. You know, they're fundamentally models of varying fidelities. And you probably have all heard the saying before that all models are wrong, some are useful, right? And so, as you kind of lay this out, or some are more useful than others in some cases. So when you start, you may have a kind of a sales concept that you're trying to push forward with drawings or a diorama or something along those lines, but kind of throughout the lifecycle, the utility of kind of that early prototyping is the way I would describe those elements. And they're just super critical in terms of being able to identify problems early, solicit feedback in a way that's productive. Like we've all sat through large decks of PowerPoint presentation at some point in our careers. And that's a challenging way to evaluate in your mind's eye. Did you truly understand what was presented or did what was presented truly capture, you know, the content of the design as the team was trying to review? But sitting in a mockup of some fidelity, sitting in a simulation of a system and how it's supposed to operate. And then the next step of how do you do interoperability between systems, right? Where you've got your fidelity simulator and that could be a physical mockup that you can walk through or it can be something more complex. Like the discovery that comes from trying to force those pieces to be integrated is super critical. I'll give you one example. You know, it was from my astronaut experience where we were going to put something together on orbit on the space station and we had seen a lot of drawings about how a MLI blanket, you know, a multi-layered insulation blanket that we needed to install was going to fit together. But until you had one and tried to connect the 30-plus Velcro tabs in the appropriate order to kind of turn that flat piece of insulation into a structure on board the International Space Station, you couldn't do it. And I certainly didn't want to discover that, you know, in the middle of a spacewalk as we went forward. Going through that process, we discovered problems with the design that were able to be rectified in a much simpler fashion than had we had to, you know, not power that system, fly a new blanket of some sort, schedule another spacewalk, right? Just simple things to kind of put the hardware in folks' hands in a way that PowerPoint or presentation materials or a 1D screen, even if it's just a simulation, doesn't provide. You know, those are opportunities and missing them, deleting them are not deleting the sales content from the front end after the project's already won. They are truly the tools of kind of successful design and successful rapid design.

Yeah. And I would add that we're like right now we're at this transition moment where the kind of people that are going to be coming up into space, they aren't like Bob, right? They aren't like Peggy and Rex and other people we were speaking about earlier. They're going to be trained on shorter cycles. They're not going to be maybe career astronauts in the same way. And I think, you know, early mockup, early testing that involves them so that you can validate some of the sort of paradigms that we've leaned on for a long time with very expert, highly trained people. That's going to help us discover things out ahead of kind of accommodating these new users and their needs and commercial enterprise and all this kind of stuff. So I think it's more important than it maybe ever has been with like the new crop of launch and access and station vehicles.

We're just a different generation of people now, right? We watch YouTube videos to figure out how to do things. We watch somebody else's mockup at their house, which might be the same as our car, but they walk us through the instruction set that they've got. And we all find that extremely valuable. We don't pick up the manual and read the instructions in the way and then expect it all to fit together. We are used to seeing, you know, and then believing the things that we see. Now we can talk and have a separate conversation about AI in that construct, but, you know, we believe the things that we can physically put our hands on and see them work together in different ways than we do from a, you know, just a PowerPoint presentation, for example, which has been typical in our design cycles. And we just have new things and new expectations as we go forward.

Yeah, Bob, I want to call you into conversations I have sometimes with clients to be like, listen to Bob. The amount of times we talked about YouTube generation when we're trying to look at like XML procedures for like fixing something, it's like, guys, we've got a giant gap here, you know. So I appreciate that.

I can answer first here. Bobby, you jump in. I think the Lockheed perspective will be super interesting. A lot of what we're thinking about is it's a combination of, you know, what are we trying to learn, right? Cost. And then I think there is always a factor of like what do we already have that we can build upon, right? So, you know, because we are a third party, we have certain tool sets and things that we can roll forward and kind of build on, right, our own IP in some ways. It's a little less like polished than that makes it sound oftentimes, but we're trying to do as little as we can to get the learning that we need to get, like during prototyping, I'd say is the main thing that's on our mind when we're thinking about that as a methodology to move the program forward.

You know, I think you touched on some really important piece there, Matt, which is what's the min viable kind of solution, if you will, that kind of gets you to the data. I'll say in some cases I really think, and this is maybe a little bit counter to what would be expected from a large prime because we can do all of those things, right? I can, from a prototype perspective, I could put something on orbit and at times we do, you know, build a system and we put it on orbit to kind of be a prototype for a mission-capable sort of a system. In general, what I focus on is kind of the quickest. You were talking of a definition of what minimum actually is. Is it cost? Is it speed? From my perspective, that speed element is super, super important. And so identifying what prototyping methodology is going to allow us to get credible data depending on who we need to convince. Right, in some cases that's ourselves. It could be an engineering team. It could be kind of the broader enterprise on the Lockheed side that we need to convince. It could be a partner, a company that we want to work with. It could also be the customer, right? And so I've got to balance all those things together. And I typically try to err on the side of, hey, what's the fastest thing that we could do based on who that audience is for that particular data and try to lean forward from that perspective. I think when we do it wrong, and it's probably a thing that the primes are often accused of doing, is we think about it a long time because we can do it so many different ways. And that speed element is one that, speed to data that we can get in front of the right audience is what I think about from a prototyping perspective.

Yeah, it's funny, like I think when I think speed, right, I think a little bit about like how some of our other clients in other sectors operate and it's almost like a different question, Mo. It's, you know, if you talk to an Amazon, you think about how well their logistics operation is like instrumented. Like they know how many times a door was opened on like a delivery vehicle, right? And can look for efficiencies in that. They'll know exactly how many seconds somebody tried to work out what the next action was on a, you know, delivery information screen. So for me, I do think there is a need to go to sort of like a next level, which sometimes space and defense can feel like it's in some places the most advanced area that you can work and then in other areas it's not pushing as far as maybe we could go. So I think there will be a shift from kind of validation or prototyping as sort of a decision point or moment in time to kind of instrumenting systems and thinking about prototyping as like continuous validation. It's kind of always happening based on what we're learning live and how systems are needing to operate and evolve based on changes because we can't have what we had with the ISS, which is where we've sort of added and kind of broken and fixed things over time because you end up with that, I don't know who called it this, but it's definitely stuck around in my head. You end up with the crazy person's garage when you like see a photo of the ISS, right? And that's maybe not the best work environment and it's not the greatest for like modularly changing missions, right? So I think thinking into the future with like a digitally enabled continuous like validation and evolution is probably like where we need to go as an industry, you know.

Absolutely, the whole element of digital twins, right, is just something that's possible in a way now that it wasn't possible in the past. And so, you know, there have been fits and starts for things as complex as the International Space Station to be a little bit more of a digital twin in terms of where is everything located. Maybe it doesn't look like the space station physically, but it allows you to find where in any cubby hole on the International Space Station some specific item has actually been stored. And that element has been out there. We just have reached a new paradigm where those tools that we use to develop can directly be translated into kind of the monitoring system, if you will, for kind of those systems once they're in operations, once they're kind of living out their lives on their way to, you know, ultimately a replacement system. And so that digital twin aspect is one that, you know, on our side we think about heavily and also how do we reflect it back on systems that already exist and bring them into a digital age because they're going to have to interact with other systems that we have digital models for. And for both of those systems to be most effective, we're going to need those tools and their availability.

Yeah. Yeah. And like third parties into that environment, right? There's going to have to be a common way to speak, you know? But yeah, love it.

You know, from my experience, you know, it's interesting having kind of grown up kind of in a test pilot community and kind of being trained that pilot comments and ratings were kind of the most important data that you could possibly collect from any test campaign that you laid out. There's an element of truth to that. But it could be for a couple of reasons, like you pointed out. One is that that early input can help you get the right design and get the feedback to make it true as quickly as possible, but it can also be the most important data because of the implications that it can have and if it's not handled correctly, you know, what can happen with a program that doesn't manage it appropriately. And so, you touched a little bit on this in one of your previous answers, Matt, and I wanted to kind of bring us back to kind of leadership setting expectations that there will be parts of the process where there are feedback and there will be parts of the process where kind of the team is left alone to kind of get through the design cycle or the sprint that they're currently executing, right? So laying out a program that does provide that opportunity but doesn't burden it with like explaining itself continuously kind of throughout the lifecycle, making sure that it's punctuated, that it's deliberate, that there's specific data that's expected out of specific milestones and having leadership set that expectation that, hey, let's get that data, let's act on it, and then let's delay collecting additional set of data potentially so that we can make the right sort of progress. And so it has to be metered because that most important comment, for the pro and the con of what the implication of kind of that user feedback is.

I wanted to touch on is the importance of the user community recognizing their role in that process. And it's not just that their comments were the most important, it's that in order to make the system ultimately successful, they have a role of providing feedback and then letting the engineering teams or the groups that are designing the overall system have a chance to innovate, right? Like their job is not to come in and tell you the way that it's been done all the other times that they've seen it. It is to provide the right sort of feedback so that the innovators can take that ball and kind of move forward. You know, I took a course several years ago when I was considering what I was going to do after I was an operator. And one of the key elements from an innovation piece was exactly that line in terms of defining a problem statement, but not necessarily defining the solution, right? And so the operators really are the place that you go for the information about the problem statement, not necessarily the folks that you want kind of delivering the solution for you, right? And navigating that is something that, you know, we think we solve it, but it's really kind of an active kind of leadership and management effort in order to make sure that that's what's happening by those opportunities.