Hello, I'm your host Grayson Brulte. Welcome to another episode of SAE Tomorrow Today, a show about emerging technology with the leaders and innovators who make it all happen. On today's episode, we're actually honored to welcome Craig McMillan, MGSE Growth Leader, GE Aerospace. On today's episode, we'll discuss how the industry-level forces are shaping the future of aerospace engineering, from safety standards to the emerging challenges of AI-enabled flight. We hope you enjoy this episode. Craig, welcome to the podcast.
Thanks, Grayson. I appreciate it. Glad to be here.
It's great to have you here. I'll just say this, little kid, aerospace is cool. And there's a lot of moving parts that go into aerospace, but behind all the coolness is the kid in me, there's a lot of complexity, a lot of moving parts in aerospace. So, the big question, Craig, is what role do standards play in enabling the complexities to turn into realities, to allow this industry to scale?
Well, I think it does two things. The first thing is it gives a common reference for different companies. So, where there are multiple companies, Boeing, Airbus, for engine manufacturers, Pratt & Whitney, Rolls-Royce, GE Aerospace, we all make common products. And having these standards allows us to talk in a common language. So, when we demonstrate to the regulatory authorities like the FAA and EASA, we can talk to them in a common language. So, the regulators don't have to learn the language from a bunch of different customers or from a bunch of different applicants. And then the second thing it provides is it provides a roadmap for handling the complexity because our systems are getting increasingly complex and we've, by having a standard, you're giving a roadmap of the minimum acceptable criteria that everybody can follow to ensure that you deliver safe products because as cool as aerospace is, safety always has to be at the forefront of everything we do and these standards give a minimum bar to meet from a system safety perspective and that's critical to ensuring that we have a safe and usable airspace system.
Where would we be? I like the term that you use a lot, common language. Would we have this manufacturer going this way, this jet engine company going this way, regulators saying what, where, how, why? Is that what would be going on if there wasn't a common language?
Yeah, yeah, because each of the applicants would try to provide a package using their own terminology and then the regulator would have to speak numerous different languages in order to understand what it is they were doing. So, by providing a common reference, the regulators are more able to assess the compliance to the safety standards in a way that makes sense to them. And so, standards like ARP 4754 — well, excuse me, ARP 4754 is not a standard, it's a recommended practice, but it gets invoked by advisory circulars and such by the regulatory agencies.
So, where would we be without ARP 4754? Where would we be as an industry?
Well, we know where we'd be because we were there 40 years ago, or 30 years ago. So, what that particular document shows is how to meet AC 25.1309, and that is the regulatory rule that we have to meet to show that we have a safe system and it doesn't do anything it's not supposed to do. For years with simple analog systems, applicants were having trouble deterministically showing that they met a minimum bar of system safety and as the systems got more complex, it became unmanageable. So, what happened in the '90s is that we came out with this concept of development assurance. So, when you cannot exhaustively verify a product, we developed a process by which you could show with a high level of confidence that a product was safe in accordance to 25.1309. So, ARP 4754, now we're up to revision B, provides a roadmap to how you can show compliance to FAR 25.1309.
And sitting here as a non-aerospace engineer, how would you describe that in simple terms to a listener or the member of the public that's, okay, yes, you're all about safety, but how would you put that into simple terms so they can grasp?
Well, for instance, you break it down in terms of function. So, an engine provides thrust. And you look at what an engine can do, there could be two forms of failures. One would be a loss of function, so you lose an engine. And so, what development assurance says, okay, if I lose an engine, what is the impact on my aircraft and how would I mitigate that? In most cases, you would have second, third, or fourth engines, and you would continue to fly on the remaining engines. A second type of error that we try to mitigate is an undetected erroneous data. So, let's say in your airspeed, you're flying at 150 knots, but your airspeed indicator shows that you're flying at 180 knots. It's not annunciating any sort of failure, but it's giving you incorrect information, but you the pilot don't know that. And so, we developed a development assurance process to show that even when that sort of undetected error occurs, you would have a means of detecting that error and taking the appropriate action. In case you would have two other airspeed indicators, and when two airspeed indicators are showing 150 knots and the third one is showing 180 knots, you could reasonably deduce that the third one is the erroneous indication and go by the other two. So, we provided a mechanism for system safety to determine when these sort of known and unknown errors occur and how to properly mitigate and respond to them.
Does that change when you're looking at software versus hardware or are they all interconnected?
So, system safety applies to all the above, but when you get into system software, it gets more complex exponentially faster because with a flight management system for instance, it has hundreds of thousands of lines of code in it. The amount of things that can go wrong in hundreds of thousands of lines of code is obviously a very large number and so DO-178C was developed to provide development assurance for software which feeds into ARP 4754B. And so, you provide this layered approach where you have development assurance at the system level and then you would do development assurance at your hardware and software levels, but it all ties together. Development assurance for aero-mechanical systems like a flap, a spoiler, something like that is a little bit more straightforward because you don't have the complexity that you would with software, but you still have the important safety-critical functions that you would need to evaluate.
And when you look at evaluation, where do environmental risks such as radiation or unknown environmental things that could happen, where does that come into the risk analysis?
So, as part of the development assurance process in the ARP 4754B, we have the safety assessment process defined in ARP 4761A. And that basically is a sibling document and that's where we get into some of those types of common mode errors like bird ingestion for instance, where you would fly through a flock of birds and lose both engines like the US Air incident on the Hudson River. Or you would look at radiation effects like single event effects from radiation or any sort of environmental factor like that, like heavy rain or lightning or things like that. So, that would be part of your risk assessment that you would do as part of your safety assessment.
As part of a risk assessment, does the industry share risks or edge cases that they've encountered with each other? Say, "Hey, we've experienced X, Y, and Z. This is how we resolved it. You might want to plan for that." Is there open communication on that safety across the industry?
Yeah, that's fairly typical because we all have an interest in safety. When there is an incident involving an aircraft, the general flying public isn't necessarily attuned to what aircraft manufacturer, what engine. It's just an aircraft accident. We all have a vested interest in having safety. System safety is never going to be a business differentiator. We all want to be safe. And so, there is a lot of collaboration. So, when we work on these committees to develop these documents, all of the companies that I mentioned previously are actively engaged working collaboratively together trying to make sure that we are all doing what's best for the industry from a system safety perspective.
Are those system safety — are they independent? Where I'm just asking this as an individual who flies, that a boss can't say, "Nope, you're going to do this." Are they fully independent where they have the say and they cannot be influenced either by an internal person or an external person and they're just purely focused on safety?
Well, that certainly is the intent. How that is implemented differs from company to company. What is typically recommended from the regulators is that you would have a safety management system. And that you would have independence of that safety management system where you could raise that issue. And so, how that is applied is unique to each company or applicant.
From a safety perspective, obviously we're very fortunate in this time of society. We have access to a lot of really great tools, simulation, digital engineering tools. When did those come into the picture from a safety perspective? When did those individuals on the safety teams and the engineers start to run different what-if scenarios for safety planning? When does that start in the development process?
Oh, pretty much from the beginning. When you even initially conceive of a product, you would typically address those types of major concerns to ensure that you don't go down the path of developing something that then is not sustainable from a safety perspective. So, those simulations would start on day one. Obviously, the initial simulations would be very high-level with low-level detail because think of it as a back-of-the-napkin calculation to make sure that what you're doing is appropriate. And as the design progresses, these simulations and analyses would become more and more refined and more detailed.
Is there a, let's say, industry-wide — not a standard, but a best practice where in the beginning for a certain period of time, simulation is done before it takes a step where you're trying to uncover things that you might have thought of and really kind of push the envelope and then at that point it graduates to the next step?
Typically, running simulations early helps find issues before you actually build hardware. And so, it's in your best interest to run those simulations to validate your design before you go build it. You still have the verification process after you build the product to ensure that it does what it's supposed to do and only what it's supposed to do, but the simulations give you a higher confidence. If you find an issue after you build the hardware, it's a very disruptive process to go fix it because you have to go back and change your design, then you have to rebuild the hardware, and then go back and re-verify it. Whereas, if you find something in simulation, you just fix it in the design before you build the hardware and before you ever tested it on a test bench. So, finding problems early has significant benefits on several fronts. And so, obviously, you want to run the highest fidelity simulation as early in the process as possible to give you the highest confidence that you're actually building the right product.
Have you seen, if just using industry averages, where a new product is in simulation for X amount of time? Is there like an industry average? Is it 8 months, 2 years that something sits in simulation before it graduates? Any data that you've seen?
It's all a function of the complexity of the product. And then the computational methods. So, it's ever-changing. I mean, back in the day, they designed airplanes completely with slide rules. And now we're using supercomputers and high-performance computers and all sorts of advanced computational techniques to develop these things, and it just seems like the systems become more complex in step with the increase in ability of computing power. So, the computing power can never keep up with the complexity of the system. So, we're always having to catch up to be able to analyze these systems. You know, think fly-by-wire and fully digital electronics flying the airplane, and things like that. Where back in the day where everything was a cable and pulley system, it was a little bit easier to analyze it, but you couldn't do as much with it. So, a fly-by-wire flight control system is more capable than a cable and pulley system, but a cable and pulley system is easier to analyze.
Do we ever get to a point where it's too complex?
I don't think it gets to the point where it's too complex. It would just take longer to analyze it and be able to provide you the confidence that it's safe because you simply just have more to analyze. And that's where digital engineering comes in, where if you have things represented in a digital form, then you would provide the means to be able to analyze it more thoroughly, more exhaustively using automated techniques. Because everything is interconnected in a digital representation. So, applying digital engineering with an increasing complexity is a very natural fit. Does that make sense?
It does. So, then on the back side of that is where does AI come into the equation and how is the industry approaching models? Are they using closed models, open-source models? And I ask that from a data leakage standpoint. There's been a lot of government documents, there's been a lot of public documents around data leakage. So, when AI enters the picture, how are the industry thinking about it from a models perspective?
The answer is in safety-critical systems in aviation, that's still an open question. So, AI — we're still at the early stages of applying AI for the development of safety-critical systems. As part of the SAE working group, we've started a working group to provide examples of how AI can be used in development of safety-critical systems using the development assurance techniques. The problem with AI is it's not deterministic. If you give it the same input five times, you could conceivably get five different answers. And so, we have this term called qualification of tools. DO-330, if you're familiar with the RTCA standard, gives a template for defining tools for software development. AI cannot be qualified because it won't give you the same answer. So, you have to apply the development assurance approach and say, "How can my AI introduce an error into my product and how would I identify and mitigate that error?" And so, that's what we're in the process of doing now. So, we're still very, very at the early stages. AI never going to replace an engineer. Or "never" may be a strong term, but because of the limitations of the determinism of AI and the lack thereof, the human is always going to be a critical element of the design process. Now, the AI can help the human go faster, provided that the human is cognizant of the problems that an AI can introduce in a product, and the human is using the AI as an aid and not as a replacement.
