X-31: Breaking the Chain: Lessons Learned

NASA 1, we have an ejection, we have an ejection. The aircraft is descending over the north base area. I have a chute. We had a highly competent team, very experienced, many flights under their belt. We had a number of pilots that flew the airplane. The pilot in particular that was flying that day had been on the program from the very beginning, highly experienced with the X-31. Each mishap has its own set of circumstances and its own sequence of events. But you find similar issues, communications, complacency, assumptions that haven't been warranted, human frailties. And you have to account for these things in a program. This is like a chain. You make a chain when you have any of these accidents, a chain of events. Any link of the chain, if it were broken, you would not have an accident. This was the A team. We had the best people from every discipline, from every organization, and we lost an airplane. So if it can happen to the best team, it can happen to any team. The X-31 research effort began in the late 1980s as an international program involving DARPA, the U.S. Navy, Deutsche Aerospace, the German Federal Ministry of Defense, and Rockwell International. The program's goal was to explore the tactical utility of a thrust-vectored aircraft with advanced flight control systems, using an aircraft designed and built specifically for that task. The X-31 was a real pioneering program. In fact, the X-31 program pretty much wrote the book on thrust vectoring, along with its sister program, the F-18 HARV. The initial X-31 flight tests were conducted at Rockwell's facility in Palmdale, California. But in 1992, NASA and the U.S. Air Force joined the X-31 research team, and the test flight program was moved to the Dryden Flight Research Center on Edwards Air Force Base. And before too long, the X-31 was turning in some extremely impressive results. By any measure, the X-31 was a highly successful program. It regularly flew several flights a day, accumulating over 550 flights during the course of the program, with a superlative safety record. And yet, on the 19th of January, 1995, on the very last scheduled flight of the X-31's ship number one, disaster struck. This particular flight had been on the books for some time to get done, and it was, by our standards, an absolutely routine flight. We were not expanding the envelope. We were not trying anything new. We were flying a new pitot-static tube, but this was in a routine mission, a routine task, routine flight environment, with an experienced pilot and experienced crew. But while the flight was routine, there had been some changes to the configuration of the X-31 since its initial flights. In particular, the original pitot tube, which supplies airspeed information to the plane's flight control computers, had been replaced with another kind of pitot tube, known as a keel probe. The keel probe gave more accurate airspeed data at high angles of attack, but it was more vulnerable to icing, especially since the keel probe on the X-31 did not have any pitot heat. We were never to fly the airplane in ice. That was a prohibited maneuver. So if you're prohibited from flying in ice, you don't need a heater. Normally, the conditions at Edwards are warm and dry enough that icing or pitot heat isn't a concern. But January 19, 1995, was not a normal day. The unusual part of the day was we had a high humidity at altitude actually conducive for freezing conditions. And the airplane was operated for in and out of some fairly high moisture content for extended periods of time. Led to some indications in the cockpit and the control room that it was causing problems with the air data system. This particular airplane had a limit to not fly through clouds, through visible moisture. That day, we were flying very close to and occasionally in and out of very thin cirrus cloud. It didn't particularly worry me because everything seemed to be going along quite normally. But some minutes, like five, before the airplane went out of control and the pilot jumped out, the pilot observed that there was some moisture around where he was. So he turned the pitot heat switch on. Now, clearly, when he turned the pitot heat switch on, he expected that the pitot heat would be working. About two and a half minutes later, which is two and a half minutes before the accident, he mentioned that fact to the control room. Okay, remind me I just put the pitot heat on. Remind me to put it off. Copy that. Are you ready? The pitot heat's not hooked up on a kill probe. Copy that. You got an enabler on? I'm able to step it on. Mysteriously, to this day, the control room gave him no response. They had an internal discussion as time, the clock, clicked down. And internally, it was commented that the pitot heat was not hooked up. But this vital piece of information was not relayed to the pilot for more than two minutes. And even when it was, the information was not stated as clearly or strongly as it could have been. And the pitot heat? Well, leave it on for a moment. Yeah, we think it may not. It may not be hooked up. That's good. I like this. We had side discussions that should have been going on on the intercom so that everybody in the control room was part of the conversation. Instead, we pulled our headsets aside so that we could talk to each other because we were sitting adjacent to each other. And that's another part of just control room discipline that we broke down on. Meanwhile, the first signs of trouble were beginning to appear. So now the pilot sees an anomaly in his airspeed. It's a 20 degrees angle of attack. And he can see that. And he says to the ground, and I've briefed this many times, he said, I'm at 277. I mean 207 knots. And the airspeed is off. I'm reading 277 knots at 28. Okay, pitch doublet. Well, anybody that's been on the program, and lots of people have been on many years, would know that 20 degrees angle of attack is somewhere around 135 knots, 140 knots. It's not 207 knots. Apparently, no one in the control room caught the possible significance of that discrepancy. And perhaps even more importantly, neither did the chase pilot, for the simple reason that he couldn't hear any of the pilot's transmissions. We had a mechanism of hot mic, very important to the pilot in the X-31 that he'd be able to talk to the control room without having to press buttons at certain key times, especially at high angle of attack, which was not going to be a factor in this flight, because he was going to go to about 20 degrees angle of attack. But it was a general operating procedure that was compounded because our hot mic system didn't work always very well. And when it didn't work, it put a lot of static in the earphones of the chase pilot who wanted to hear the hot mic to know what's going on. So it was the one-sided nature of the communication that kept me from having the situational awareness to be able to step in and say, hey, I'm reading X knots, and you guys are reading Y knots, and these two numbers should be the same, and they're not. The X-31 did indeed have an air data problem. The unheated keel probe had frozen over in the cool, moist conditions, causing it to start giving incorrect airspeed information to the X-31's flight control computers. In terms of the accepted risk, the failure of the pitot-static system or damage to it was well known. It was well understood. The pilot himself had simulated the failure in simulations before we even got the airplane. And it probably helped him understand that he had to get out of the airplane because the time is short when the airplane is diverging. And we went through quite a thorough review of the hazards that we knew or could come up with based upon the design of the flight control system. And we thought we had a good handle on that. We thought we could lose the whole nose boom. We could take a bird strike, wipe out the whole nose boom, fly home safe. As a result of that, we thought we had a pretty robust system. The reason the team thought they had a robust system was the X-31's flight control system was designed with three backup reversionary modes the pilot could select in the event of an air data problem or other system's failures. So in the case of if you saw something that was not right or the control room saw something that was not right with respect to the airspeed system, they could tell the pilot to go to R3. R3 was a reversionary mode that would have removed within two seconds the airspeed data inputs into the flight control system. The control surface response to pilot inputs would then be independent of airspeed, allowing the airplane to remain controllable for the remainder of the flight back to the landing. The accepted risk was probably reasonable, but here's the kicker. The consequences of a failure are so high here that you really needed to put some special attention on this. The designer did by putting R3 in. But nobody on the test team, including the pilot, realized that the X-31 was experiencing an air data problem that would require implementing the R3 reversionary system. For several minutes, we had indications that the airspeed was becoming poor, both in the cockpit and the control room. In our last-ditch catch, nobody stood up and yelled, wait a minute, this can't be right. Because had we realized what was going on, the control system had the ability to go to fixed flight control gains and with fixed flight control gains, it would not have been a problem. They would have been able to land the airplane safely. But we just never got enough information to make the decision to do that. We had an alternate airspeed indicator that used a different pitot tube, which would be less susceptible to icing than this special tube. It was at the pilot's right-hand knee, and he never looked at it. We had a lack of attention to the reversionary modes. Gradually, we were not thinking. We learned to depend on the control room. They're going to tell us when we need to go to R2 or R1 or R3. We need to know as pilots, which we kind of forgot, where are the safety nets? The safety nets push the right button. Didn't get the test data, but you bring the aircraft back. So if he didn't understand what was happening, we should have been constantly reminded, push the button and talk about it. The pilot obviously wasn't concerned. He was experienced. Probably if you look at the control room, the pilot, and everybody involved in that day's activity, he was the most experienced X-31 in that day's activity. He had been on the program since Palmdale. So he noticed something, but he wasn't concerned. He didn't ask for help that I was aware of. So I think the control room said, well, he's not that panicked. I'm not that panicked. And I think that fed off each other a little bit. The team moved on to the final test point of the day, a simple automatic control response test that required only a command from the pilot to initiate. But once again, the airplane did not respond as expected. He hits the box, presses the button, and he says, I don't get anything. Well, he didn't get anything because the box was designed not to put any input if you went beyond a certain speed, like 200 knots. So it was seeing the false airspeed of 200-plus knots. And when he pressed the button, it didn't work. 3, 2, 1, go. Doesn't do anything. But it didn't work because something was wrong. And the control room came back and finally just kind of ignored that and said, it's all okay, RTB now. It's almost like expecting to hear that went fine. You know, after this program of hundreds of flights and everything going perfectly, in your mind you're hearing things that weren't happening. Everything's fine, worked fine, let's come home. The normal operation of the system was expected that the system would identify the problems itself, that it would not be the people on the ground identifying an air data problem and calling for fixed gains. Although it was certainly capable of putting that, the expectation would be that the system would do its own self-diagnosis and identify failures. But the failure we had was a slow failure of the tube, slowly building the ISO. So the changes in the speed were within perfectly reasonable numbers for a real airplane. Software is just not capable of detecting that failure for that system. There was one or two people that actually knew that there was these little tiny areas that, yeah, it couldn't handle it. But that word never got out. They never stood up and said, oh, boss, that's not quite right. You know, you can handle it over 95% or 99% of the area, but there's really a couple little areas the automated system can't handle it. And that didn't come out until after the accident. I never did get to talk to them about it, but I just kind of felt they didn't want to stop the program, thought it was no real issue because of the difficulty of getting to such a small area of the envelope. But as the X-31 began to descend on its return to base, the problems caused by the failure of its air data system became far more pronounced. We have frozen the pitot tube now, and it's stuck. It's got what it had in it, and it's going to hold that. That pressure. Now when you start down with a frozen pitot tube, the airspeed, what you see, the false airspeed that he saw, will decrease as he decreases altitude. But we are seeing, we, the control room is seeing, they have a big display, this big. The pilot is seeing every time he turns his head, he's seeing the airspeed in the HUD. And now it's, perhaps at one point, it's at 150 knots. It cannot be at 150 knots. And then it's at 100 knots. And it cannot be at 100 knots. And going on down, it finally, right just before the accident, it gets to 48 knots, which is the minimum it's going to read. But the control system in the airplane is getting this wrong information and this is a complex closed-loop system. And when you put too much gain in, it will start to get unstable and it will start moving the controls, which it did in a matter of seconds. And finally, it dramatically pitches up. The pilot, of course, tries to prevent that. And I'm sure the instant that he hit the forward stop and realized he was out of control, he did the natural thing, was eject from the airplane. We were RTB, return to base. And I started to rejoin on the X-31. As I came up on his right side, about 100 yards away and closing, I saw the airplane start to go into a small wing rock that progressively got larger and larger. And as I got within about 200 feet of him, the airplane pitched up vertical. And approximately the time that I passed to beam him, I saw the pilot eject. Okay, NASA 1, we have 584 has ejected. The aircraft is descending over the north base area. I have a chute. Report, NASA 850. How do you read? 850, say it again, please. Yes, sir. NASA 584 has ejected from his aircraft. The aircraft is descending north of north base. The pilot is in the chute at this time, descending. So there was the knowledge and training in the simulation I taught the pilot to, that when he started seeing the airplane was oscillating, was not controlled, he knew that he had to get out of the airplane very fast or else the airplane would go into a tumble. And he did do that, and that saved his life. I also know that the pilot, as he was ejecting from the airplane, had thoughts of maybe I should have tried a reversionary mode. But at that point, if he would have hesitated any longer, he would have been probably lost with the airplane. I did not connect until after the plane departed. While the plane was tumbling, I made the connection. The pitot system had to be frozen and just didn't come to the realization soon enough to do anything about it in the control room. Less than four minutes after the first comment about pitot heat was recorded between the pilot and the control room, the X-31 crashed just north of Edwards Air Force Base. How could such a routine operation have ended in disaster when flights with far higher risk had been completed safely? And more importantly, what can we learn from the answers to that question? Every person involved in an experimental flight research program should actually study the mishaps of all experimental aircraft in the past 20 to 30 years. There's a lot of things you can learn because human nature doesn't change, the processes don't change. It's always the same set of contributing factors, just the names and the details change. Of the ten things, for example, that I describe as causes, contributing causes of the mishap, six of them occurred prior to the day of flight. Four occurred within about two minutes. So we had a better chance of working on the six than we did on the four. In some senses, the X-31 accident started six years earlier when the plane was first developed and tested at Rockwell. We had a hazard analysis from initial design, and in the accident, that had to actually get dusted off. You should never have to dust off one of those. Everybody familiar with the program, all those levels need to have a really good, comfortable feeling on what those hazards are and what is accepted in the risk. There was a redo of that analysis as we moved to NASA in 1992, and I think it was clear after the accident not everybody really understood what that design was, the detail you needed to to understand the full risk of the program. Clearly, from 1990 to 1995, you have a large team turnover. We changed locations. We expanded the objectives of the program. And as time rolls on and the new people come in, not everybody has the same understanding or appreciation of the kind of vehicle we're operating. It's a special airplane. It's not the same risk as any other airplane, and to operate it every day, you really ought to have the same appreciation for the risk. And I don't think we as a team did a good job of keeping everybody that came to the program with the same level of understanding of both the design and the risk on the airplane. We shouldn't have had a control room, a pilot, and a team that day that didn't understand that fundamental fact. And it's not elaborate. It's just straightforward. The airspeed I see in the HUD is the airspeed the computer uses. The airspeed I see has got a problem. The airplane's got a problem. And that fact, it didn't get communicated correctly from old team members to the new team members. And if it had have, I don't think there would have been anybody in that room that wouldn't have yelled, stop, and jumped off the bridge to make it happen. There were errors made. And the pitot heat circuit breaker was disabled. But there was no placard in the cockpit to say no pitot heat. And notices of the configuration were sent around. But here also, we probably lacked one step, and that is to know that everybody got the message. It's one thing to send it out. It's another thing to verify everyone has read and understood it. And so that procedure was changed, by the way, so that people ripped off the bottom of the page and sent it back. I've seen it. Ironically, the X-31 program also may have been a victim of its own success. I never saw complacency in this team. I went to tech briefs, crew briefs. It was treated very professionally. And in fact, to some extent, it was treated like an experimental airplane every flight. But certainly you have to think that after hundreds of flights, excellent results, and the fact that none of these hazards, these terrible things that you predict could happen has ever happened, it could lead you to be less sensitive to things that are happening. Maybe just a little bit of the edge comes off. Those single-point failures were identified, and we made some actual changes to the design of the airplane to account for that. Again, that was 1989. Why all those were there and what the concerns were and how to mitigate them and how to worry about them became, we hadn't had any problems with that for five years. And I think, again, the complacency just got built into the team. It worked fine. We never had a problem. And those little hairs on the back of your neck weren't geared to stand up when people started having airspeed problems. Our control rooms used to have a saying on them to prepare for the unexpected and expect to be unprepared, and I think that's a truth in the flight test business that we need to keep that in mind continuously. I wish that sign was still up there because that reminder needs to be enforced all the time. Well, certainly in the case of the X-31, we were returning to base after two exhausting days, seven flights. Ship one was now going into the boneyard, or at least it was being retired from the test program. And so we're finally finished. Is everybody paying attention like they should be? Obviously not. And while the X-31 program flights were highly successful, they did not include an element that might have helped prime the program team to take the one mitigating action that could have brought the X-31 home safely. We've debated amongst ourselves whether we actually would have been able to convince anybody to use the fixed gain system because there was not an obvious need for it. The pilot may have been better prepared when things started to go awry to select fixed gains, but I don't know if we ever really would have done it in that situation because we didn't have a real problem. We did have a real problem, but it hadn't been diagnosed as a real problem. On the previous program, the X-29 program, we had the same sort of thing. We had an analog reversion mode, a digital reversion mode, and the normal mode of the airplane. We routinely at every test point selected those backup modes, flew them around, so the pilots were much more familiar and much more comfortable with selecting those modes. On the X-31 program, we never selected those modes intentionally. We only used them when we had a sensor failure or the system told us to select those modes. On the day of the mishap itself, there were additional links added to the chain. There were unusual weather conditions that created an uncommon and unexpected kind of flight hazard, and the team was working with a flawed hot mic system that kept the chase pilot from hearing critical communications from the X-31 pilot. So, some links in the chain are already built there. Management links, control room is now talked internally. They've heard some things. They haven't said anything. Some more links are built. We've got this chain that's building now. The chase pilot didn't hear anything about this. He didn't know anything was wrong with the airplane until he saw the airplane pitch up and the pilot jump out. Whereas he could have stopped this any time. At any rate, it's a total team concept, and the chase pilot has to be part of that team. The team has to have total communication, so the use of a hot microphone frequency that did not allow the chase pilot to stay up with what was going on with the airplane was essentially keeping me from doing my job, at least at a certain level. And that's one of the things that we changed in the way we do business here at Dryden, is to allow the chase pilots either access to the hot mic or to ensure that all critical communications are transmitted so that all the players are kept up to speed with what's going on. And that was a direct fallout of how the X-31 operation was handled that day. If one or more of these contributing factors had been caught and addressed prior to January 19, the chain of events leading up to the accident might have been broken before the flight even took place. Yet there were still opportunities to avoid the mishap, even in the last few minutes of the X-31's flight. So why didn't the team manage to recognize, communicate, and respond to the X-31's pattern of anomalies in time? So we were seeing inconsistencies between the data from the aircraft system and what we knew of the physics of the problem, that it could not be, that you could not have that airspeed and that angle of attack simultaneously. And for me, I just remember thinking, gosh, I can't wait until we get the data from this flight because I want to see what's going on. I knew there was an anomaly. We had talked about it between the engineers. We didn't talk about it on the intercom, though. It was sidebar conversations in the control room. Well, many of us are engineers, and we see an issue, oh, this is interesting. I don't know what's causing that. And you start thinking about it and trying to figure out what is the answer. In the meantime, the seconds are clicking by. And really, the right response is, something's going on I don't understand. Let's call a halt here and let's just figure it out. We should have, at the first call of an airspeed failure, puckered up. Whether you're RTB at that point or not, it wouldn't have changed. The kind of failure that was occurring should have triggered a lot of emotion anywhere in the flight envelope. In the case of any discrepancy, anything that doesn't sound right, feel right, smell right, let's stop and think it over. And I think that kind of attitude has been built in now into the control room, mission control room processes since then. We were flying lots of flights. At the peak of the program, there would be days when there would be five flight days. I think on that particular day, we were only doing three flights. And it was the last flight of the day. It was the last flight for the first airplane. And we had completed all the test points for that mission. In addition, we were going through the RTB or return to base checklist. And at that point, every one of us kind of relaxed. Like I said, what was going through my mind is, I can't wait to get this data. Something funny is going on and I want to figure it out. And that's another lesson learned that, you know, when we talk about it all the time, the mission's not over until the airplane's on the ground and the engine's shut down. And you see it a lot in the control rooms. We start getting ready to land and everybody relaxes a little bit. And that's a lesson I've carried with me is you need to continue the vigilance there in the flight. Communication is what it's all about. So everybody, we have to have the communication links. We didn't have it to the chase. Hot mic was a contributing factor. We didn't have it in the control room. We discussed things internally. It was not transmitted to the pilot. We have to have an environment built where people can speak up when they think something's wrong. They don't have to be right. If they're concerned, they should be able to speak their mind, put their hand up, and we stop the train. And then we say, no, you weren't right. It's okay. Fine, we go on. We didn't do that. We never stopped the train. We had a problem and we didn't stop not only testing. We didn't stop flying and come home. But you can't stop for every problem. I mean, that's unrealistic. You have problems in flight. The combination that went with that is we didn't understand the severity of the problem. So you have to understand your vehicle and the consequences of failures. And if one of those failures has a serious consequence, you need to stop and come home. Clearly, there are lessons to be learned in the entire progression of events that led up to the X-31 mishap. And yet, the X-31 program did not end with that crash. The next chapter of its story is an equally important reminder of why flight tests remain such a valuable step in proving a concept or technology, despite the hazards that come with the territory. The X-31 had been scheduled to fly at the Paris Air Show in June of 1995. But after the loss of one of the two X-31 ships less than six months before the show, it seemed an impossible goal. Having lost the airplane, pretty much everyone thought that's it. Because flying the kind of maneuvers that this airplane can do at 500 feet sounded a lot riskier to me after you lose an airplane. The team really talked a lot about this and decided that it did not want to end this program on a low note. And so, we made the decision to press on with the Paris Air Show. A huge thing to sign up for was to take an airplane that just crashed and to turn it around to go do a low-altitude, high-angle attack flight demonstration. That took a lot of guts on everybody's part and a lot of good engineering work to make that happen. We actually flew the X-31 84 days after the mishap. This required the board to reach its conclusions, to write a report for the team to react to all of the issues and problems and contributing factors brought up, solve the problem, and get it into an airplane and get it qualified for first flight. It was all done in 84 days. It does tell you about the quality of the team. Totally different airplane with a most remarkable X-31 technology demonstrator. You know, after the mishap, I think the program made a spectacular recovery and made one of the finest appearances ever at the Paris Air Show. The airplane did things that no other airplane could do. The Russians had demonstrated post-stall maneuvers with the Cobra, but it was really an open-loop maneuver. They pulled back on the stick and then you flew out of it at the end, whereas the X-31 just demonstrated the ability to control all axes of the airplane, pitch, roll, and yaw simultaneously while operating at the extremes of the flight envelope. So, fantastic air show. Absolutely the most spectacular I've ever seen, and I saw every one of them. And I stood with the crowd on some of them, and I was in the control tower on others, and out right underneath it at other times. But to be with the crowd and watch even hardened veterans, military, had no concept of what it could really do, and seeing it was jaw-dropping for the crowd. It was spectacular. The announcement that the X-31 was next to fly, as you look down the row of chalets, you see all the people coming out of the chalets, out against the railing to watch the flight. If the events leading up to the X-31's mishap are a reminder of how much vigilance is required in order to mitigate the risks inherent in a flight test program, the X-31's Paris air show performance was a reminder of why those risks are still worth undertaking. Flight tests of all kinds is inherently dangerous. There are risks involved in it. Never can you or anybody else bring it to zero. Well, you can. That's keep the airplane in the hangar. Don't fly. Well, if you don't fly, you don't move forward. You don't discover. You don't prove things. So you need to take some risks, but you need to do it in a controlled fashion. The reason we spend time on looking at these accidents is that there aren't many accidents. We don't lose many airplanes in flight research activities at Dryden. We haven't over the years. And so when you do have one, you better learn everything about it. In fact, you should do the same thing for close calls. The lessons to be learned. Don't assume that they've been learned. We can always, at every new group, we'll have to learn the same lessons. And you don't want to do it the hard way with an accident. Safety is everybody's business. Flight test safety is everybody's business on the team. And there are no processes. You have to have processes. But there are no perfect processes that will not require good judgment from all levels of the program. If you're a program that's been operating for a long time, potentially, you've got a lot of turnover. You're in your mature years. All your documentation is years old. Maybe you better make sure that all your new people are as good as your old people. That you've reviewed your documentation and it's still correct and you all understand it. And that what you're doing today still makes sense from how you started. So maybe one of those, if you're in that area, you ought to take a look at yourself. It always is clear what you should do after the fact, or should have done, rather. And nobody thinks it's ever going to happen to them. To lose judgment. To lose this communication link. To not do the right things. So what is the message? What is the message for the team? It may mean that I am a part of the chain. And that if I don't catch this, and if other people don't catch their mistakes, we will run through the entire chain and lead to a mishap. So it means that every individual in the program, from beginning to end, no matter what the job is, from the highest level job to the lowest level job, in terms of detail, they have to take it very seriously. And that's a message that you have to keep promoting, pronouncing, and explaining. It sounds trite, but everybody is responsible for safety. If you think some safety office analysis is going to find these things, they won't. Mishaps can occur everywhere. But the point is, you have to fly safely, but fly.