First there was flight operational quality assurance (FOQA), which analyzes flight data to improve pilot performance, and now there is SOQA, which compares simulator-derived data with FOQA data. Modern simulators are now so good at mimicking the real world and recording simulated flight data that they can provide tremendously useful feedback during the training process, allowing comparison of simulator and flight data to develop evidence-based training tools. And simulator manufacturer and training provider CAE is tapping into this capability to offer SOQA evidence-based training tools as a new product.
CAE’s first SOQA implementation was for the U.S. Department of Defense. Concurrent Technologies selected CAE Flightscape and CAE USA in late 2009 to compare military FOQA and military full-flight simulator data in a SOQA program designed to prevent accidents and incidents. The program lasted 11 months using a Boeing 737 simulator at CAE SimuFlite in Dallas. The U.S. Air Force flies the C-40 variant of the 737.
“The results will be published shortly,” said Mike Poole, CAE Flightscape executive director of flight sciences and chief investigator. “And we’re in the final development implementation stages to launch this in the aviation industry.” Airlines and business aviation will be beneficiaries of SOQA as it moves into the commercial arena. “When you collect data on how you operate, you should look at it,” he said, referring to FOQA data. “With simulators, we’ve got data, so let’s look at it.”
The basic idea behind SOQA is that simulators are repositories of valuable information, as are the recorders on aircraft that log information used in FOQA programs. SOQA offers a quantitative bridge between a safety management system (or any safety program) and training, explained Lou Nemeth, chief flight safety officer for all of CAE’s flight training operations, including simulator-based and flight academy operations. Typically, he said, when a safety issue is raised or as he put it, a latent unsafe indicator is discovered, “90 percent of the time the mitigation strategy is ‘we need to do more training.’”
But how does anyone know whether the training prescribed to deal with that latent unsafe indicator is effective and ultimately improves safety? SOQA provides quantifiable training data that can be compared with FOQA data from actual operations, all of which helps safety officers evaluate the benefits of the training to fix the latent unsafe indicator.
A key element of both FOQA and SOQA is that the data gathered is analyzed for trends, and this can help identify areas where pilots are flying the simulator differently from the way they fly the airplane, according to Poole. Another advantage of SOQA is that the computer is better than an instructor at spotting pilot deviations, and allowing the computer to do its job frees up the instructor to focus on human factors and other hard-to-measure behaviors.
“SOQA will help the instructor detect things that in the past have gone undetected,” said Nemeth. A simulator instructor might not, for example, notice that a student nearly scrapes a wingtip during an engine-out takeoff. But the computer records this exceedance. And the data is used to drive an animation of the training event so the student and instructor can easily see that undetected event and figure out a way to prevent that.
“People are not good flight recorders,” said Poole. “Today the historical system is reliant on the instructor pilot. But they don’t pick up on all these things and also not always in the right time sequence. With SOQA it’s going to tell you ‘this is the way you did it,’ precisely. Or maybe you did the right procedure but not in the right order. It’s an objective measure that eliminates controversy. It also changes attitudes. People start to realize that the simplest of problems can be a link in a chain that can be catastrophic. You approach the aircraft differently.”
“With SOQA in the environment,” said Nemeth, “it alerts the instructor to the things that heretofore have gone undetected.” As another example, he pointed out, “If a pilot exceeded maneuvering speed in the simulator, it may very well have gone undetected.” And that kind of flying behavior can cause serious consequences in the real world. “It’s training based on evidence instead of training based on prescriptive regulations,” he said.
Of course, regulators will need to be on board as SOQA becomes more popular, and CAE is working with the FAA and other entities that, he said, “are gathering evidence to get the FAAs of the world to change the regulations to allow evidence-based training based on objective data.” The beauty of the SOQA system, he said, is that it allows all involved to see if the training is working, based on the trend analysis from training and operational data.
Poole noted another benefit of SOQA, the ability to create animations of accident events and replay them for pilots during training. CAE used an animation of a 737 nose-low, high-bank situation as a tool to poll instructors on how they would teach pilots to handle the airplane if that happened.
“This allows us to make a comprehensive video quickly. We posted it and are now starting to get feedback on what instructional technique is necessary to overcome any forthcoming problems,” said Nemeth.
The animations, Poole added, are much better teaching tools than having a pilot just read about an accident. “Their attitude is that ‘this wouldn’t happen to me because I’m a good pilot,’” he said. “But when they watch the animation, like any good movie, they see things they didn’t see before. It gives pilots a lot more insight into what happened. You’re going to get that learning during training. You have access to real events. They see that and hopefully they will recognize those symptoms right away.”
Using an animation of Airbus A300 uncommanded pitch-up issues helped pilots understand the event better, recognize it and correct it, Poole explained. “Without the animation, they didn’t recognize it [during training]. [When reading about the incident], we all take away different interpretations of those words. But the video is not ambiguous. It tells what happened in detail, in a highly intuitive manner. Then you start to look at why and how you would prevent it.”
As SOQA moves into the airline and business aviation fields, someone will have to pay for the service, including equipping simulators for the necessary data gathering and replay capability as well as analysis. Because CAE is offering SOQA services as its own product, it will use its own simulators. The use of SOQA will benefit both initial and recurrent training, according to Nemeth, and the desire for SOQA should come from safety officers who see the benefit of mitigating safety threats through use of SOQA data.
“There is a high level of interest,” said Poole. An Asian airline is interested in equipping its simulators for SOQA. For operators that don’t have their own simulators, CAE Flightscape will help manage the program with its simulators, acting as a virtual safety officer to analyze flight data in a FOQA system and develop a SOQA program.
“I think SOQA will be popular with airlines and business jet operators that see safety as a strongly held value in their organizations and believe in a systems approach versus a prescriptive regulatory approach,” Nemeth concluded. “This is the way forward. Prescriptive regulatory training is going to end.”