Safety Board Asks for Better Stats

 - January 9, 2008, 8:57 AM

The NTSB has called for improvements in the way the Transportation Department collects data, including the FAA’s Accident/Incident Data System (AIDS), the Near Midair Collision System (NMACS) and NASA’s Aviation Safety Reporting System (ASRS).

The Safety Board said it studied transportation safety databases to evaluate data quality issues and to encourage improvements in this area because poor data quality often hampers its ability to analyze important safety issues. According to the Board, the enhanced figures will allow it to better monitor accident risks, support the analysis of risk factors and evaluate the effectiveness of accident-prevention strategies.

The conclusion was part of an NTSB safety study examining data quality issues within all of the government’s transportation agencies. The study also determined that the Bureau of Transportation Statistics’ audits of DOT safety databases should be accelerated to support timely, coordinated reengineering efforts by the modal agencies.

The Safety Board initiated the study to highlight data quality issues within the external databases it uses when performing accident investigations, safety studies and special investigations. Since 1968 the NTSB has issued 233 data recommendations asking various organizations to develop, modify, improve, address underreporting or analyze existing data within the respective database.

Although federal, state and local government agencies all use transportation safety databases to monitor transportation accidents and to develop programs for improving safety, 83 percent of the NTSB data recommendations were issued to federal agencies.

While the Safety Board relies on many external databases when performing accident investigations, safety studies and special investigations, most of these databases are sponsored and operated by the modal administrations of the DOT.

The four objectives of the study were to highlight the value and potential uses of transportation safety data; describe some accident and incident databases commonly used by the Board; summarize past Board recommendations involving transportation data; and evaluate Bureau of Transportation Statistics (BTS) efforts to establish data quality standards, identify information gaps and ensure compatibility among the safety data systems maintained by the DOT.

According to the NTSB, the FAA’s Accident/Incident Data System contains “incidents” only, and it uses the Safety Board’s own Aviation Accident Database as the primary source for accident information. Because of its broad reporting criteria, the AIDS database contains many different kinds of incident ranging from airport events involving collisions between aircraft and catering trucks to the loss of a cabin door in flight.

Near-midair collisions are handled through separate reporting procedures and are contained in a separate database, the Near Midair Collision System. Operators, airport personnel and air traffic controllers inform FAA inspectors of aviation incidents, and the inspectors investigate the incidents and submit data on standardized forms. Thousands of incidents are entered into the AIDS database each year, but the FAA does not publish an annual statistical summary of the data, though the information can be searched via the Internet.

The Aviation Safety Reporting System, which is managed by NASA Ames Research Center, is a confidential reporting system containing operator-submitted narratives describing events or conditions related to the safety of flight. Individuals who submit reports are granted immunity, for the most part, from enforcement action on the part of the FAA.

ASRS is designed primarily as an early warning system for the FAA and for participants in the National Airspace System. Pilots, air traffic controllers, flight attendants, mechanics and ground personnel are encouraged to submit reports after witnessing any event during which they feel aviation safety has been compromised, with the information used to detect and correct unsafe conditions.

Because the reports are mostly narrative, however, statistical analysis is difficult. And because of resource limitations, ASRS staff store only a fraction of the 30,000 reports submitted each year. A representative sample of reports–approximately 10 percent of the total–is entered in the database.

Analysts examine the remaining reports and retain any they believe describe significant issues. These reports are coded differently from the randomly selected cases, so the two types of case can be distinguished. In total, about one-third of the ASRS submissions are retained. Although the FAA’s Office of System Safety provides the capability to search ASRS database records online, neither the FAA nor NASA publishes an annual statistical summary of ASRS reports.

As a result of the NTSB safety study of data quality, the Board recommended that the Bureau of Transportation Statistics develop a long-term program to improve the collection of data describing exposure to transportation risk in the U.S.