The AirSafe Journal - Issue 7
The AirSafe Journal
Issue 7 - 15 February 1997
Todd Curtis, PhD
Highlights - How to Deal With FAA Airline Safety Data
In response to recent high profile U.S. airline fatal events, the U.S. government has taken steps to make it easier for the airline consumer to find safety related airline information. In late January, the acting administrator of the Federal Aviation Administration, Linda Daschle announced a plan make a wide variety of information more available to the public. Among other things, the FAA promised the release of information on significant enforcement actions, accident and incident data, and background information on the roles and responsibilities of various parts of the airline industry. This issue of the AirSafe Journal, will explore the issue of the analysis of this data and will provide a number of basic tools for turning this raw data into useful customized information.
- Biases in Risk and Safety Information
- Asking Good Questions is the Key to Good Answers
- Categories of Typical Questions
Bias of the Source: While processed risk and safety related information is available from sources as diverse as national aviation authorities, aircraft manufacturers, and even this Web site, they all have some biases in what is reported and analyzed. For example, on this Web site, my focus is on plane crashes - events that cause the death of one or more passengers. I avoid the term accident because I include fatal events that were the result of deliberate actions such as hijackings and bombings. On the other hand, the FAA and NTSB focus their attention on accidents, a definition that would include events causing serious injuries or deaths to passengers and crew members but would exclude sabotage and hijackings. More specifically, the accident rates quoted by the NTSB and FAA exclude hijacking and sabotage events from the computation of the accident rate, but the deaths would be included in the overall death total for a particular time period.
Source of the Bias: The aviation risk and safety information presented by any source is created in response to a need or to answer a question. In my site, most of my information is created to identify the risk of passenger death in a particular context such as by aircraft or by airline. Any information provided by a government, an airline, or other aviation organization also has a purpose. The key question for anyone seeing that information is whether that information coincides with one's information needs. If it does coincide, the need will likely be satisfied. If it does not coincide, then one must take the next step.Back to Contents for this Issue
While the decision of the FAA to provide more information to the public is a good one, it does not go far enough. Whatever amount of raw or processed information the FAA eventually supplies will not answer every possible airline safety question. You are the best judge of what information you need to satisfy your personal information needs. If you are satisfied with accepting only that information that is given out or analyzed by someone else, then read no further. If you feel that you may want to take the FAA data and analyze it on your own, then read on.
If you have a particular question about airline safety, the most important thing to keep in mind is to first ask questions and then look for the data to support the answer. Asking yourself the questions ahead of time also makes it easier to decide if the data that is readily available will meet your needs. To do it the other way around by asking questions only after seeing what information is easily found is a lot like trying to find a set of keys lost on a dimly lit street. If you only look where there are street lights, you may never find the keys.Back to Contents for this Issue
The questions that one would commonly ask would fall into several different categories. Below are descriptions of some of these categories and the kind of information it takes to answer the question.
- Definition Question: A question that can be answered by referring to a specific authoritative source. A dictionary or an encyclopedia is usually not useful because the definition is not given in the same context.
Example Question: What is the NTSB definition of a serious injury? It is also legitimate to create your own definitions so long as you apply them consistently. On this site, fatal passenger event is one such definition.
- Reference Data Question: A question that can be answered by referring to some source of factual information.
Example Question: How many takeoffs were performed by United Airlines jets in 1984? Keep in mind that different sources may give slightly different answers. In cases like that it is best to rely on the most objective and authoritative source.
- Combination Question: A question where the answer is a combination of two or more definition and reference data questions.
Example Question: What percentage of plane crashes in the U.S. from 1985 to 1996 involved discount carriers? To answer this, you would need to first find out or create the definitions of fatal passenger event and discount carrier and find the number of plane crashes for both discount and non-discount carriers.
This kind of question can also be a simple reference data question if some authoritative source has already produced that information. One example is the NTSB generated accident rate statistics. If the data sources and the definitions used by the NTSB are satisfactory to you, then use that statistic rather than creating one on your own.
- Risk Questions: Risk is defined as a combination of a specific consequence and a probability. A question of the form "What is the chance of (some consequence) happening?" is a risk question. To answer it, you would have to define the consequence and find out the probability of that event happening.
Example Question: What is the chance that a jet airliner will have a fatal midair collision in the United States in the next five years? The easy part is defining the consequence because you can make it as general or as broad as you feel is appropriate.
The definition of the consequence will determine what information you need to define the chance part of the risk equation. In the example question, one way to predict the future chance of a midair collision is to look at the historical data to determine a midair collision rate and then to take that rate and apply it to the estimated number of flights during the next five years.
- Safety Questions: These questions are the most difficult to answer because safety is not a concept that lends itself to a simple definition. Some safety questions are actually risk questions in disguise. For example, about a year ago, the U.S. government announced a goal of zero accidents as a goal of the U.S. airline industry. The zero accident goal may be a safety goal, but it is also clearly a risk goal where the consequence is an accident and the chance of that consequence is zero. A true safety question deals with subjective questions and judgments that can't be objectively measured and that are different for each person. In that light, safety is very much like beauty in that there is no clear division between beauty and the lack of beauty and that two people can have entirely different standards for beauty and yet the standards would both be legitimate.
Example Question: Is it safe to fly on a specific airline? I am unable to give anyone clear guidance on how to answer this question. If you want a useful answer, you must first ask yourself what does safe or safety mean to you.
One example I like to use is a comparison of Qantas and Southwest Airlines. They are based in different parts of the world and fly entirely different kinds of routes, but they have one thing in common - neither airline has ever had a fatal passenger event or an event that caused the destruction of an aircraft. Does that make them equally safe? Southwest has performed about five times as many flights as Qantas has, but Qantas flights tend to last for several hours while most Southwest flights are less than one hour. Does that make one more safe than the other? Southwest is a carrier that is well known for relatively low fares while Qantas has no such reputation. How does that change your opinion of their level of safety?
The analysis and interpretation of airline safety and risk data is a broad and complex subject that I have barely begun to address in this AirSafe Journal issue. The key point to remember is that simply having access to raw data from an authoritative source like the FAA will not answer many of the questions that you may have about airline risk and safety. In order to find answers that will satisfy your needs, you should make the effort to both first decide what questions you want to ask and then make the effort to find and analyze the appropriate data.Back to Contents for this Issue
Back to the AirSafe Journal Index
http://airsafe.com/journal/issue7.htm -- Revised: 24 May 2015