Aviation Safety

The loss of Malaysia Flight 370

An analytical look at what it means and doesn’t mean about aviation safety.

By Arnold Barnett

The disappearance of Malaysia Flight 370

The disappearance of Malaysia Flight 370 in the Indian Ocean triggered a wave of worldwide aviation safety concerns. Are the concerns warranted?

Given that no one knows what happened to Malaysia Airlines Flight 370 (MH 370), it could well seem presumptuous to discuss the meaning of the disaster. But certain points about the crash and its aftermath can safely be made despite our ignorance, and these points are the subject of this article. I will raise six questions here, then discuss them one-by-one, and then end by summarizing my various answers.

1. Did the crash of MH 370 herald a year that signified a decline in aviation safety? 

On Jan. 22, 2014, Travel and Leisure magazine announced that 2013 was the “safest year for flying since 1945.” It based this conclusion on the number of worldwide deaths in commercial aviation. (Actually, 2013 was enormously safer than 1945 because of the huge increase in flying.) Even in a shorter time frame, the magazine saw an impressive gain: There were 75 percent fewer deaths in 2013 than in 2005.

Rest assured that no glowing stories about aviation safety will be written for 2014. After intense coverage of MH 370 that went on for months after the flight disappeared on March 9, three crashes occurred in quick succession in July; one of them was the loss of MH 17 that was shot down as it crossed Ukraine. Other planes crashed with few survivors in Nepal and Iran while, in Pakistan, a passenger died when shots were fired at a landing jet. The year ended with the widely publicized crash of an Indonesia Air Asia jet, with grim news reports showing scattered wreckage in the sea. 

In one respect, the perception that 2014 was a far worse year than 2013 was fully warranted. The number of passenger deaths on scheduled flights worldwide rose from 160 in 2013 to 866 in 2014. But does that sharp increase mean that aviation safety has declined, and that passengers should reasonably feel less safe at the start of 2015 than they felt 12 months earlier?

Common sense suggests that the answer is “no.” Is it really plausible that those who performed so magnificently in 2013 suddenly became a factor-of-five worse the year afterwards? As those in the analytics profession realize, data associated with rare events are subject to great volatility, and drawing great meaning from year-to-year ups and downs can be like saying a coin has changed its molecular structure because two heads follow two tails.

Another perspective on the 2013-14 data acts to underscore this last point. In my aviation-safety research, I have often focused on the metric “death risk per flight,” which is the answer to the question: If a passenger picks a flight at random among the set of interest, what is the probability that he or she will perish in consequence? A key ingredient to calculating that statistic for a particular period is the number of full-crash equivalents over the period. To find that number, each flight is assigned the fraction of passengers who did not survive it, and a summation is taken over all flights. For example, if two of the flights suffered passenger deaths, with everyone killed in one of the crashes and a 20 percent death rate in the other, there would be 1.2 full crash equivalents.

Table 1: Worldwide full-crash equivalents in 2013 and 2014

Table 1: Worldwide full-crash equivalents in 2013 and 2014, scheduled passenger flights.

The pattern for 2013-14 is shown in Table 1. We see that total full-crash equivalents were almost identical in the two years. Indeed, because more flights were performed in 2014 than 2013, death risk per randomly chosen flight was actually lower in the later year. A major reason for the widely differing death tolls is that the passenger jets that crashed in 2014 were close to full, while those that crashed in 2013 were closer to empty.

In short, the perception of diminished safety in 2014 – although understandable – is apparently not matched in reality.

2. Does MH 370 (and MH 17) tell us anything new about the safety of Malaysian airlines?

We should note at the outset that the December 2014 crash of Indonesia Air Asia is not a Malaysian air disaster; Indonesia Air Asia is a separate company that has its headquarters in Jakarta. It is true that Air Asia has its home office in Kuala Lumpur and has a strong relationship with Indonesia Air Asia, but United Airlines has a strong relationship with the other airlines in the Star Alliance. Yet one would not hold United Airlines responsible for a crash on (say) its alliance partner Asiana Airlines.

In a 2010 paper (Barnett, 2010), I offered empirical evidence that the nations of the world fall into three distinct categories in terms of aviation safety. These categories were:

  • Traditional First World, which consists of the United States, Canada, Western Europe, Israel, Japan, Australia and New Zealand
  • Advancing Nations, which are making substantial progress toward industrialization and/or now approach or meet First World standards on GDP per capita and life expectancy
  • Less Developed nations, which cover the rest of the world

In data for the eight years 2000-07, there were no statistically significant difference in death risk per flight within each group, but strongly significant differences across the three groups. Table 2 summarizes passenger mortality risk in the various groups over 2000-07.

Passenger death risk per flight in three groups of nations, 2000-07.

Table 2: Passenger death risk per flight in three groups of nations, 2000-07.

Malaysia was classified in the Advancing group. Over 2000-07, it suffered no aviation fatalities at all, but numerous Advancing group nations did experience passenger deaths: Bahrain, Brazil , China, Cyprus, India, Mexico, Philippines, Singapore, Taiwan, Thailand and Turkey.

If one had to estimate death risk per flight on Malaysian aviation as of Jan. 1, 2008, based on the previous eight years, would one say “zero” because of the observed record? Or might one instead say that, when 11 nations similar to Malaysia suffered crashes, and the overall death risk per flight for the 21 Advancing nations was 1 in 2 million, this statistic is more reasonable to apply to Malaysia? I myself would consider the estimate “zero” to be absurd and would go with the latter figure.

Table 3: Passenger death risk per flight in three groups of nations, 2008-14.

Table 3: Passenger death risk per flight in three groups of nations, 2008-14.

Table 3, the analogue of Table 2 for 2008-14, shows improvement in both the Traditional First World and the Advancing nations. However, the factor-of-five difference between the groups did not change, and the Least Developed nations did not improve at all. The Advancing nations that suffered passenger deaths were: Brazil, China, India, Malaysia, Mexico, South Africa, South Korea, Taiwan and Turkey.

Malaysia fared worse that other Advancing nations over 2008-14. But if we would not attach significance to its superior record in the previous eight years, should we suddenly attach great meaning to its inferior record in the last seven? Over the full period 2000-14, there was no statistically significant difference between death risk per flight in Malaysia and that for all Advancing nations. More precisely, if the three dozen fatal crashes in Advancing nations over 2000-14 had been randomly distributed among all their flights over the period, then it would not have been statistically abnormal for Malaysia to amass the number of full-crash equivalents that it actually did amass (2.08).

The upshot is that, both before and after Malaysia’s terrible experiences in 2014, it would reasonably be classified as a middle-range nation in terms of aviation safety. These middle-range nations are not as outstanding in death-risk statistics as the traditional First World is now, but they are as safe as the First World was in the mid-1980s.

A Malaysia Airlines Boeing 777-200 similar to the one that disappeared

A Malaysia Airlines Boeing 777-200 similar to the one that disappeared and presumably crashed on March 9, 2014. Photo credit: Markus Mainka

3. If MH 370 was deliberately destroyed, does that circumstance suggest an upsurge in criminal/terrorist attacks on aviation?

While it is possible that the loss of MH 370 was accidental, I myself believe that the “preponderance of the evidence” points to a deliberate act. The strange trajectory of the aircraft after its transponder stopped working seems hard to reconcile with an on-board fire or mechanical failure. On the panel on which I appeared at the 2015 Transportation Research Board conference, three of the four participants suspected that the crash was intentional, although the fourth (a pilot) leaned toward an accident tied to lithium batteries.

Table 4: Worldwide passenger deaths in criminal/terrorist attacks against aviation and railroads, 1982-2014.

Table 4: Worldwide passenger deaths in criminal/terrorist attacks against aviation and railroads, 1982-2014.

Even if the disaster is classified as a criminal/terrorist act, however, it does not reverse a time-trend away from successful attacks on aviation. Table 4 shows that, in both frequency and death tolls, successful acts to destroy planes dropped by more than two-thirds from 1982-91 to 2002-14. Moreover, the proportion of aviation deaths among casualties in air/rail attacks fell from 89 percent to 24 percent over that period. These time trends are statistically significant.

4. Did MH 370 and subsequent air crashes in 2014 make potential passengers less willing to fly?

Available data about total numbers of passengers or of passenger-miles do not directly answer this question. After all, if an airline had to cut its fares in half to get skittish travelers to fill its seats, then it would be odd to cite crowded planes as evidence that fear was nonexistent. Fortunately, there is a data set that accounts for fare-slashing as well as other variables. The website Statista has a chart that presents the total amount that passengers spent worldwide to purchase air tickets in a given year. That number is the product of the number of passengers by the average fare that they paid. If either or both of these quantities dropped because of fear of flying, that circumstance would be reflected in the total-payment statistic.

Table 5 presents total amounts paid for airline tickets for the three years 2012-14. This number increased by $30 billion in 2013, a year when highly publicized air crashes were all but absent. But it increased by the very same $30 billion in 2014, when air disasters received intense and continuing media coverage. It would be an imaginative analyst who discerned in these numbers a major drop in passenger demand in 2014.

Table 5: Total amount spent by passengers on airline tickets worldwide in the three years from 2012 to 2014.

Table 5: Total amount spent by passengers on airline tickets worldwide in the three years from 2012 to 2014.

To be sure, these raw numbers offer imperfect comparisons: They do not take into account the level of available air service, background economic conditions, currency fluctuations or any other relevant variables that changed from one year to the next. But there is no reason to believe that adjusting for these variables would change Table 3 dramatically. (In the United States, the inflation-adjusted average air fare, the fraction of airline seats occupied, the number of passengers and the number of passenger-miles all increased between 2013 and 2014.) And, while it would be farfetched to assert that the air disasters in 2014 did not frighten a single person on Earth away from flying, such fear was apparently sufficiently rare as to be all but statistically undetectable.

I personally find this pattern comforting, for it implies the courage and good sense of airline passengers. Without any complex probability models, they might intuitively sense that year-to-year fluctuations in crash records say little about the risk of flying, which is low everywhere and minuscule in many countries. (I further suspect that American passengers did not fail to notice that the 2014 crashes all took place on airlines and in places that they would never frequent.)

5. Will Malaysia Airlines suffer severe and lasting harm because of MH 370 (and MH 17)?

Malaysia Airlines ended 2013 as one of the very few airlines awarded five-stars by Skytrax for excellence on many dimensions that included safety. By late 2014, the airline’s losses of two Boeing 777 jets led to widespread speculation that its survival was in doubt. What can be said about Malaysia Airlines’ prospects?

The historical evidence is mixed. Malaysia Airlines was in serious financial trouble even before MH 370, as was Pan Am before it lost Flight 103 to a terrorist bomb and was Spanair before one of its jets crashed on takeoff from Madrid in 2008 and 154 people died. Both of these airlines went out of business within a few years of these disasters, and the crashes are believed to have contributed to those outcomes. (When the CEO of Pan Am visited MIT about a year after Flight 103 went down, he said that the crash did not greatly reduce overall passenger numbers, but that crucial business and first class traffic had fallen very sharply.) And it may not be a coincidence that the U.S. commuter carriers Air Midwest and Colgan Air – which suffered fatal crashes with no survivors in, respectively, 2003 and 2009 – no longer operate.

On the other hand, there are examples of airlines and aircraft types that overcame immense negative publicity. US Air suffered five fatal crashes between 1989 and 1994, but it soldiered on and took a leadership role in its recent merger with American Airlines. (The carrier, incidentally, has had no fatal events since 1994.) Valujet Airlines was grounded after its Everglades crash in 1996, but it resumed operations, changed its name to Airtran and recently merged with Southwest Airlines, which has a brilliant safety record. The McDonnell Douglas DC-10 suffered three horrific disasters – and one fleet-wide grounding – that cast doubt on its reputation, but DC-10s continued in service until their natural retirement ages.

A special circumstance for Malaysia Airlines is that it is now wholly owned by the Malaysian government. The government’s involvement could allow the carrier to keep operating in the near future despite financial losses that would overwhelm a private company. (In August 2014, Malaysia’s weekly bookings were down by 33 percent, despite price cuts and extra commissions to travel agents.) Barring another disaster, the carrier can be expected to keep flying, although perhaps on a reduced network that focuses on domestic routes. Yet it seems unarguable that the Malaysia Airlines brand has been severely damaged, and that the effect will last for years rather than months.

6. Will we ever learn what happened to MH 370?

Your guess is as good as mine. But my guess is that we will not.

It is not certain that the wreckage of the plane will ever be located. In contrast to the crashes into water of Air France Flight 447 and Indonesia Air Asia Flight 8501, MH 370 has so far yielded no floating debris. (Indeed, such a responsible figure as Tim Clark, president of Emirates Airline, is skeptical that the plane actually crashed into the sea.) Still, the sheer sophistication of the effort to locate the plane in the South Pacific leads me to consider it more likely than not that the plane will eventually be found.

Yet locating the plane is by no means tantamount to learning why it crashed. The cockpit voice recorder only reveals what happened in the last two hours of flight, when experts suspect all those on board were dead and the plane was flying on automatic pilot until it ran out of fuel. (The irregular events that started the crisis took place perhaps six hours before the flight’s demise.) And the flight data recorder might only replicate what is already known about the plane’s bizarre flight path. Conceivably, the evidence from the data recorder will strengthen the hypothesis that what happened to MH 370 was not an accident, or it might suggest some huge mechanical or other malfunction that started the deadly sequence of events. Similarly, the pattern of damage in the wreckage might indicate whether a fire arising from (say) the lithium batteries in the cargo section made the crash inevitable. But altering the relative probabilities of sabotage or accident falls short of eliminating uncertainty. I would be surprised if we go beyond such an alteration, if it occurs at all.

Summary

So how does it all add up? MH 370 cast a dark shadow in 2014, but events that year did not signify that aviation is less safe than before. Nor is there evidence that passengers overreacted to a year in which air disasters got enormous publicity. Nor do the two spectacular losses of Malaysia Airlines jets in 2014 justify a fundamental reassessment of the safety of flying Malaysian airlines. Even if the plane suffered a criminal or terrorist act, aviation sabotage has been declining in recent years (while railroad sabotage has been increasing). Malaysia Airlines will likely survive, though what happened to MH 370 on the night of March 9, 2014, may well be unknown forever. 

In short, MH 370 is a mystery and a terrible tragedy, but it has no cosmic significance. Especially in the First World, the safety of passenger aviation is both staggering and steadily improving. If one seeks a single incident that illuminates aviation safety today, it is not MH 370 but instead US Airways Flight 1549, which overcame devastating damage to land without injuries in the Hudson River.

Arnold Barnett (abarnett@mit.edu), George Eastman professor of Management Science and Statistics at MIT, has worked on aviation safety and security for 16 airlines, six airports, the FAA and the TSA. He has received the President’s Award from the Flight Safety Foundation for “truly outstanding contributions with respect to safety.”

References

1. Barnett, A., 2010, “Cross-National Differences in Aviation Safety Records,” Transportation Science, Vol. 44, No. 3, pp. 322-332.

2. Barnett, A., 2015, “Has Successful Terror Gone to Ground?”, Risk Analysis (forthcoming).