Evaluating the FAA Standard

Whether the FAA's new regulation would have prevented the Air Canada disaster or, more important, another catastrophe in the future is difficult to determine, partly because serious accidents are so rare. There were no deaths on U.S. commercial aircraft in 1980, for example, and only four in both 1981 and 1984. Even 1985, considered by many "the worst year ever" for aviation safety, was remarkably safe by comparison to other modes of transportation (particularly if deaths due to terrorist acts are subtracted from the total).^[22] Accidents are so rare, according to an actuary Metropolitan Life, that "it is almost pure chance as to which [major commercial airline] has a total loss" in any given year.^[23] The chances of a serious in-flight fire are even more remote. Most aviation injuries and fatalities are caused by impact, not fire. As an airline safety engineer put it, the fatal in-flight fire is "a rare animal of a rare breed." The first recorded fatalities from an in-flight cabin fire in the United States were in the Air Canada calamity.

The almost random nature of accidents creates the first paradox of developing (or analyzing) aviation safety regulations. Accidents precipitate strong political pressures for regulatory change, but they provide little factual basis for making meaningful improvements. Preventing random events is practically an impossible task. Instead of trying to anticipate the unknown, attention tends to get focused on preventing any repetition of what has already happened, regardless of the likelihood that it will happen again.

The politics of aviation safety also conflict with broader political trends concerning government regulation. Many members of Congress openly challenge the notion that aviation safety regulations should be undertaken only if benefits are at least commensurate with costs. Testifying before a House committee about several recent proposals for upgrading cabin safety, the chairman of the NTSB said he "would hate to see [their] implementation delayed for a cost-benefit analysis."^[24] Nevertheless, the FAA is bound by the same executive orders that require all agencies to conduct an economic analysis of proposed regulations.^[25]

This raises the second dilemma of aviation regulation: the FAA must justify in economic terms regulations that sometimes can only be justified on other grounds. This is not to say that the FAA's Regulatory Analysis Group engages in trickery or deception. The agency has refined the use of economic analysis over the years through its capital improvement projects for airport facilities. A detailed FAA manual spells out the procedures and many specific values (including the always controversial value of a life) to use in cost-benefit analysis.^[26] This simplifies and standardizes the agency's analysis. Nevertheless, this case demonstrates how the use of favorable assumptions can make a regulation of questionable economic benefit look economically desirable.

The FAA's Regulatory Analysis Group wrote a forty-three-page cost-benefit analysis of the proposal for smoke detectors and fire extinguishers. The analysis is systematic, comprehensive, and prominently featured in the Federal Register notice. The bottom line, according to the analysis, is that "total expected benefits equal $42.8 million and total costs equal $13.8 million, resulting in total expected benefit-cost ratio of 3.1 and a total expected net benefit of $29.0 million."^[27] There is much to take issue with in this analysis. For example, smoke detectors, assumed to cost $50 each, will very likely cost much more if airlines choose even a few of the features justified by the aviation environment (for example, tamper-proof battery packs, better vibration tolerance).

More important, there are several reasons to call into question not only the magnitude of estimated benefits but whether they actually exceed expected costs. One problem concerns predicting catastrophic in-flight fires. Recognizing the difficulty in predicting the distribution of essentially random numbers, the FAA staff utilized an elegant statistical solution: the Poisson distribution.^[28] The elegance of the solution masks the significance of one critical underlying assumption: the "expected mean value" of two catastrophic cabin fires in the next ten years. This

figure is purportedly based on "historical data." The Varig fire occurred in July 1973 and the Air Canada fire in June 1983. But the Varig fire was neither on a domestic carrier nor a domestic accident. The Air Canada fire caused the first fatalities in the United States from an in-flight cabin fire. To expect two similar fires every ten years seems overly gloomy, especially since there have been improvements in aviation fire safety since the Varig incident. After that fire, several airlines installed heat-sensitive fire extinguishers in trash receptacles. Moreover, there is nothing magic about a ten-year period. Thomas Hopkins, an economist at the University of Maryland's School of Public Affairs, notes in connection with the FAA's analysis of Floor Proximity Emergency Lighting—another proposal linked to the Air Canada fire—that "if the past five years are considered more representative of what lies ahead, there were no pertinent fatalities and so no plausible benefits."^[29] One problem with the agency's cost-benefit analysis, then, is that it does not test the sensitivity of this important assumption. Assuming a mean of one catastrophic fire (instead of two) every ten years could cut the potential benefits of the rule in half.

Other assumptions of vital importance to the analysis include the estimated "coefficients of effectiveness" for smoke detectors and fire extinguishers. Here the analysis purports to incorporate "conservative" estimates. These estimates are not as unambiguously "conservative" as the FAA staff suggests. One assumption is that a smoke detector could avert 50 percent of catastrophic lavatory fires. The staff provides no basis for this estimate or for labeling it "conservative." Stated another way, the coefficient is based on two assumptions: first, that in 50 percent of the lavatory fires that become catastrophic, quicker detection would prevent the catastrophe; and second, that off-the-shelf smoke detectors would actually provide quicker detection. Both assumptions are doubtful. Speed of detection certainly was not the problem in the Air Canada fire. The short-circuit alarm probably alerted the pilots faster than a smoke alarm would have. Delays and uncertainty in responding to the alarm—something as likely to occur with smoke detectors as with circuit breakers—were the main problems. Second, the detection capability of household smoke detectors in an aviation environment, as already mentioned, is quite uncertain. False alarms are likely to be a problem; some fires may go undetected. Combining these considerations, it appears that 25 percent, or maybe even 10 percent, is as reasonable an estimate as 50 percent. Similar arguments apply to the coefficients used for calculating the benefits of trash receptacle extinguishers.

A third problem with the economic analysis is that it overstates the benefits by failing to take into account existing "compliance" with the proposed rule. At least half of the commercial carriers voluntarily complied with the FAA's 1980 request to carry Halon extinguishers (a few even installed smoke detectors without any FAA advice). The benefits attributable to the proposed rule should reflect only the incremental benefit of adding Halon extinguishers to the remaining carriers. Instead, the FAA adjusted the cost figure to reflect the marginal cost of Halon extinguishers but not the marginal benefits.^[30]

The cost-benefit analysis also did not take into account the special training necessary to use a Halon extinguisher effectively. This means either overstated benefits or understated costs, depending on whether the airlines voluntarily improve existing training programs. Most airline employees receive cursory instruction in fire fighting. Only two airlines provide actual "hands on" training with fire extinguishers.^[31] "Hands on" training is particularly important with Halon extinguishers, which discharge in a liquid stream that must be carefully applied because it lasts less than ten seconds. Those not properly trained in the use of these sophisticated extinguishers may actually be less effective than with other extinguishers. Even with less-sophisticated extinguishers, untrained operators are generally able to extinguish only about half as big a fire as those with training.^[32] The carbon dioxide extinguishers aboard the Air Canada jet were discharged without effect in the early stage of that disaster. As an engineer with Factory Mutual Research Corporation put it, "It wouldn't have made any difference if you had given them another thirty extinguishers." The cost-benefit analysis never considered this issue. In fact, the provision for Halon extinguishers was given only superficial treatment. The staff considered these extinguishers "clearly cost-beneficial" because they are lighter than older extinguishers, so "fuel savings alone are expected to pay for this proposal." In short, they were assumed to be more effective. The projected $2.9 million in "pure safety benefit" might be more than outweighed, however, by the cost of training flight attendants adequately to ensure effective use.

Finally, even if the FAA were predisposed to write its own standard, the rulemaking staff should have been aware that the private standard NFPA 408 existed. At least one FAA employee has been on the NFPA's Aircraft Rescue and Firefighting Committee, charged with standards for aviation fire extinguishers, since NFPA 408 was first adopted in 1956. Two FAA representatives were on the committee when the standard

was revised in 1980 and later when the FAA decided to draft and adopt its own standard.^[33] One reason the rulemaking staff was unaware of NFPA 408 is organizational. Neither of the FAA employees were from the Airworthiness Division—the division that wrote the FAA's standard for fire extinguishers—and neither took a particular interest in NFPA 408. They were more interested in other aircraft rescue and firefighting standards.^[34] Beyond these peculiar circumstances, however, the fact that the rulemaking staff did not think to check for the existence of an NFPA standard is evidence of the low profile that currently characterizes these standards.

None of these issues was raised by those commenting on the proposed rule or, surprisingly, by the Office of Management and Budget in its review of the cost-benefit analysis. Politics, it seems, loomed larger than economics. "This was a motherhood issue," explains an FAA rulemaking staff member. "Who is going to argue about fire extinguishers in airplanes?" Indeed, the FAA received hundreds of handwritten letters and postcards from individual citizens in favor of the proposed rule.