Posted by Sten Westgard, MS
There was some very interesting testimony given in front of the US Congress this week. Boeing and other FAA officials discussed their failure to adequately test the lithium-ion batteries found in the 787 Dreamliner. As you probably already know, on two of the Dreamliners, those batteries malfunctioned and caused fires, which then caused the entire fleet to be grounded.
In the New York Times, it was reported "Boeing’s chief engineer on the 787, Mike Sinnett, said the calculation that a battery would fail only once every 10 million flight hours applied to the design of the battery and did not include possible manufacturing flaws."
In fact, two battery failures and serious fires occurred after less than 52,000 flight hours. The risk was actually over 350 times higher than their estimate!
How could they be so wrong? More after the jump...
Again the Times reports:
Mr. Sinnett defended the plane maker’s choice of lithium-ion batteries, saying at the hearing that the 787’s certification by regulators took 200,000 hours and was the most “extensive in our company’s history.” He added that the battery’s certification was “very rigorous and subject to close scrutiny by the F.A.A.”
Yet the fire in Boston also showed something that the regulators and the company did not expect: that a flaw, possibly no larger than a grain of sand, in a single cell could set off a chain reaction that would cause smoke or fire in adjacent cells.
“We were focused on the hazards of the battery,” said Ali Bahrami, a senior F.A.A. official in charge of the 787 certification. “We knew the hazards were always there. The awareness was always there. We did the best we could under the circumstances, and the knowledge that existed then, to come up with standards that address the requirements for this particular battery.”
This is not a particularly impressive demonstration of the Risk Assessment process. Despite 200,000 hours of effort, despite the involvement of experts, regulators and manufacturer representatives, Risk Assessment failed. And it failed in a way that put lives in serious danger.
As we contemplate using Risk Analysis to determine how we assure quality in the laboratory, we should bear this in mind: is Risk Analysis Reliable? The current proposal of Risk Assessment is hardly thorough. It's a reduced-factor assessment that will be further hampered by the lack of risk information from the manufacturer. That is, the laboratory is toing to have to make guesses, albeit educated ones, about the possible failure modes of different medical devices. Manufacturers are under no obligation to help them.
Finally, it appears that the ultimate assessment will not be a probability of failure, a Risk Priority Number (RPN), or a criticality, which are the traditional outcomes of a Risk Assessment (particularly FMEA). Instead, labs may only be required to make a reductive "Yes, the risk is acceptable" or "No, the risk is not acceptable" judgment. This oversimplifies a complex process and opens up the Risk Assessment process itself to serious risks. Furthermore, it is improbable that the laboratory staff has the time to set aside to thoroughly estimate the risks of their testing processes. They may make rushed assessments using a simplistic model.
"We did the best we could under the circumstances at the time," is not the quotation you want coming from your laboratory after a Risk failure. Our professional ethics - particularly those enshrined in ISO 15189 principles - demand better than this.
Here are the story links:
Boeing Acknowledges Tests Underestimated 787 Battery Risks, By MATTHEW L. WALD and JAD MOUAWAD, Published: April 23, 2013. Link tested 4/25/2013.
FAA followed Boeing's lead on 787 battery testing, By JOAN LOWY | Associated Press – Tue, Apr 23, 2013. Link test 4/25/2013.
Comments