Posted by Sten Westgard, MS
Alan Greenspan gave some instantly notorious testimony to the US Congress recently:
“I was right 70 percent of the time, but I was wrong 30 percent of the time,”
I'm not sure what was more concerning; the fact that he admitted that almost a third of the time he was wrong, or that he clearly believes that being right only 70% of the time was acceptable. Clearly, for a man once called "The Oracle" and Maestro, he was lowering the bar on the standards for judging him.
Testimony like that raises some instant questions:
- If Alan Greenspan was responsible for setting the direction of the world's greatest economy, is a 30% error rate acceptable?
- If Alan Greenspan was an airline, would you fly him?
- If Alan Greenspan was a core laboratory test, would you buy the instrument?
- If Alan Greenspan was a glucose meter, would you buy the device?
Clearly, error rates are relative. We don't expect economists and the head of the Federal Reserve to be perfect. But if Greenspan knew he was only 70% right, it probably would have benefited the US economy if he had been a tad more cautious. The financial wreckage of 2008, 2009 (and on?) is part of his legacy.
In contrast, we do expect pilots and airlines to be nearly perfect. As it happens, thankfully, airline safety lives up to that expectation. On a Sigma scale, crashes per departure are at a better than a Six Sigma level of quality.
As for laboratory and point-of-care tests, however, what constitutes an acceptable error rate is more open to question. Witness the recent controversy on glucose meters.
On March 16 and 17th of this year, the FDA held a public meeting on the subject of Glucose Meters. While the transcripts are not yet available, the reports of the meeting are already showing some important differences between the laboratory, the manufacturer and the patient's expectations of acceptable error.
This time, the telling quote comes from an industry consultant:
"[H]ow much accuracy you need depends on who you are," said industry consultant Barry Ginsberg. "Those with type 2 diabetes who treat their condition with diet changes and oral drugs don't need to monitor their blood levels as closely as those with type 1 diabetes who take insulin," he said.
Source: FDA eyes improvements for diabetics' glucose devices, Susan Heavey, Reuters.com, Gaithersburg, Maryland, March 16, 2010.
On the face of it, that comment isn't that offensive. As laboratorians know, the use of the test result should rightfully dictate the quality of the method. For a test with higher demands in interpretation, higher quality methods are required.
But witness one reaction from the online diabetic community:
"That's a lame excuse. Why should anyone with diabetes, regardless of type, have to deal with such inaccuracies?" Scott's Web Log, March 18, 2010: Tell the FDA +/- 20% is not good enough.
And another reaction, this time from a patient advocate who took part in the FDA public meeting:
"As patient advocate, what I was hearing was them playing down the impact of outliers — you know, how you sometimes you get a 300 when your BG is really 70.
"They spent a lot of time going over everything a patient does that can alter accuracy — like not washing your hands, or leaving meter in the car — so much emphasis on,”what is the patient doing wrong?’ This, to me, was consistent with the ‘blame the patient’ approach in general.
"My point, was: maybe you want to develop some guidelines for dealing with the outliers, and also the things that can cause inaccuracies.
"My defensive part felt like they were looking at the outliers as if patients were just dots on a Clark error grid that sometimes fall outside the grid parameters. I kept thinking, we are not just a dots on on a grid — these are people whose lives are impacted! They’re making insulin dosing decisions off of this information.
"Every single outlier is a potential problem, and we have plus or minus 20% error margin. How is this permissible?"
Source: Ellen Ullman interview posted on Diabetes Mine: FDA Hearings on Blood Glucose Meters - An Advocate's Perspec
Clearly, there is a disconnect between [at least some] patient perceptions of meter results and the manufacturers' perspective. Patients tend to believe
that the meter number is the number. Any error seems unacceptable. Clinicians are generally better educated on the meter variation, and adjust for the variation in their treatment decision. But with home meters increasingly used in such demanding settings as hospitals with TGC protocols, the FDA was getting concerned.
One other event helped to trigger the FDA public meeting. Recently, the CDC conducted a study of home glucose monitors:
Variability among five over-the-counter blood glucose monitors, Clin Chim Acta 2006, Feb;364(1-2):292-7.Kimberly MM, Vesper HW, Caudill SP, Ethridge SF, Archibold E, Porter KH, Myers GL.Division of Laboratory Sciences, National Center for Environmental Health, Centers for Disease Control and Prevention, Mailstop F25, 4770 Buford Hwy NE, Atlanta, GA 30341-3724, USA. [email protected]
And the findings of that study? The differences between glucose meters varied from 5.7% to as much as 32%.
So again comes the question: if Alan Greenspan was your glucose meter, would you be satisfied?
Comments