Posted by Sten Westgard, MS
As we write, the National Basketball Playoffs are underway. The home team for Westgard QC, the Milwaukee Bucks, just went down to defeat in the final game of a 7-game series.
Those who follow basketball know that in a series like this, the teams basically alternate home court advantage. The Bucks went to Atlanta to play the Hawks for two game,s then the Hawks came to Milwaukee to play the Bucks for two games, etc. But while there is a home court advantage in the audience (which didn't work: both teams won away games), the courts in every stadium are the same. The basketball hoop in Atlanta is the same as the basketball hoop in Milwaukee. Why is that?
Because the rules specify a consistent goal. The NBA has a rule book which states the precise size of the court, equipment and basket size. A basket, for example, "shall consist of a pressure-release NBA approved metal safety ring 18" in inside diameter with a white cord net 15" to 18" in length." Every stadium must comply with this rule.
Likewise, there is a single standard for the basketball: "The National Basketball Association (NBA) allows only one official ball: The ball must be the official NBA game ball manufactured by Spalding. The ball is orange in color, 29.5 inches in circumference and weighs 22 ounces (size 7). It must also be inflated to between 7.5 and 8.5 pounds per square inch."
Sorry for the long prologue. But wouldn't it be nice if labs were the same?
We're all aware that from laboratory to laboratory, methods can vary. So a TSH test in one hospital may be very different than a TSH test in another hospital. And some methods, like troponin, may be so different that they should probably never be compared to each other. But not only are we playing with different methods, we're also playing with different goals.
A recent review in Clinical Chemistry shows just how variable our goals are. Dr. George Klee, from the Mayo Clinic, conducted this review: Establishment of Outcome-Related Analytic Performance Goals.
Dr. Klee looked through six different techniques for determining quality requirements and found that not only do we not agree on the goal, we don't even agree on how to decide what the goal should be. If laboratorians were trying to play basketball, in other words, we would still be arguing how to write the rulebook.
Worse still, there are some in the laboratory who think we don't even need to discuss this anymore. No matter the method, without even knowing where the goal is, they think every time they shoot, they score. Call it the "Hot Hand" theory of Laboratory Quality.
Even for tests where the interpretation is increasingly standardized, where we should be able to agree, the method goals still are not uniform. At a recent public hearing by the FDA about the quality of glucose meters, devices which are used millions of times a year, one slide presented six different goals for glucose performance:
- CLIA: 10% or 6 mg/dL (whichever is greater) for main lab methods
- ADA: Total analytic error less than 5% for all values
- CAP WBG Survey: 20% or 12 mg/dL (whichever is greater)
- CLSI: 20% for values over 75 mg/dL and 15 mg/dL for values less than 75 mg/dL - but only 95% of all values must achieve this performance
- ISO 15197: 20% for values over 75 mg/dL and 15 mg/dL for values less than 75 mg/dL - but only 95% of all values must achieve this performance
- FDA: 20% for values over 100 mg/dL and 12 mg/dL for values less than 100 mg/dL - but only 95% of all values must achieve this performance
Is there any wonder why the FDA held the meeting? Just to recap, we disagree on the rulebook (how to set the goal), on the size of the hoop (size of the goal), and on the size of the ball (bias, imprecision, lack of method standardization).
Dr. Klee's review provides several evidence-based approaches to derive useful analytical goals from actual test interpretation data. Rather than pick out nice round numbers, there are techniques to analyze patient populations and experiment with the impact of different interpretation lines (or cutoffs). From those cutoffs, we can work backward to determining the necessary analytical quality in the lab.
We still have a long way to go, but Klee's review is an important step toward setting the same goal for performance.
Comments