Q & A: Why not just change our SD?

Posted by Sten Westgard, MS

Here's a question that came in about setting the control limits (or range) for a test:

"for some assays we're using this formula: actual SD * 3 and then divided by 2 plus or minus the mean is this acceptable or not because when we use that give us abit wider range than using the mean plus minus 2SD."

When we asked for an example, we got this data:

Manufacturer Data: SD = 22.5, Mean = 224
Actual Data: SD = 8.79, Mean = 223

"We're multiplying ourSD (8.79) by 3 and then we divide it by 2 to give us the new SD which is 13 (8.79*3/2 = 13).
Then we multiply this new SD 13 by 2 to give us the real 2 SD range which is 26.
So our range is now 197 - 249.
Are we following the right way or not?"

The answer, after the jump...

While it is always tempting to widen control limits, it's not good practice to widen them without justification. Just because they seem "too tight" doesn't mean it's okay to recalculate them with some convenient formula.

What was described has changed the limits from 2 SD to something else. Effectively this is the calculation:

SD * 3 / 2 * 2 = 3 SD limits

What's even more concerning is that while the lab has now shifted to 3 SD limits, the sense from the email is that they've convinced themselves that this is simply the "new" 2 SD limits. There's a world of difference in the error detection and false rejection characteristics of 2 SD and 3 SD limits. With 2 SD, you have a false rejection problem (which leads to repeats and a sense of "too tight") while with 3 SD limits you generally have an error detection problem (you don't get false rejections but you probably aren't catching medically important errors until they've grown pretty big and are already impacting patient care).

This is a case where asking the wrong question leads to the wrong solution. The question here shouldn't really be, can we widen our control limits in this arbitrary way? The lab should instead be asking: what's the quality required by the test? And then, what's the appropriate QC to assure that level of quality?

Here's a different take on the same scenario: If the SD is 8.7 at a mean of 223, that's a CV of 3.9%. If we want to use 3 SD control limits, we need to be achieving a level of world class performance (Six Sigma). In that case, if our quality requirement was about 24% or higher, we'd be safe with 3 SD control limits. On the other hand, if our quality requirement was smaller, say more like 14%, then we would need to be using "Westgard Rules" with additional control measurements in order to provide adequate quality assurance and detection of medically important errors. So the use of the control limits really depends on how good the test needs to be. Given the right quality requirement, 3 SD limits are acceptable.

Part of the problem here is the use of 2 SD control limits. Because 2 SD limits generate false rejection rates of 9% for just 2 controls, this QC practice often leads to frustration in the laboratory. In an effort to eliminate the false rejections, labs often seek out rationalizations for widening the limits beyond 2 SD. It's an understandable impulse, but by trying to fix one wrong, the labs may instead do the wrong thing wronger.

This is a common problem that many laboratories face. QC Design tools are a useful way to get appropriate solutions.