Posted by Sten Westgard, MS
[ Did you want to find out about the 2of3:2s rule and 6 controls? Then Click Here ]
Here's a question I think many labs often have:
"If you were to calculate you own control limits, and one of the 2 levels you were using violated your own limits (while the other is within the limits). You have investigated all possible causes in accordance to a checklist constructed thoroughly, you changed the reagents, you re-calibrated, you confirmed that all maintenance was performed in due time and as should be, yet the IQC fails to return within your limits, would it be acceptable to release the results if this violating QC is within the manufacturer's range? If not, how would you proceed?"
Think you know the answer? See you after the jump...
I think the root of this question is really, "At what point can we just report results already?" (We tried hard, it's still out, there's nothing more that we can do, etc.)
This is a very hard question to answer in the abstract, because it imposes a lot of conditions and makes many assumptions about performance of the method. Let's take inventory of the assumptions:
- We know the quality requirement of the test - and it's the appropriate quality requirement
- We have constructed our Levey-Jennings charts appropriately for this test, based on the lab's own mean and SD, not the manufacturer's range (note that the question already made a distinction between these two).
- We have designed QC appropriately for this test. That means using Sigma-metrics and a design tool like OPSpecs to choose the right rules and right number of control measurements.
If all this was true, and despite going through a trouble-shooting checklist, changing reagents, re-calibrating the method, confirming all maintenance, the QC value is still out, what then?
If QC is still out, it's still out. The manufacturer needs to do something more. The method is not meeting the quality required by the test.
There are some solutions within the power of the manufacturer, including replacement of suspect parts, and they should have an even more elaborate set of trouble-shooting procedures. But if even that doesn't work, and QC is still out, what then?
The laboratory can improve the test by using replicate measurements, thus improving imprecision and (we hope), bringing QC back into line. However, this is an expensive proposition, doubling or tripling the test volume. It's an expense that the manufacturer should reimburse the laboratory for, since it's likely at this point that the method has been engineered with inadequate precision to meet the quality requirement (unlikely that it's the laboratory's fault, unless gross incompetence is present in the staff).
But let's back up to our assumptions again. Remember our first big caveat was that we assumed that we had chosen the right quality requirement. What if we got that wrong?
If we chose a quality requirement that was too stringent, one that NO method on the market can hit, then we are holding the instrumentation to a standard that is impossible to achieve. That doesn't necessarily mean we release the results and move on, but it might mean that we need to examine what the realistic and practical quality requirement should be.
If you look at our recent series of articles examining quality requirements and current method performance, you'll see that there are many cases where no current instrument can hit the most stringent target:
- What's the Right Goal?
- What's the Right Goal? An Example
- Can any method hit the Sodium target?
- Can any method hit the Potassium target?
- Can any method hit the Albumin target?
- Can any method hit the Chloride target?
Then let's look at the second assumption. What if we calculated our own SD incorrectly? If we had a tight SD and repeatedly excluded more and more data points, our own SD could become incorrectly small. That would impose extremely tight control limits and rules. Again, by another route, we'd reach a QC design that no method could acheive. In that case, the solution is more simple: make sure the SD is calculated correctly, without unnecessary exclusions.
Finally, let's look at the third assumption. What happens if we're "over-QC-ing" the method? If we're imposing too many rules that aren't necessary. In the above example, one level is out and one isn't, but we don't know how "out" that level is.Is it out just 2 SD, or is it out 3 SD? That is a big difference. If we apply 2 SD rules, we'll get a lot of false rejection. Our current advice is to design QC so that you can avoid using 2 SD rules whenever possible. That's a simple solution, too.
I apologize if it seems like we didn't really answer the question here. It's a valid question, but to truly answer it, we need to know the details of a specific test scenario. And we need to make sure that none of us are making assumptions about how we're designing and interpreting our QC.
Please, keep those questions coming, though. This is the only way we can all find out what's going on, and how we can move forward together.
I would contact Tech Support of the instrument you are having trouble with your QC
It could be an instrument issue.
Posted by: Carol Antivo | November 19, 2013 at 12:09 PM
Most laboratories would just exclude the data point that is out of control during establishing QC, without investigating the reason behind it. Hence they ended up with a very tight SD. During the routine operation, they would inevitably encounter out of control situation, which they would exclude and repeat and repeat.
So I do advice them that; during the evaluation period of an assay, lab should inform manufacturer if do not conform to manufacturer claim. Suppose the CV conform to the manufacturer's claim but did not conform to the Biological Variation Imprecision requirement. Can lab still set the SD based on the manufacturer's claim? (Lets assume the lab had accepted the assay based on the manufacturer's claim and subsequently a new manager wants lab to set SD based on the Ricos Biologic Variation ;desirable imprecision requirement)
Posted by: KALA DEVI NADARAJAN | November 19, 2013 at 09:13 PM
When applying your own calculated mean and SD value it is very important to keep in mind that lot changes might cause a shift in the mean value,and enlarge the SD. The impact of this lot change depends on the tolerance the manufacturer applies to the reagentproduction. It might be the cause of an out of control situation, after a long period of stable performance.
Posted by: Hans van Schaik | November 20, 2013 at 08:11 AM
Questions I would ask. Were the QC limits calculated on the basis of the same lot number of the QC that is now repeatedly out? Has a change occurred in the QC material itself-(storage, concentration, source.) If the QC material in question were run repeatedly, does the mean differ from the recent historic mean-and not so recent mean ( someone tweaking control levels rather than addressing changes over time?). I am assuming a reserved lot number. if so it is a QC material problem and I do not see that this has been ruled out in the example.
Posted by: Kathleen M. Spiegel PhD MT(ASCP) | November 20, 2013 at 12:27 PM
One night (in 1996), I was working alone and my Creatinine was out on one control, -2SD in the same direction. I did the usual, tried to use my backup instrument, reconstituted new QC material, and no change, maintenance on both instruments. Nothing helped. So I pulled the last 30 days QC records (QC every 2 hours back in those days) and recalculated the QC. The mean dropped by 0.1 and the 1sd expanded by 0.04 mg/dl. This was enough to bring my QC to within 1sd. Wrote everything up for my supervisor and reported my work. Took about an hour, but I was confident my results were valid and my instruments were wroking. Two day's later, the supervisor published new creatinine ranges using many of the justifications I used in my report to her.
Posted by: Steve Young | November 21, 2013 at 03:04 PM
I have a question about CV. One portion of AAPS advocates use of a CV that uses SD/nominal rather than SD/mean. Since the widely accepted formula and its definition (CV) is SD/mean, would it not be more appropriate to push for an alternate abbreviation so the two could be distinguihed? Also what is the advantage of calculating SD/nominal?
Posted by: efoconnor | December 06, 2013 at 06:17 PM