Posted by Sten Westgard, MS
A recent email question came in, asking about when it's best to use a single control rule and when it's better to use multirules:
Let's start by identifying two issues and addressing them separately. In the first place, from a compliance point of view, if the accreditation guidelines have specific written instructions to use multirules, you probably need to use multirules. (Obviously, these are not the rules affecting US laboratories, where single rules are the norm.) Arguing the scientific merits of a procedure with an inspector can have unexpected results.
Next, we need to address how the QC procedures are chosen for a laboratory. If a laboratory chooses a single or multirule approach for QC procedures based on tradition, or because they simply apply one set of rules for all tests, regardless of performance, that is one thing. On the other hand, if a laboratory assesses the performance and quality requirements of its methods, choosing QC procedures via a QC Design process, that's another thing entirely.
In the first scenario, where performance of test methods is unknown, multirules might be a better approach. The laboratory isn't adapting its QC procedures to fit the needs of the testing methods, so the additional information supplied by multirules may be valuable. It may also be overkill.
In the second scenario, QC Design has been used to determine the optimal QC procedure based on test performance and the quality required by the test. Thus, if QC Design has determined that the laboratory need only a single rule to QC a particular test, the additional information and detection power of a multirule is not necessary.
The email is correct that when a single-rule QC procedure is violated, for example, the 13s control rule, it is difficult to determine immediately whether or not that violation was a random or systematic error. A multirule would be able to help.
But in this example, where QC Design has determined a 13s rule is sufficient, that does not preclude you from applying multirules once an out-of-control rule has occurred. That is, you can use a 13s rule as your rejection rule, then apply multirules during trouble-shooting to determine the type of error. In some ways this is like the old 2s "warning rule" of the classic "Westgard Rules." The difference here is that the violation is not a warning rule, it's a rejection rule, and the additional rules are being used to determine what kind of error has occurred. (In the classic rules, the additional rules were used to determine if the run should be rejected, not why.)
Take a look at an example with a 5.0 Sigma method (10% TEa, 1.8% imprecision, 1.0% inaccuracy). The first graph will show you the operating point of the method in comparison with the 13s rule:
Now, let's highlight a multirule approach:
Can you spot the difference? It doesn't look like a big difference. The operating point is below both operating lines of the QC procedures. There's a bit more room under the multirule highlighted in the second chart.
Here's where a Sigma-metric / Critical-Error graph is more informative:
In this case, the multirule has approximately 3% false rejection and 100% error detection (it will detect the error within the first run that the error occurs), while the 13s has 1% false rejection and 99% error detection. Given that the rejection and detection numbers are determined experimentally and have a margin of +/- a percent or two, these procedures are almost the same. But the use of the multirule will generate some additional false rejections.
Bringing all of this discussion back to the concrete scenario of the question, I would show these graphs to the inspector (obviously, the graphs would have to illustrate the real tests), which should demonstrate the equivalence of a single rule and multirule QC procedure. I would further explain that a a multirule approach is used in trouble-shooting. That combination agrees with the spirit of the law, if perhaps not the letter.
I had a similar experience with an accreditation inspector in the UK. Our QC rules were designed to ensure Total Errors based on Biological Variation, and as such there were different rules for different requirements.
However, in each case only simple rules were used (2.5s,3.0s, or 3.5s form the mean) - there were several reasons for this, including the inability of the analyser software to correctly apply multirules. The inspector jumped on this stating that by using simple rules we had no control over bias whatsoever.
No ammount of Power Function demonstration could have convinced the inspector otherwise. The only reasonable outcome from their perspective would have been (an incorrect) application of the 10x rule...which he probably uses in his lab.
How should we deal with incorrect (or at best incredulous) advice from our accreditors?
Posted by: AndyBiochem | April 01, 2009 at 09:26 AM