James O. Westgard, PhD
Here are some notes, comments, and links that go along with my presentation at the AACC teleconference on “New Directions in Laboratory QC: EQC, Alternate QC and Risk Assessment,” which was on September 4, 2008.
There is a strong push by CMS and CLSI to introduce new QC guidelines that are based on “risk analysis.” This direction is consistent with ISO’s emphasis on risk analysis for manufacturers. Soon we will see how it influences for laboratory QC. I realize that US laboratories may not yet have much interest in ISO, but we need to recognize that ISO will play an increasing role for guidance in quality management in the future.
QC Options. CLIA rules allow laboratories 4 different options for QC. Here I call these options statistical QC (SQC), deFault QC (FQC), Equivalent QC (EQC), and Alternate QC (AQC). They are discussed in the context of the CLIA regulation as well as ISO 15189 guidance for medical laboratories. The reason for including ISO is that the emphasis on risk analysis originates with ISO, thus we should also understand the context in which ISO describes guidance for QC. Statistical QC. The first option is to do statistical QC (SQC) to monitor the precision and accuracy of the process, detect immediate errors, and monitor test performance over time. This can be done following guidelines from ISO 15189 and CLSI C24-A3, as outlined in the slides. A Sigma-metrics tool will soon be available for download from this website. It will include directions, a Sigma-metrics graph for 2 levels of QC, and a sigma-metrics graph for 3 levels of QC. Note that the C24-A3 document does not include a tool for 3 levels of material. There are many other materials on Sigma-metrics and QC planning on this website. Start with the following if you want more information and background:
- Some background on “power curves” – those detector response curves that show the probability of rejecting runs have different sizes of errors – can be found in the lesson “Power Function Graphs” at www.westgard.com/lesson4.htm.
- The application of “Sigma metrics” for selecting the “right QC” procedures is described in the lesson “CLIA Final Rule: Appropriate QC Procedures” at www.westgard.com/cliafinalrule9.htm.
- There also is a PDF file available for a paper “Internal Quality Control: Planning and Implementation Strategies. www.westgard.com/IQCpaper2003.html
Default QC. There is little to say! This is what most laboratories do today – two levels every 24 hours. However, you should understand that this was never the intention of CLIA that this would become the standard QC practice. It was supposed to be temporary guidance until such time as the FDA provided a QC clearance process that would provide for review and approval of a manufacturer’s QC directions. After five postponements and some 10 plus years of default QC, it was concluded there no longer was a need for an FDA QC clearance process. For some historical background on CLIA and its original recommendation for QC clearance, see the following:
- For an explanation of the QC clearance provision, see “QC Clearance Postponed Again & Again & Again & Again.” www.westgard.com/essay37.htm
- For a brief summary of the changes in the final, final, final, final, final version of CLIA, see “The Final CLIA Rule.” www.westgard.com/essay50.htm
Equivalent QC. There is a lot of discussion on this website. Start with the following:
- See “Equivocal QC: Coming soon to laboratory near you!” www.westgard.com/essay119.htm. At the end of this essay you will find links to the most important earlier discussions. There’s probably more than you want to know, but you may get interested in seeing how this has evolved over the last few years.
Alternate QC. There’s not much specific information available unless you can find drafts of the EP22 and EP23 documents. Those will be hard to come by because they are pretty well protected until the documents are ready to be released. That means the first public drafts will likely become available in 2009 and the final approved drafts probably in 2010. Meanwhile, you can set up a process by which you can take advantage of this information when it becomes available. That’s what I’ve briefly described as a “Total QC Strategy” near the end of this presentation – see slides 34-36 and the following discussions on this website:
- “EQC, AQC, and QC Clearance. From Quality to Compliance. From Compliant to Complicit.” www.westgard.com/essay121.htm
- “Waiting for the FDA” by Sten Westgard. www.westgard.com/guest38.htm
- “Formulating a Total QC Strategy.” www.westgard.com/lesson55.htm
Risk Analysis Too. Be sure to read Dr. Kent Dooley’s essay on “Frequency of QC: A Patient Safety Perspective. www.westgard.com/guest35.htm
What’s SAFE?
A few comments about my thinking here.
- SQC is first on my list because we currently have good design tools for selecting the right SQC rules and numbers of control measurements. The main limitation is that we don’t have an objective methodology for determining how often controls should be run. Here’s where Dr. Parvin’s ideas about “event” and “non-event” QC provide the best available guidance.
- AQC is lower on the list because we don’t yet have the guidance on how to do this. However, if you consider that SQC will most likely be necessary under most conditions to monitor “residual risks,” then the ability to design SQC sets the stage for what should follow with AQC. It’s possible that AQC will move into the #1 spot if the guidelines turns out to be practical and easy to implement, which will wreck my SAFE acronym. I’m not sure how long that will take, but I’m betting that I’m SAFE for another 2 years and by that time I’ll probably really be retired. Also, those tools in the QC tool box may not be so easy to implement, in which case AQC may never become practical. Patient data QC sounds good and can be very useful, but it’s not trivial to optimize the design of patient data algorithms to assure the attainment of the intended quality of results.
- FQC gets an F on my grading sheet. The only plus is that you’re still running controls, but you really should implement the right QC design, which would then move this into SQC above.
- EQC is even worse because you’re not running controls as often. Remember that the laboratory director must take full responsibility for any decision to implement EQC. CLIA allows you to do this, but if you do it, you are responsible and you can’t blame CLIA if you run into problems. By that I mean that CLIA will not be a “get out of jail” card here. You do this at your own risk.
Q&A. Some additional information that may provide more complete answers to some of the questions that were asked:
- Concerning how to review, assess, and evaluate your QC procedures and process, see “Good Laboratory Practices for Statistical QC. Part I. Designing the Right QC.” www.westgard.com/essay102.htm. It is also very important to implement SQC properly, which is discussed in “Part II. QC Limits and Limitations.” www.westgard.com/essay103.htm
- Concerning slide 14 and the figures in column Ped in the key. Those figures should be 0.00. I generate the underlying power curves here using a computer program that typically draws a single line for a particular error condition or sigma-value. Here I need a figure without that line, so I have to fake out the program to get the underlying graphic. It looks like I must have set the error condition to be off-scale on the left, therefore ended up with the same figures as for false rejection (Pfr). Someone has pretty sharp eyes because no one has ever caught me at this before! Now I will have to figure out how to get zeros.
- Concerning the question about instruments that have internal statistical control: Yes, there is at least one BGAS system that has a complete internal SQC procedure, as well as many additional built in instrument checks. And that system still must be qualified for EQC according to CMS, even though the internal SQC is much better than any external SQC in this case because of the sample handling difficulties with BGAS controls.
- Concerning the use of patient data algorithms, such as “Bull’s algorithm” or the more generic “average of normals” or “average of patients,” see “Six Sigma: Patient Data for Assessing Process Performance and Stability” at www.westgard.com/qcapp17.htm. For some explanation of “multi-stage QC,” i.e., the idea of using multiple QC procedures in a complementary way, see the discussion “Sage Advice about New Approaches to Quality Control” at www.westgard.com/essay30.htm
Compliance vs Excellence.
CAP has introduced a new accreditation program CAP:15189 that is based on the ISO document. CAP says that this doesn’t replace the LAP accreditation which is based on CLIA. And they explain that “although it is not currently a standard in the US, CAP believes that a laboratory that is working to achieve best practices in quality management systems will seek out accreditation to ISO 15189:2007 and its benefits.” It looks to me like we’re heading for a two-tiered system where CLIA will be used for compliance with minimum standards and ISO 15189 will be used for excellence. That’s why we need to start paying attention to ISO 15189 if we want our laboratories to remain competitive in the global marketplace. For additional discussion about CAP and its intentions with 15189, see Sten’s interview with CAP officials at www.westgard.com/interview8.htm.
Comments