By Sten Westgard, MS
This is the last thing I have to say about the CLSI and CMS update at the AACC/ASCLS conference in DC. There was one striking point by Judy Yost (Director, Division of Laboratory Services, Centers for Medicare & Medicaid Services) during her presentation (PDF link) on frequent laboratory deficiencies: Nearly 80% of laboratories in the US do not have any routine oversight
The slide above breaks down the most recent CMS information (December 2007) on the types of laboratories in the US. As you can see, there are 119,938 waived labs (labs that do only waived testing), and another 38,903 labs perform only Provider-Performed Microscopy testing. Since neither of these types of testing requires inspection, there is no routine oversight for nearly 160,000 laboratories.
In contrast, CLIA inspects 19,827 laboratories (called Compliance labs), and the Joint Commission, CAP, and other "deemed" providers inspect another 16,098 laboratories (the Accredited labs). Finally, since New York and Washington state are exempt from the CLIA rules, those states inspect the remaining 6,000 laboratories. All told, there are about 41,925 laboratories that do get some form of routine oversight.
When I heard this, I was reminded of the physics concept called Dark Matter.
Dark Matter is term coined by astronomers. Its meaning is pretty simple: it is matter and it is dark. Specifically, dark matter is matter "that does not clump together in stars and hence does not give off light and that thus exerts a gravitational pull without revealing itself visibly."1 Basically, we, here on earth, cannot observe Dark Matter in the universe. We can only infer that it exists because we see its effects: without Dark Matter, the universe would fly apart. There is so much Dark Matter, it helps hold the universe together. Less than 5% of the universe is visible matter (stars, planets, gases, and other things we can see with various kinds of telescopes and observation technologies), while Dark Matter is close to 25% of the universe.2 So there is five times more stuff we can't see than stuff we can.
It seems that we have reached the same situation in healthcare laboratories. While we argue vociferously about regulations and complexity and validation and quality control in those nearly 42,000 laboratories that are regulated, but there are four times as many laboratories that have almost no supervision and we don't talk about them even though they are majority of testing facilities. The increased scrutiny and requirements of regulated labs may have driven more and more laboratories to the "dark side", since waived testing allows them to escape tough requirements for testing. This would be a good thing, if we knew that waived laboratories were operating correctly.
But we don't really know what waived laboratories are doing. Without any routine oversight, we have no data to look at, so we can really only guess at their performance.
The few studies of waived laboratories have not been encouraging. The GAO report on laboratory testing found that between 5 and 10% of laboratories did not follow manufacturer's instructions, and about 5% did not run two levels of QC.3
A 2002-2004 survey of waived laboratories found that 12% did not have manufacturer's instructions, 21% did not perform QC, 45% did not document the name, lot number and expiration date for all tests performed, and 35% did not maintain a quality control log at all.4
The CMS presentation listed the top 10 deficiencies of laboratories as of 2007. Here they quote the specific rule that is being violated:
- 493.1236--Alternative assessment for test not in PT (5.9%)
- 493.1252--Lab must select test system (5.7%)
- 493.1252—Lab defines storage conditions (5.6%)
- 493.1289—Analytic systems QA (5.6%)
- 493.1251—procedure manual content (5.0%)
- 493.1239—General lab systems QA (4.2%)
- 493.1407—Lab director establishes QA plan (4.2%)
- 493.1291—Lab report content (4.0%)
- 493.1252—No expired reagents (3.8%)
- 493.1403—Unqualified lab director (3.5%)
The most recent numbers appear to represent improvements in waived laboratory performance. Nevertheless, even these lowered rates are not low enough. The problems we see in waived laboratories are larger than what we see in more professional laboratories. Even if only 5.6% of the labs aren't doing their QA and QC, that could mean that about 7,000 labs in the country are running without proper quality control in place.
When the CDC issued its report on Laboratory Medicine: A National Status Report, it also issued a report on the Quality of Proficiency Testing. In that second report, the authors made the extraordinary recommendation that all waived labs should have to perform proficiency testing - a major change which would require an modification to existing law. Those authors clearly saw that waived testing needs to be brought back into the regulatory fold, because too many errors are happening in places where we are blind.
It is unlikely that the regulatory, industry, or political will exists to make such a change to the US laws. And until then, unfortunately, a lot of laboratories will continue to operate in the dark...
Further articles on this topic:
- The CLSI description of the meeting
- Update on EP22, EP23, and "Alternative QC"
- The Quality of Proficiency Testing
- Brian Greene, The Fabric of the Cosmos, 2005, Vintage Books (New York, NY), p.295
- You probably note that 5+25 does not equal 100. Evidently, even the existence of Dark Matter doesn't explain the make up of the universe. The other 70% of the universe is made up of Dark Energy. I don't know what that is, and it's even less relevant to this discussion, so let's not go there. Wiki Dark Matter if you're curious.
- Memorandum from Mark McClellan, CMS Administrator, to Leslie Aronovitz, GAO, May 17, 2006. Baltimore, MD; Centers for Medicare and Medicaid Services, 2006. Quoted on page 284 in Laboratory Medicine: A National Status Report, a report prepared for the Division of Laboratory Systems, National Center for Preparedness, Detection, and Control of Infectious Diseases, of the Centers for Disease Control and Prevention.
- Howerton D, Anderson N, Bosse D, et al. Good Laboratory practices for waived testing sites: survey findings from testing sites holding a certification of waiver under CLIA 88 and recommendations for promoting quality testing. MMWR Recommendations and Reports 2005;54(RR-13):1-25. As quoted on page 278 in Laboratory Medicine: A National Status Report, a report prepared for the Division of Laboratory Systems, National Center for Preparedness, Detection, and Control of Infectious Diseases, of the Centers for Disease Control and Prevention.