The Westgard Rules

Happy Holidays 2008 and a Hopeful New Year 2009

James O. Westgard, PhD

[Editor's note: this essay contains some discussion of politics]

As we near the end of December, it is -7 F on the thermometer and +31 inches on the snow gauge. The winter shows all indications of being a bad one, at least here in Wisconsin, but the next year shows many signs of hope in the US. We have new leadership in our government, a more centrist political disposition, (we hope) less partisanship in our political process, and a high priority for reform of our healthcare system. Many wonder if healthcare reform will be compromised by the financial and business crises facing the country, but I think such reform will be necessary as part of resolving those crises. Businesses and even the medical community are now supportive of fixing healthcare!

I am encouraged by the appointment of Tom Daschle as the Secretary for Health and Human Services and also as the “healthcare czar.” I don’t know much about Daschle, except that he comes from South Dakota, which makes him a neighbor and gives me some sense of his background and character. Although I’m originally from North Dakota, remember that at one time “Dakota Territory” included both North and South Dakotas, so anyone from Dakota is a neighbor who will befriend another Dakotan at any time or place. Likewise, I think Daschle will befriend all Americans in their need for better healthcare. I realize that there is little logic in that reasoning, but character and commitment will be the key ingredients of the new leadership.

I still worry about the need for improvements in the quality of laboratory services because the financial crisis will provide an ongoing rationale for minimizing the cost of laboratory testing at the possible expense of quality. The CLIA regulations went in the same direction as the US financial regulations, i.e., reduced or lenient regulation to aid business, which here includes the healthcare business as well as manufacturers and suppliers. Certainly CMS’s efforts to reduce QC through supposedly “equivalent QC” follows the pattern of reduced regulations. CMS’s support of “alternative QC” based on manufacturer’s risk assessment is another step in that direction. While we hear a lot about the need to review the regulatory process for financial institutions, there is no evidence that anyone understands that the same may be true for laboratories. Lax regulation of quality by CLIA may be turning our testing sub-prime.

I have a high respect for manufacturers' efforts to improve measurement technology, but there is always a need for independent review and assessment of that technology and its performance in routine applications. We rely on the FDA to approve new methods and analytic systems, but remember that the FDA works on the principle of “truth in labeling." The FDA primarily checks that the manufacturer has data to support the claims that are made, not that performance is acceptable for the intended clinical applications, which requires define goals for quality. The FDA has begun to advise manufacturers of “waived” testing devices to define a goal for analytical quality and to document its achievement, but that approach has not yet been applied to the 510k process for approval of “non-waived” tests. Let’s hope that they move in that direction soon.

Analytical quality will become more critical as programs for “Pay-for-Performance” (P4P) are implemented. Some of these are already underway and Medicare is taking a step in that direction by implementing P4P incentives for quality improvement. As we saw this year with the wave of "never event" non-payment policies, other healthcare payers willl follow Medicare's lead on this issue. We will see these P4P efforts evolve and expand in the next few years.

On the surface, P4P seems like a reasonable approach, but there is little research that shows it really works! Undoubtedly, laboratory tests will be included as measures of performance because they provide "hard numbers" upon which to judge performance. However, the use of laboratory tests in this way assumes that the test results are comparable from method to method and laboratory to laboratory and over time. Such comparability depends on traceability, i.e., the availability of reference methods and materials to provide a traceable chain between a laboratory method and “trueness.” Furthermore, such comparability in principle (traceability) needs to be monitored and documented by External Quality Assessment and Proficiency Testing surveys. Yet such surveys today show much method to method variation for many laboratory tests. We still have much work to do if laboratory tests are to provide reliable and rugged measures for such applications.

Thus, the need for analytical quality in laboratory tests is becoming more important in this age of evidence-based medicine and the advent of P4P programs. This reliance on quantitative quality for P4P lays bare the assumption that the quality of laboratory testing is adequate for clinical care. Even though most professionals outside the laboratory have little understanding of the actual performance issues, they seem convinced that they can base future payment decisions on those testing results.

I often find myself having to defend the need for analytical quality in today’s patient safety era (with its emphasis on pre-analytic and post-analytic errors). Having been singled out by Clinical Laboratory News as “one of the most outspoken critics of the EQC guidelines…” (CLN, November 2008, Risk Management for Clinical Labs), I’m sure some people are tired of hearing me speak out on this issue. But somebody needs to do it and I find myself having to take on that role given the absence of concern by our professional organizations, which is evidenced by their acceptance, if sometimes only grudging, of EQC and their support for AQC. It will be interesting to see what they have to say about P4P!

Yet there is reason for hope that analytical quality will receive the attention it needs! There’s nothing like putting a price tag on something to increase interest. I believe P4P will bring more attention to the issues related to analytical quality. Also, the CLSI process of developing the risk management guidelines is proceeding slowly, despite the fact that these documents are supposed to be on the “fast-track.” There are still opportunities for CLSI member organizations to review and comment on these guidelines before they become final. Healthcare, in general, will be under the microscope in the next few years and that will also bring to light the issues with quality in laboratory testing.

Ultimately, each of us will have to exert our influence if quality is to be achieved in laboratory testing and in healthcare. We each have different opportunities to do this, different ways of making our voices heard, and one of the most important is taking care of quality in our daily work. Quality in healthcare often happens because individuals make the effort to do the right thing right. Individuals often save the day for quality, in spite of the many difficulties in their daily work. That’s why you are such an important part of the process of achieving quality in healthcare!

The rewarding part of quality for those of us at Westgard QC is your continued interest and support. We know we have a loyal audience that is dedicated to providing quality laboratory services. Our biggest hope for the New Year is that we can continue to work with you and help you in your efforts to improve quality.

From the whole staff (and family) at Westgard QC, Seasons Greetings, Happy New Year, and Thank You for your continued patronage, support, and friendship.

Glucose vs. HbA1c

By Sten Westgard, MS

Over on the main website, Dr. Westgard has a new essay discussing estimated average Glucose and the new push to use HbA1c methods for screening and diagnosis of diabetes. There have been two front page articles in Clin Lab News on these topics.

Two additional points worth making about HbA1c. First, the National Glycohemoglobin Standardization Program (NGSP) has set a goal of reducing the imprecision of HbA1c methods. So CAP proficiency testing criteria, for example, are going to be tightened from 12% down to 6% over the next four years.

Let's repeat that. The quality requirement is going to be reduced from 12% to 6% in the next four years.That's cutting the quality requirement in half. Wow.

That's real leadership and courage right there. That is using the power of regulation to force improvements in instrumentation. I know my experience of the laboratory world is limited, but I haven't seen that kind of agressive goal-setting before. Have you?

It will be very interesting to see how the diagnostic manufacturers respond to tightening quality requierments. Simply put, I don't think many of the methods on the market now can make the cut. They will not be good enough to hit a 6% target. ( Also, will the other PT providers follow suit, or will they keep their targets wide to try and scoop up customers? Here's where competition might work against quality. )

Second point. The article doesn't talk about a big issue in HbA1c methods: differences and bias between methods. For all the work on traceability, the methods in the field remain considerably different. There's a recent paper out in the Journal of Clinical Pathology, where Holmes et al seriously question the use of HbA1c testing:

“Our results also question the validity of using HbA1c test results as a measure of the quality of diabetes care.  Given the present state of the art for HbA1c testing, the proposed pay-for-performance use of these results could lead to an ironic scenario in which analytic methods that are biased low relative to the NGSP reference method are considered to reflect a high quality of care and secure a better level of reimbursement for the provider while at the same time promoting clinical inertia for the intensification of diabetes treatment and underestimating the risk of diabetic complications.”

“The gap between the level of analytic quality that experts consider necessary to obtain the maximum clinical usefulness from an HbA1c test and the level of performance presently provided by the methods used in the United States suggests that there is, indeed, cause for concern.  The shortcomings of contemporary HbA1c testing need to be more widely recognized by the end users of the results.  In addition, individuals and organizations involved in the design, manufacturer, performance, accreditation, and regulation of HbA1c testing need to devise and implement plans for improving the analytic quality of this important test.”

Holmes EW, Ersahin C, Augustine GJ, Charnogursky GA, Gryzbac M, Muttell JV, McKenna KM, Habhan F, Kahn SE. Analytic bias amoung certified methods for the measurement of hemoglobin A1c. Am J Clin Pathol 2008;129;540-547.

When you add pay for performance on top of the method bias problems, you've got a scary scenario. Just by switching to a new, lower-reporting method, clinicians and institutions could increase their compensation - without ever changing the quality of healthcare delivered to patients. That's not pay for performance. That's pay for method bias. 

Mentor of the Month: Don Wiebe

By Sten Westgard, MS

Over at the AACC website, they have a "Mentor of the Month" section, where they focus on a distinguished scientist who has made contributions to helping the next generation of laboratorians.

I'm happy to point out that the December 2008 Mentor is another one of the "Wisconsin Mafia": Don Wiebe, Ph.D., DABCC, FACB. They also posted an interview with him. You even get a flavor of Don's warmth and humor, even though there aren't any of Don's famous jokes in the article. [Note: I'm using the Google cached versions of those pages, since the AACC changes the Mentor every month.] 

Don has worked with Dr. James O. Westgard for decades - and taught him a thing or two about the laboratory. Recognition of Don's contributions to clinical chemistry is long overdue and well-deserved. He's a great scientist, laboratorian, teacher and friend.

Congratulations to Don. And we encourage you to read more about him.

Failure is an option?

By Sten Westgard

On November 4th, the Joint Commission issued an interesting press release, titled "Lab Decisions Will No Longer Affect Hospital Decisions."

The specific language of the press release stated:

"Beginning January 1, 2009, under new Joint Commission policy, laboratory accreditation decisions will no longer immediately impact hospital accreditation decisions."

I have subsequently seen comments on a listserve wondering if it's now acceptable for JC-accredited hospitals to have laboratories that fail inspections. The simplistic interpretation of this rule is that laboratory problems no longer impact the hospital. Hospitals can keep running regardless of the state of their laboratory.

But that's not really the case.

I contacted Megan Sawchuk, Associate Director of the Standards Interpretation of the Joint Commission. She elaborated on the new policy and cleared up any ambiguity:

"The December 2008 Perspectives announcement regarding laboratory accreditation decisions has two important elements. One, the Accreditation Committee voted to eliminate the automatic, direct weight of an adverse decision in the laboratory on the hospital. And two, an adverse laboratory decision from The Joint Commission, CAP or COLA will be added to the hospital's Priority Focus Process (PFP) data. PFP data is presently used by The Joint Commission to monitor the hospital's overall performance and prioritize the timing of their unannounced survey in the 18-39 month window. Thus, an adverse decision in the laboratory will significantly increase the likelihood of an earlier hospital survey to assess compliance at the organizational level.

"By using this method, the hospital decision is based on their actual overall performance with consideration of that of the laboratory. This is an improvement over the current process of automatically applying an adverse laboratory decision to the hospital, which assumes an overly simple relationship between the two integrated but separate entities. Noncompliance in the laboratory is often associated with poor performance in the overall organization, but not always. This method also maintains the integrity of the the laboratory as an essential service in the hospital's accreditation decision process."

To be clear: a failing laboratory will still take down a hospital with it. The downward spiral to revocation of accreditation may not be as fast as it used to be. But the usual regulatory process takes time in any case. Inspections generates citations, which require responses, which may then generate additional inspections, additional responses, etc. Immediate action happens very rarely. The Joint Commission retains all the policies and tools they need to come down hard on a lab and hospital. This new policy just gives them a little more latitude.

One last thing: this is a clear admission that many laboratories in America have significant problems. If laboratories were operating perfectly (or even just in compliance) and there weren't any worries about them, we would have no need to decouple their accreditation decisions from the hospitals.    

What progress will we make?

by Sten Westgard, MS

As regular readers of Westgard Web know, Dr. Westgard has not been shy about making political comments in elections past. 

This election cycle, we didn't make many political comments on the blog or the website. It seemed to us that the candidates and the parties did just fine by themselves.

Over the past few days, we have received many emails of hopes and congratulations from people all around the world who have watched the US Presidential race. We thank you for those kind messages. It is a testament to the interconnectedness of the world that we all have a stake in each other's politics.

We celebrate the election of the new President. But the celebration can be only temporary. The real work remains before us:

"This victory alone is not the change we seek - it is only the chance for us to make that change. And that cannot happen if we go back to the way things were. It cannot happen without you."
President-Elect Barack Obama, November 4th, 2008, victory speech

CLIA at 20

posted by Sten Westgard, MS

This week, AACC is going to host an audioconference on the latest update of CLIA. We're celebrating 20 years of CLIA in 2008.

Over at the online store, you can celebrate by saving $20 off the price of the CLIA Final Rules manual.
Use the coupon code clia20 during the checkout and you'll save $20.

This offer will run from now (October 6th) until Halloween - that's more than 20 days of savings.

DNV Healthcare - Hospitals now, Labs later?

After some follow-up, we need to note that DNV Healthcare has only been deemed by CMS for accrediting hospitals. They did not get deemed status for laboratory accreditation.

In other words, they cannot inspect or accredit laboratories. Any hospital that switches to DNV Healthcare will need still need to to accredit their lab by some other agency. [In an earlier version of this post, I inadvertently implied that only CAP or COLA accreditation would be accepted by DNV. That was a mistake on my part. I didn't intend to imply that only those two accreditations would be accepted by DNV. My apologies for my imprecise statement. The important point is that DNV currently will not accredit labs. ]

Whether deemed status for laboratory accreditation is in the future for DNV Healthcare is anyone's guess. It took years of battling for DNV to achieve the hospital deeming authority. Getting to laboratory deeming authority may require years more.

Still, this reality may make it less attractive for JC accredited hospitals to switch. If you currently use JC for both your laboratory and hospital accreditation, switching to DNV will require you to switch to COLA or CAP as well. Two switches for the price of one?

CAP may be a better fit with DNV's hospital accreditation, since CAP offers an ISO 15189 approach and DNV is going to stress ISO 9001 as part of the accreditation. But COLA has also been using a lot of ISO terminology and quality systems approach in their accreditation.

The New Kid on the Block

Sten Westgard, MS

The game of accreditation agencies hasn't changed much over the last 40 years: Joint Commission, CAP, COLA. Laboratories didn't have many other choices.

Now there's a new kid on the block.

CMS recently approved DNV Healthcare as a new hospital accreditation organization. DNV's hospital accreditation program has met all the CMS requirements to deem hospitals in compliance with the Medicare Conditions of Participation. The DNV program is called NIAHO^SM (National Integrated Accreditation for Healthcare Organizations). [ DNV stands for Det Norske Veritas which is a Norwegian-based company that provides accreditation, certification, risk management and other services to many industries. What is it about Scandinavians, quality, and regulations?]

What's more interesting than just the entry of a new player into the accreditation market is their approach. NIAHO is not another compliance-oriented program - participation in this accreditation program requires the hospital to seek and achieve ISO 9001 certification. So hospitals will have to be accredited by NIAHO and certified in ISO 9001.

Here's the schedule DNV proposes for accreditation and certification:

• Year One: NIAHO Accreditation Survey and ISO 9001 Pre-assessment Survey
• Year Two: NIAHO Accreditation Survey and ISO 9001 compliance or Certification Survey
• Year Three: NIAHO Accreditation Survey and ISO 9001 Periodic Survey
• Year Four: NIAHO Accreditation Survey and ISO 9001 Periodic Survey
• Year Five: NIAHO Accreditation Survey and ISO 9001 compliance or Re-Certification Survey
• Year 6 through Year 8 and Beyond: Continue to repeat Year 3 through Year 5.

DNV will conduct annual unannounced surveys on hospitals. That's a significant change right there.

DNV's NIAHO is different than CAP's nascent ISO 15189 program. CAP is offering an ISO certification on top of the usual certification. That is, you have to do the usual CLIA-based certification, but you can add ISO 15189 on top of it. If you choose DNV Healthcare, you'll have to seek ISO 9001 certification as part of the process. Compliance alone is not a DNV option.

We have yet to see what kind of specific laboratory rules DNV Healthcare will provide. As with a lot of the ISO standards, specifics are often hard to find. Many ISO standards provide broad goals without technical specifics, leaving it up to the managers to adapt and apply the rules. Will there be Checklists? Tracers? Something else? So far, we don't know.

Obviously, whatever DNV Healthcare applies will have to be in compliance with CLIA regulations. But how will ISO 9001 and CLIA minimums mix? Will DNV require more from hospitals and laboratories than JC or CAP?

The even bigger question is - will DNV Healthcare compete on cost, quality or another feature? The cynic in us wonders if more competition will drive down prices and possibly sacrifice quality. The optimist in us thinks it would be interesting to see an accreditation body make excellence, instead of compliance, its competitive strategy.

Stay tuned.

Has Quality gone Sub-prime?

Posted by Sten Westgard, MS

As our Congress ponders a $700 billion dollar bail-out of Wall Street, and the rest of us on Main Street ponder our financial futures, it seems like as good a time as any to draw some parallels between the larger US economy and the laboratory economy.

For those of you who aren't watching the news, this Wall Street crisis got its start in Sub-Prime mortgages. (A nifty picture here illustrates how and why this happened and a handy little timeline on the most recent events in the crisis can be found here) Subprime mortgages used to be something that only people with poor credit history could get, perhaps only 15% of the mortgages. They used to be considered a risky proposition that most investors avoided. Yet they become very popular with both borrowers and investors alike, until nearly 25% of all mortgages were subprime.

What happened?

The Start: The Government is here to help

The seeds of this financial disaster were planted, ironically, in the ashes of the last crisis, the Internet bubble. At that time, the Federal Reserve lowered interest rates to just 1% in order to spur the economy. These low rates made it easier to borrow money, making home ownership possible for previously marginal home owners.

In the aftermath of laboratory scandals, the government passed the CLIA legistation. However, since they were unable to work out the procedure for manufacturer's QC clearance, they delayed that provision. In the interim, they gave some devices and QC methods temporary waivers.

The New Rules

Eager to attract grow their margins, the mortgage industry created a new raft of "financial instruments" to extend loans to riskier and riskier customers. These loans included the following types:

• Teaser Rates, aka Adjustable Rate Loans. This offered a low interest rate at the beginning of the loan, followed later by a dramatic rise in the rate. Save and buy now, Pay a lot later.
• No Doc or "Liar Loan." This loan allowed the applicant to "state" their income but not provide any proof of this income. It was perhaps first intended for people who work in jobs where the wages are primarily earned in cash, perhaps day laborers and waitresses.
• NINA Loan: No Income. No Asset Verification. For this loan, the applicant did not have to state an income, nor did they have to verify their assets. In theory, these were loans for people who wanted their privacy and were willing to pay through the nose to keep it.
• NINJA Lona: No Income. No Job. No Assets. This was a loan given to people who could fog a mirror. This isn't just tortured logic. This is logic that has been murdered.

At the height of the bubble, when housing prices were rocketing up, these loans seemed to make sense. Why not take out a subprime mortgage, if only a few years from now, the value of the house is going to double. If housing prices rise forever, you can sell the house to the next person for far more than your mortgage, no matter what the terms.

What excerbated this problem even more was the unscrupulous nature of some of the loan officers. Since they were paid commissions up front for getting new subprime loans, some officers would falsify mortgage applications for their unknowing clients.

After repeated delays, CMS and the FDA cannot find an agreeable way to implement manufacturer QC clearance. So it's back to the drawing boards. CMS writes a new clause in the law - "Equivalent Quality Testing" - and invents a new set of "Equivalent QC Procedures" which are added to the Interpretative Guidelines of the State Operators Manual: Option 1: Run controls for 10 days, then run QC only once a month Option 2: Run controls for 30 days, then run QC only once a week Option 3: Run controls for 60 days, then run QC only once a week "Option 4" or "Alternative QC" has been the subject of numerous CLSI committee, with years of work put into developing so-called Risk Management approaches The absurdity of these options have been exhaustively documented and discussed on Westgard Web. Yet despite the near-universal condemnation of the the options, despite the admission from CMS that "We blew it", it appears that these options are here to stay.

The Collaborators: Pervasive Leadership Failure

It wasn't inevitable that subprime mortgages got out of hand. It wasn't inevitable that Wall Street would become so addicted to these dubious financial instruments. If the heads of these institutions had demonstrated leadership (and ethics), instead of relentless greed, we wouldn't be in this mess.

Part of the problem was the dispersion of risk and responsibility for these subprime loans. Since no single institution owned an entire subprime loan, it didn't feel the need nor the obligation to ensure that it was a good loan. Back in the old world of mortgages, if a single bank made too many subprime loans, it would become obvious that there was too much risk and the market would force the bank to stop its practices. However, banks and mortgage lenders didn't hold onto their own loans anymore. Instead, they packaged them up into "collateralized debt obligations (CDOs)," a type of "mortgage-backed security," which they then sold off to the broader market. Thus, the banks and mortgage brokers got paid up front, outsourced the responsibility to someone else, and had little incentive to be responsible or think for the long term.

Furthermore, competitive pressures meant that choosing to be ethical was a money-losing proposition. "Everyone was doing it." If one bank refused to get into the subprime business, they suffered and another bank got that business.

Worse still, the rating agencies, which could have put a stop to the mortgage-backed securities by giving them bad (risky) ratings, instead felt pressure (and reaped rewards) to give high ratings. Thus, a security made up of many risky subprime mortgages was given a AAA rating by agencies like Moody's and Standard & Poor's. Since mortgage securities were outside federal regulations, the government did not act to rein in the risky practices, either.

Individual criticism of Equivocal QC wasn't that hard to find. But it was strange to see that most manufacturers, agencies, and professional organizations were silent on the issue, particularly at first. Once the popular verdict on EQC started to come in, more institutions were willing to join them. But quality seems to be a subject like politics in the US; everyone has an opinion on it, but usually they don't want to talk about it in public. Manufacturers face the market pressure to match whatever "cost saving" features their competitors offer. So if EQC means reduced QC costs, it's difficult for a single company to hold fast and insist that their customers need to spend more on QC. Since the professional organizations draw their support from manufacturer sponsorships, advertisements and other support, they have little incentive to discuss a topic that might offend them. The accrediting agencies, like JC, CAP, and COLA, were probably in the best position to halt the downward slide of quality. But they faced competitive pressures, too. If CAP was too strict on POC devices and EQC, labs could and would switch to the laxer standards of JC. Finally, when CMS decided to push EQC through, it simply flexed its deeming authority powers and forced the agencies to adopt some form of EQC or lose its accreditation status. Responsibility for quality has also been spread out until everyone and no one is responsible. Customers talk about quality but make decisions based on cost. Manufacturers struggle to maintain a balance of the two, but cost is the overwhelming driver of their business. The EQC protocols insist that the laboratory director must weigh the "clinical and legal responsibility" against the EQC options, a tacit admission that EQC probably is neither clinically nor legally appropriate. Everyone wishes quality was better, but acts in ways that reduces it.

The Bubble Pops

It's hard to summarize the factors that brought the end to speculation. But as interest rates rose, it became harder to take out loans. Housing prices began to drop, which meant you could no longer sell your house to pay off the mortgage. As housing values began to drop further, the value of some houses dropped below the mortgages (known as negative equity). People began to miss, even default entirely, on their mortgage payments, as teaser rates reset, as impossibly large payments came due on people who had no means to pay. As defaults rose, more houses came back onto the market, depressing the pricing. It turned into a negative feedback loop - more colorfully put, a death spiral.

Wall Street made it worse. Banks and hedge funds were highly leveraged in mortgage-backed securities (i.e. they had made all these investments using lots of borrowed money). Now their loans were coming due and their investments increasingly looked like they weren't worth the paper they were printed on. As banks and firms "wrote down" the value of these investments (admitted the money they had invested was gone), they began to run out of money. Some went bankrupt, as we've seen. Others have been taken over by the government. As fear gripped the financial community, there wasn't enough money left for the usual, normal and necessary loans - even those that aren't as risky as subprime mortgages. And here we are, looking at hundreds of billions, probably trillions of money, that will be needed to fix the mistakes.

Will the quality bubble ever pop for the laboratory? If we continue to debase the rules and loosen the regulations, and if our institutions acquiesce in this drift, it's probable that we will fall outside the margins of safety. If we reach the point where testing QC once a month becomes an accepted practice, it's probable we will see bad test results impacting large numbers of patients. The Maryland General scandal, which was the case of an individual laboratory with debased quality practices, is merely an hors d'oeuvre compared to the systemic problems we could face. I believe healthcare is more resilient than Wall Street. We believe more in mission than mere profit. Our professionals have more scruples and commitment to good practices. But all of that means that the errors can grow larger, the bubble can get bigger, before it overwhelms the system. A quality crisis is not inevitable. All we need is leadership. And even then, leadership isn't necessary if enough voices are raised. Your voice. The voices of laboratory professionals. If the chorus is loud enough, the leaders and the market and maybe even the rules will follow.

Offered in the spirit of a previous essay, Enron and Quality