December 31, 2008

Happy Holidays 2008 and a Hopeful New Year 2009

James O. Westgard, PhD

[Editor's note: this essay contains some discussion of politics]

As we near the end of December, it is -7 F on the thermometer and +31 inches on the snow gauge. The winter shows all indications of being a bad one, at least here in Wisconsin, but the next year shows many signs of hope in the US. We have new leadership in our government, a more centrist political disposition, (we hope) less partisanship in our political process, and a high priority for reform of our healthcare system. Many wonder if healthcare reform will be compromised by the financial and business crises facing the country, but I think such reform will be necessary as part of resolving those crises. Businesses and even the medical community are now supportive of fixing healthcare!

I am encouraged by the appointment of Tom Daschle as the Secretary for Health and Human Services and also as the “healthcare czar.” I don’t know much about Daschle, except that he comes from South Dakota, which makes him a neighbor and gives me some sense of his background and character. Although I’m originally from North Dakota, remember that at one time “Dakota Territory” included both North and South Dakotas, so anyone from Dakota is a neighbor who will befriend another Dakotan at any time or place. Likewise, I think Daschle will befriend all Americans in their need for better healthcare. I realize that there is little logic in that reasoning, but character and commitment will be the key ingredients of the new leadership.

I still worry about the need for improvements in the quality of laboratory services because the financial crisis will provide an ongoing rationale for minimizing the cost of laboratory testing at the possible expense of quality. The CLIA regulations went in the same direction as the US financial regulations, i.e., reduced or lenient regulation to aid business, which here includes the healthcare business as well as manufacturers and suppliers. Certainly CMS’s efforts to reduce QC through supposedly “equivalent QC” follows the pattern of reduced regulations. CMS’s support of “alternative QC” based on manufacturer’s risk assessment is another step in that direction. While we hear a lot about the need to review the regulatory process for financial institutions, there is no evidence that anyone understands that the same may be true for laboratories. Lax regulation of quality by CLIA may be turning our testing sub-prime.

I have a high respect for manufacturers' efforts to improve measurement technology, but there is always a need for independent review and assessment of that technology and its performance in routine applications. We rely on the FDA to approve new methods and analytic systems, but remember that the FDA works on the principle of “truth in labeling." The FDA primarily checks that the manufacturer has data to support the claims that are made, not that performance is acceptable for the intended clinical applications, which requires define goals for quality. The FDA has begun to advise manufacturers of “waived” testing devices to define a goal for analytical quality and to document its achievement, but that approach has not yet been applied to the 510k process for approval of “non-waived” tests. Let’s hope that they move in that direction soon.

Analytical quality will become more critical as programs for “Pay-for-Performance” (P4P) are implemented. Some of these are already underway and Medicare is taking a step in that direction by implementing P4P incentives for quality improvement. As we saw this year with the wave of "never event" non-payment policies, other healthcare payers willl follow Medicare's lead on this issue. We will see these P4P efforts evolve and expand in the next few years.

On the surface, P4P seems like a reasonable approach, but there is little research that shows it really works! Undoubtedly, laboratory tests will be included as measures of performance because they provide "hard numbers" upon which to judge performance. However, the use of laboratory tests in this way assumes that the test results are comparable from method to method and laboratory to laboratory and over time. Such comparability depends on traceability, i.e., the availability of reference methods and materials to provide a traceable chain between a laboratory method and “trueness.” Furthermore, such comparability in principle (traceability) needs to be monitored and documented by External Quality Assessment and Proficiency Testing surveys. Yet such surveys today show much method to method variation for many laboratory tests. We still have much work to do if laboratory tests are to provide reliable and rugged measures for such applications.

Thus, the need for analytical quality in laboratory tests is becoming more important in this age of evidence-based medicine and the advent of P4P programs. This reliance on quantitative quality for P4P lays bare the assumption that the quality of laboratory testing is adequate for clinical care. Even though most professionals outside the laboratory have little understanding of the actual performance issues, they seem convinced that they can base future payment decisions on those testing results.

I often find myself having to defend the need for analytical quality in today’s patient safety era (with its emphasis on pre-analytic and post-analytic errors). Having been singled out by Clinical Laboratory News as “one of the most outspoken critics of the EQC guidelines…” (CLN, November 2008, Risk Management for Clinical Labs), I’m sure some people are tired of hearing me speak out on this issue. But somebody needs to do it and I find myself having to take on that role given the absence of concern by our professional organizations, which is evidenced by their acceptance, if sometimes only grudging, of EQC and their support for AQC. It will be interesting to see what they have to say about P4P!

Yet there is reason for hope that analytical quality will receive the attention it needs! There’s nothing like putting a price tag on something to increase interest. I believe P4P will bring more attention to the issues related to analytical quality. Also, the CLSI process of developing the risk management guidelines is proceeding slowly, despite the fact that these documents are supposed to be on the “fast-track.” There are still opportunities for CLSI member organizations to review and comment on these guidelines before they become final. Healthcare, in general, will be under the microscope in the next few years and that will also bring to light the issues with quality in laboratory testing.

Ultimately, each of us will have to exert our influence if quality is to be achieved in laboratory testing and in healthcare. We each have different opportunities to do this, different ways of making our voices heard, and one of the most important is taking care of quality in our daily work. Quality in healthcare often happens because individuals make the effort to do the right thing right. Individuals often save the day for quality, in spite of the many difficulties in their daily work. That’s why you are such an important part of the process of achieving quality in healthcare!

The rewarding part of quality for those of us at Westgard QC is your continued interest and support. We know we have a loyal audience that is dedicated to providing quality laboratory services. Our biggest hope for the New Year is that we can continue to work with you and help you in your efforts to improve quality.

From the whole staff (and family) at Westgard QC, Seasons Greetings, Happy New Year, and Thank You for your continued patronage, support, and friendship.

December 23, 2008

Comments on "HbA1c for screening and diagnosis of diabetes?"

By Sten Westgard, MS

Dr. Westgard's recent essay on HbA1c for screening and diagnosis of diabetes and an accompanying post generated an interesting reply. Offered here in its entirety:

Jim, Sten,
 
I would like to comment on your article on HbA1c for screening and Diagnosis.  First, the comment by Sten that the tighter CAP criteria are not addressing the issue of bias.  It does indeed address both bias and imprecision because the CAP is using an accuracy base - the NGSP assigned values.  So if a method is either biased from NGSP or shows variability within or between labs, this will decrease the lab's chances of passing.  The CAP no longer uses peer group grading for HbA1c. 
 
I'm not sure what you mean by "scientific marketing" and if this should be taken as a negative comment.  Sure manufacturers would like to see HbA1c used for screening since this would increase use of the test.  But use of HbA1c for screening has been discussed for a long time and the discussion is based on very sound arguments.
 
The main reason to use HbA1c rather than fasting glucose is not for "convenience of any particular group".  It, in fact, will get people screened and diagnosed that may not otherwise be diagnosed.  There is still a large group of people who have undiagnosed diabetes.  This is why so many people (25% of those diagnosed) have complications at the time of diagnosis!  Using HbA1c will allow screening and/or diagnosis more readily e.g. at regular doctors' appointments where patients usually don't come fasting.  Furthermore, many physicians already use HbA1c but there are no specified cutoffs so it's hard to know exactly how it is being used and it may not be used appropriately. 
 
Another drawback of fasting glucose is that it has a large (much larger than HbA1c) biological variability.  And measurement of glucose isn't perfect either.
 
The screening cuoff for HbA1c of 6% is equivalent to 125 mg/dL AVERAGE GLUCOSE, not fasting glucose.  The equation (eAG =28.7*HbA1c-46.7) gives you a mean blood glucose, not fasting.   You can't easily equate a patients mean glucose with his or her fasting.
 
And I disagree that discussions of using HbA1c for diagnosis/screening are guided by market forces and not quality of care.  The main goal is to get more people with undiagnosed diabetes diagnosed.  It is not for the pockets of manufacturers or scientists. 
 
Randie
 

Randie R. Little, Ph.D.

NGSP Network Coordinator

Co-Director Diabetes Diagnostic Laboratory

Depts. Of Pathology & Anatomical Sciences and Child Health
University of Missouri School of Medicine
Columbia Missouri


Just to be clear, we meant no disrespect of the NGSP efforts in either the essay or the post. Indeed, they are doing far more to improve quality than most other organizations of a similar nature.

As they say, it's important to be able to disagree without being disagreeable. We welcome feedback, comments, and different points of view.

December 15, 2008

Sigma metrics in veterinary testing

By Sten Westgard, MS


I just wanted to draw your attention to a seminal paper that came out recently:

Quality control validation, application of sigma metrics, and performance comparison between two biochemistry analyzers in a commercial veterinary laboratoryAlison J Farr and Kathleen P Freeman, J Vet Diagn Invest. September 2008;20(5):536-44. (subscription required)

In this paper, Kathy Freeman, who has previously written a guest essay for Westgard Web, and her colleague Alison Farr apply Sigma metrics and QC design to common veterinary diagnostic tests. QC in veterinary testing is not a subject that is well covered in the scientific literature.

What's most interesting from our point of view is that in order to calculate Sigma metrics and perform QC Design on these tests, Freeman and Farr needed key information: quality requirements for veterinary tests for the major non-human species (cat, dog, horse). For the most part, the requirements don't exist, or at least a set of goals is not in widespread circulation. So Freeman and Farr did a survey review and made some assessments of test interpretation in their own laboratory. The result is the largest list of quality requirements for veterinary testing that we've ever seen. We excerpted the table, with Freeman's permission, here

It's a comprehensive paper, covering the method principles, decision levels, quality requirements, Sigma metrics, and QC Design with rule implementation. Highly recommended.

December 10, 2008

Glucose vs. HbA1c

By Sten Westgard, MS


Over on the main website, Dr. Westgard has a new essay discussing estimated average Glucose and the new push to use HbA1c methods for screening and diagnosis of diabetes. There have been two front page articles in Clin Lab News on these topics.

Two additional points worth making about HbA1c. First, the National Glycohemoglobin Standardization Program (NGSP) has set a goal of reducing the imprecision of HbA1c methods. So CAP proficiency testing criteria, for example, are going to be tightened from 12% down to 6% over the next four years.

Let's repeat that. The quality requirement is going to be reduced from 12% to 6% in the next four years.That's cutting the quality requirement in half. Wow.

That's real leadership and courage right there. That is using the power of regulation to force improvements in instrumentation. I know my experience of the laboratory world is limited, but I haven't seen that kind of agressive goal-setting before. Have you?

It will be very interesting to see how the diagnostic manufacturers respond to tightening quality requierments. Simply put, I don't think many of the methods on the market now can make the cut. They will not be good enough to hit a 6% target. ( Also, will the other PT providers follow suit, or will they keep their targets wide to try and scoop up customers? Here's where competition might work against quality. )

Second point. The article doesn't talk about a big issue in HbA1c methods: differences and bias between methods. For all the work on traceability, the methods in the field remain considerably different. There's a recent paper out in the Journal of Clinical Pathology, where Holmes et al seriously question the use of HbA1c testing:


“Our results also question the validity of using HbA1c test results as a measure of the quality of diabetes care.  Given the present state of the art for HbA1c testing, the proposed pay-for-performance use of these results could lead to an ironic scenario in which analytic methods that are biased low relative to the NGSP reference method are considered to reflect a high quality of care and secure a better level of reimbursement for the provider while at the same time promoting clinical inertia for the intensification of diabetes treatment and underestimating the risk of diabetic complications.”

“The gap between the level of analytic quality that experts consider necessary to obtain the maximum clinical usefulness from an HbA1c test and the level of performance presently provided by the methods used in the United States suggests that there is, indeed, cause for concern.  The shortcomings of contemporary HbA1c testing need to be more widely recognized by the end users of the results.  In addition, individuals and organizations involved in the design, manufacturer, performance, accreditation, and regulation of HbA1c testing need to devise and implement plans for improving the analytic quality of this important test.”

Holmes EW, Ersahin C, Augustine GJ, Charnogursky GA, Gryzbac M, Muttell JV, McKenna KM, Habhan F, Kahn SE. Analytic bias amoung certified methods for the measurement of hemoglobin A1c. Am J Clin Pathol 2008;129;540-547.

When you add pay for performance on top of the method bias problems, you've got a scary scenario. Just by switching to a new, lower-reporting method, clinicians and institutions could increase their compensation - without ever changing the quality of healthcare delivered to patients. That's not pay for performance. That's pay for method bias. 

December 09, 2008

Mentor of the Month: Don Wiebe

By Sten Westgard, MS


WiebePhoto Over at the AACC website, they have a "Mentor of the Month" section, where they focus on a distinguished scientist who has made contributions to helping the next generation of laboratorians.

I'm happy to point out that the December 2008 Mentor is another one of the "Wisconsin Mafia": Don Wiebe, Ph.D., DABCC, FACB. They also posted an interview with him. You even get a flavor of Don's warmth and humor, even though there aren't any of Don's famous jokes in the article. [Note: I'm using the Google cached versions of those pages, since the AACC changes the Mentor every month.] 

Don has worked with Dr. James O. Westgard for decades - and taught him a thing or two about the laboratory. Recognition of Don's contributions to clinical chemistry is long overdue and well-deserved. He's a great scientist, laboratorian, teacher and friend.

Congratulations to Don. And we encourage you to read more about him.

December 03, 2008

Failure is an option?

By Sten Westgard

On November 4th, the Joint Commission issued an interesting press release, titled "Lab Decisions Will No Longer Affect Hospital Decisions."

The specific language of the press release stated:

"Beginning January 1, 2009, under new Joint Commission policy, laboratory accreditation decisions will no longer immediately impact hospital accreditation decisions."

I have subsequently seen comments on a listserve wondering if it's now acceptable for JC-accredited hospitals to have laboratories that fail inspections. The simplistic interpretation of this rule is that laboratory problems no longer impact the hospital. Hospitals can keep running regardless of the state of their laboratory.

But that's not really the case.

I contacted Megan Sawchuk, Associate Director of the Standards Interpretation of the Joint Commission. She elaborated on the new policy and cleared up any ambiguity:

"The December 2008 Perspectives announcement regarding laboratory accreditation decisions has two important elements. One, the Accreditation Committee voted to eliminate the automatic, direct weight of an adverse decision in the laboratory on the hospital. And two, an adverse laboratory decision from The Joint Commission, CAP or COLA will be added to the hospital's Priority Focus Process (PFP) data. PFP data is presently used by The Joint Commission to monitor the hospital's overall performance and prioritize the timing of their unannounced survey in the 18-39 month window. Thus, an adverse decision in the laboratory will significantly increase the likelihood of an earlier hospital survey to assess compliance at the organizational level.

"By using this method, the hospital decision is based on their actual overall performance with consideration of that of the laboratory. This is an improvement over the current process of automatically applying an adverse laboratory decision to the hospital, which assumes an overly simple relationship between the two integrated but separate entities. Noncompliance in the laboratory is often associated with poor performance in the overall organization, but not always. This method also maintains the integrity of the the laboratory as an essential service in the hospital's accreditation decision process."

To be clear: a failing laboratory will still take down a hospital with it. The downward spiral to revocation of accreditation may not be as fast as it used to be. But the usual regulatory process takes time in any case. Inspections generates citations, which require responses, which may then generate additional inspections, additional responses, etc. Immediate action happens very rarely. The Joint Commission retains all the policies and tools they need to come down hard on a lab and hospital. This new policy just gives them a little more latitude.

One last thing: this is a clear admission that many laboratories in America have significant problems. If laboratories were operating perfectly (or even just in compliance) and there weren't any worries about them, we would have no need to decouple their accreditation decisions from the hospitals.    

November 13, 2008

What's New: November 2008

New Download: Sigma-Metrics Tool (and audioconference presentation)

On September 4th, 2008, Dr. Westgard spoke as part of the AACC audioconference on "New Directions in Laboratory QC" -  which was subsequently quoted in the Clin Lab News article on Risk and CLIA.  We are pleased to make available Dr. Westgard's complete presentation, as well as two Sigma-metrics Charts and instructions on how to use them in QC Design.




Westgard Sigma Analysis: A new direct HbA1c Method

At the 2008 AACC/ASCLS convention, a poster was presented for a new direct enzymatic assay for %HbA1c. How does it stack up against HPLC and immunoassay methods? How do you judge a method when you've got multiple comparison methods and multiple quality requirements?




Interview: Dr. R. Neill Carey (A brief introduction to EP 15)

We were fortunate to get R. Neill Carey, PhD, the chair of the CLSI EP 15 committee, to present and 
explain that new standard at the Chicago Method Validation workshop. He also contributed a chapter to the new Basic Method Validation manual on the same topic. But for those who have never heard of EP 15 before, we conducted a short interview with Dr. Carey. This short introduction to EP 15 may pique your interest in this new guideline.




Thinking about Three Sigma: 2 thoughts on troublesome performance

In a previous lesson, we discussed some possible actions to take when the Sigma-metric for a method is higher than Six. But what about those methods with low Sigma-metrics? What do you do when Sigma analysis delivers bad news?




November 11, 2008

Talking about a Training Trilogy in MLO

By Sten Westgard, MS

Exec1

Dr. Westgard is the featured executive profile in the November 2008 issue of Medical Laboratory Observer. The editors asked him how he developed three popular training references: Basic QC Practices, Basic Method Validation, and Basic Planning for Quality.

November 05, 2008

What progress will we make?

by Sten Westgard, MS


As regular readers of Westgard Web know, Dr. Westgard has not been shy about making political comments in elections past. 

This election cycle, we didn't make many political comments on the blog or the website. It seemed to us that the candidates and the parties did just fine by themselves.

Over the past few days, we have received many emails of hopes and congratulations from people all around the world who have watched the US Presidential race. We thank you for those kind messages. It is a testament to the interconnectedness of the world that we all have a stake in each other's politics.

We celebrate the election of the new President. But the celebration can be only temporary. The real work remains before us:

"This victory alone is not the change we seek - it is only the chance for us to make that change. And that cannot happen if we go back to the way things were. It cannot happen without you."
President-Elect Barack Obama, November 4th, 2008, victory speech

Talking about CLIA and Risk in Clin Lab News

by Sten Westgard, MS
Updated 11/7/08


Dr. Westgard is quoted in the November 2008 issue of Clinical Laboratory News
in their report on "Risk Management for Clinical Labs" [subtitle: As CLIA Turns 20, A New QC Paradigm Lies Ahead for Labs] . Here's the most interesting quote:

One of the most outspoken critics of the EQC guidelines has been James Westgard, PhD, FACB, emeritus professor in the department of pathology and laboratory medicine at the University of Wisconsin Medical School and president of Westgard QC, Inc. “So what are equivalent QC procedures? Equivalent in performance? Equivalent in detecting medically important errors? Equivalent in assuring the necessary desired quality is achieved?” asked Westgard at a September 4 AACC audioconference called “New Directions in Laboratory QC: EQC, Alternate QC and Risk Assessment.” “No, it’s none of the above. It’s just a name. It has nothing to do with any practical meaning in terms of what we think of equivalents. It’s just a name,” he said.

Back a few years ago, there were more critics of Equivocal QC, for example CAP, but then their deeming status came up for review. Now all the deemed providers have implemented equivalent QC options. CMS flexed its muscle, and now there are fewer people to be outspoken.


The whole article(pdf) is worth a read. Again, one of the most interesting details is how the FDA might handle Manufacturer Risk Information - and use it against manufacturers in the event of a problem:

Steven Gutman, MD, director of FDA’s Office of In Vitro Diagnostic Device Evaluation and Safety indicates that FDA will not directly review a manufacturer’s risk assessment material. “Because they’re providing straight-forward information on risks and risk management, we would view that as an acceptable practice, and there might be various formats for them to do that,” Gutman explained. “It might be technical bulletins, it might be on their websites, but it wouldn’t be a part of FDA-sanctioned labeling unless they violate FDA regulations, in which case our compliance program would become concerned.” Gutman said the FDA would look to see if companies were going overboard with explicit or implicit claims about their products, reviewing the information only if the agency thought there was a problem.


As we noted before in What are the Risks of Communicating Risks?, this is a tough spot for a manufacturer.

Update: You can now download Dr. Westgard's complete presentation from the September 4th, 2008 audioconference. The pdf download includes his slides, two Sigma-metrics QC Design charts, and instructions on how to use them.





My Photo
Blog powered by TypePad

Pages

Paying the Bills