Posted by Sten Westgard, MS
Earlier this month (July) I came across a series of revealing posts on a listserv about the quality of glucose meters. For me, it raised the question, just what defect rate is acceptable at the point of care?
What level of defect rate do you believe is being seen at the point of care? the answer (after the jump) might astonish you...
For those of you who frequent the Westgard website, this may not come as much of a surprise, but the first defect rate is just 3.4 defects per million opportunities, or Six Sigma (on the short-term scale). This has long been held as the goal for performance of processes, however elusive and hard to achieve.
The other numbers like 1%, 5%, and 10% are more commonly used allowable defect rates, which are frequently just rules of thumb. They correspond to about 3.8, 3.1 and 2.8 Sigma, respectively. The Sigma scale shows us that what we commonly think of as "acceptable" error rates are in fact, from an industrial perspective, unacceptably high.
Now as for those very high defect rates, those represent device failure rates that have been reported on the AACC Point-of-Care listserve. These represent real-world failure rates of glucose meters. I'm going to quote these rates in context, but the different names of the glucose meter manufacturers will be withheld to protect them from shame:
"As I am packing up 13 Glucose Meter X’s and 2 wireless totes for return and replacement this month, I am wondering if other places that use the Glucose Meter X are experiencing as many malfunctions as I am....Too bad X doesn't have a 'Costco' return policy, as long as you have a receipt.
"The problems that happen most often are:
Scanner not working
Broken dock on meter
Strip port broken
Broken Touch screen
Cracked meter bodies
"The time needed to document the problems, issue the replacement meter to the unit, and the expense of performing linearity on the meters that are sent back is starting to take a toll. We purchased 150 meters and 116 have been returned so far and we went 'live' last June. That's 77% returned. WOW!
"I am a solo POC Coordinator in a very busy 500 bed hospital and my time would be better spent on other issues throughout the other POC instruments within the hospital."
Here's a response to this first post:
"Glucose meter X has been 'live' here since April 2013. Of the 123 meters that have come in the door, 56 have been returned for various reasons (46%). The number one reason: cracked casing. Laser outages and broken battery holders (caused by swollen batteries) tie as the number two reason. My running joke about cracked casings is ‘Are you using this device for batting practice?’
"Previously, I worked with a competitor's meter at an academic medical center (lot more meters, lot more patient + QC testing). I recall the same return rate with cracked casing and fluid infiltration (manifested by all kinds of electronic issues) being the number one and two causes for distress. I took comfort in my visits to the Biomedical shops where I saw lots of other medical equipment that was either under repair or waiting for replacement."
And then another POC coordinator shared her sympathy:
"We see all those issues too. If you haven't bought the extended warranty, you better start working on it. In our first 2 years, our return rate was almost 40% each year so we purchased the extended warranty. Even though this year's return rate is looking slightly better, I still think it would be more expensive and more time consuming to order new meters. Plus then your totes and docks are covered on that warranty. Without the warranty, the broken hardware pieces are just paperweights....there is no repair shop for Glucose meter X meters."
And still another POC coordinator chimes in:
“We've replaced 125 this year for an assortment of reasons. We're pretty good at cleaning the strip ports but can't get them all to work after cleaning. The upgrades seem to have helped with the scanning issues but we're still replacing from our original 2013 fleet of about 270. We replaced 100% of our original meters because the [component y] fell off. X validated both shipments for us. The bulging batteries have also caused the clips on the back of the meters to break. We have a standardized return/replacement process built into our routine work and have just come to expect it. With a weekly volume of roughly 14,000, it's inevitable because of the high use. We always have extras on hand and replace them out on a daily basis.”
For those of you who are curious, a 40% defect rate corresponds to 1.8 Sigma; a 77% defect rate corresponds to about 0.7 Sigma; and a 100% defect rate is essentialy Zero Sigma.
Is it any wonder that the FDA is considering making more demands on glucose meters, given this level of performance? If anyone is wondering why the FDA doesn't want glucose meters to be used for "off-label" purposes, look no further.
Place that error rate in context:
- If your core lab methods were failing 40, 77 or 100% of the time, would your lab still be in operation?
- If you cell phone calls were dropped 40, 77 or 100% of the time, would you still use this phone company?
- If your airline lost 40, 77 or 100% of all its checked luggage, would you fly this airline?
- If your surgical procedures had complications 40, 77 or 100% of the time, would your hospital still be operation?
We seem to have grown accustomed to an extremely high level of error and defects with waived and point-of-care devices, a rate that would be considered
And again, while this is an example of waived devices (which aren't mandated to perform any QC other than what is required by the manufacturer), this is part of the overall point-of-care sphere. And this is the area where we're supposed to be starting to implement risk-based quality control plans (IQCPs).
If you have a 40, 77 or 100% defect rate in a critical part of your POC testing, how would you rate your risk? How would you justify reducing your QC frequency to just once a week or once a month in a situation like that?