Comparing MU to Ricos Desirable CV: Like comparing an apple to basketball

Posted by Sten Westgard, MS

I was looking over a recent study of a compact blood gas analyzer conducted over multiple sites, multiple cartridges, multiple days, etc. In other words, this was a study that was doing a very thorough job.

Then I reached a section where suddenly it all fell to nonsense: measurement uncertainty.

So they calculated measurement uncertainty, but then they compared the results against Ricos desirable imprecision performance specifications.

Huh?

There is a big problem with using "Ricos goals" as benchmarks for measurement uncertainty.

1. The Ricos goals are designed to be used with the Total Error model, not measurement uncertainty. The Ricos goals were never designed to be used as benchmarks for measurement uncertainty.
2. By using only the desirable CV specifications of Ricos goals, this ignores the bias of the methods. Since there are a proliferating number of models to calculate uncertainty, many of which incorporate uncertainty of bias, this is either forcing the bias and imprecision uncertainty to be compared against an benchmark meant exclusively for imprecision, OR this is entirely ignoring bias. Which is a convenient, if entirely unrealistic, validation technique. (If only we could pretend imprecision didn't exist as well, think of how easily we could validate and verify all of our methods...)

More broadly, this shows how metrologists often talk out of both sides of their mouth. They decry the use of total error and any associated performance specifications from total error, but when it comes to setting performance specifications for measurement uncertainty, they fall silent. Despite decades of extolling the virtues of measurement uncertainty, metrologists have never developed a widely accepted, commonly implemented set of target measurement uncertainties. Instead, when they are pressed to compare a calculated measurement uncertainty (no matter which model has been used to calculate it, and there are now so many to choose from), they fall back on the very model they have been criticizing: they use total error goals.

This is simply yet more evidence that measurement uncertainty is not a useful indicator for the routine laboratory. There is no demand for the statistic, except by inspectors. There is no uniform way to calculate it, so many calculations of MU are uncertain when compared to other MUs. And to this we add, there are no real standards for MU. So once you have calculated MU, all you can be certain of is that you will be uncertain about whether or not your measurement uncertainty is acceptable.

Well done, metrologists. Well done.