Posted by Sten Westgard, MS
The last few weeks have seen an explosion in stories about Theranos and its current troubles, and not a little schadenfreude from the laboratory diagnostic community. I suppose it's understandable that when a unicorn start-up boasts it will wipe out an entire industry, that industry takes a certain amount of glee when the unicorn stumbles.
But it's too soon to conclude Theranos is just a bluff, a snake-oil of a diagnostic company promising things too-good-to-be-true while raking in hundreds of venture capital millions. According to most of the stories, Theranos is still valued at $9 billion, about the same as the actual value of Quest Diagnostics or LabCorp.
This is a question that could easily have been solved by data, a long time ago. And could still easily be answered by data. If Theranos wants to prove it's a game-changer, show us the method validation.
In other words, method validation still matters. If you have a new method, new device, you still need to prove that it performs according to the necessary requirements of patients and customers. It doesn't matter if you've got Henry Kissinger on your Board of Directors - what matters are core concepts like imprecision, bias, and dare I say, the quality required by the test.
Because of all the problems, Theranos actually only has 1 test cleared by FDA. All the other tests must be performed the "old-fashioned" way. No Edison machine. No Nanotainer. No revolution in testing.
I almost wish that Theranos hadn't pulled its testing off the Edison machines. Then we could easily mount a posse to head to Arizona, where the Walgreens Theranos clinics are. If we got a good number of patient volunteers to get tested at Theranos and then tested at a local hospital, we could build an informal comparison of methods experiment. Rather than the anecdotal evidence, we could do some really regression analysis to figure out what level of bias exists between the Edison machine and the current generation of analyzers.
Theranos is struggling with a problem of its own making. Evidently when the FDA inspected the Theranos laboratory, they found that
have not monitored your Quality Management System through internal quality audits; you had no documented internal quality audit schedule to monitor your Quality Management System until after the start of this inspection."
At least one of Theranos' devices "was not validated under actual or simulated use conditions."In one of the forms, the FDA documented a number of complaints, including that a Theranos device (it's not clear in the form, due to redactions, which one) had a design evaluation that didn't ensure the device "conforms to defined user needs and intended uses." A Theranos spokesperson confirmed that the device referred to in this document is the Nanotainer, which is used to contain the blood samples from a finger prick. Additionally, the Nanotainer "was not validated under actual or simulated use conditions," and the risk analysis for the device hasn't been adequately documented. In another form from the same inspection period, FDA said that there were inadequate procedures for logging customer complaints, and that complaints that Theranos' technology didn't work weren't reviewed or investigated."
The above is an excerpt from http://www.theverge.com/2015/10/27/9621026/fda-theranos-lab-quality-inspection-elizabeth-holmes
Some of the failures noted above are Quality Systems 101. These are novice failures that even the smallest CLIA lab would have already understood and put into place.
At some point, I hope Theranos releases a true method validation study of the Edison Machine. They have been promising this - but the delay suggests that they hadn't done the study already (again, why wasn't this data already available ?) Then, and probably only then, will we have enough information to make an objective assessment of the quality of this new diagnostic offering.
Some other links that might be useful for those who haven't been following every twist and turn: