THE BLOG

How A 99% Accurate Medical Test Can be Wrong Most Of The Time

02/04/2015 8:12 AM IST | Updated 15/07/2016 8:25 AM IST
NEW! HIGHLIGHT AND SHARE
Highlight text to share via Facebook and Twitter
SIA KAMBOU via Getty Images
A laboratory technician at the AIDS Research Center of the Treichville hospital in Abidjan works on blood samples of people living with HIV on September 13, 2013 after the launch in the Ivorian capital of the Unitaid-funded project OPP-ERA, which aims at improving HIV monitoring through more affordable access to HIV viral load testing (VLT) for HIV patients and early infant diagnosis. Ivory Coast is the first country in Africa to join the two-year project, implemented by a consortium of French partners. AFP PHOTO/ SIA KAMBOU (Photo credit should read SIA KAMBOU/AFP/Getty Images)

The news recently featured the heart-wrenching story of a man (a skilled professional to boot) who reportedly believed he and his family were HIV positive (HIV+), tested positive, and then started seeing symptoms everywhere, including in his family. He tragically believed a family murder-suicide pact was the best option, and almost succeeded. His wife and children died, but he survived. Then, via the police he was re-tested, and this time the test came back negative. Was the original test faulty? NO!

Let's all understand how a test that is, say, 99% accurate can show an incorrect result most of the time.

More than just a misunderstanding of statistics in this case, there were also issues of information and misinformation (isn't the Internet wonderful?). At the same time, those who should have known better (medics and technicians) perhaps didn't do what they should have.

Before you think statistics are dry, abstract, or even worth skipping - realise they can save your life! It's a lack of understanding of even the basics by most people, which makes it easy for others to manipulate the meaning of statistical data. This led Mark Twain to famously comment "Lies, damned lies, and statistics." It's not that the statistics necessarily lie, but we have to understand what the numbers are, and what they do and don't mean.

"Before you think statistics are dry, abstract, or even worth skipping - realise they can save your life!"

Most people think about accuracy as the important thing. It turns out life is not so simple. It's not the accuracy only, it's also the direction of the results that matter, i.e., is this a false negative, or a false positive? While our judicial system is geared towards avoiding false guilty verdicts, and asks for irrefutable proof (..."The law holds it better that ten guilty persons escape, than that one innocent party suffer"), in health a false negative can be far worse than a false positive. Someone who is actually sick but thinks they are well might not just miss treatment, they may unknowingly spread disease.

Accuracy of tests vs. accuracy of results

Coming back to HIV, say we have a total population of 10,000 people. And we have a diagnostic test that is 99% accurate in each direction. The important missing ingredient is how many people are actually HIV+ (termed the underlying prevalence)? If it is 1%, it means in reality only 1% of the population is HIV+. If we test everyone, regardless of the results, then the underlying reality is out of 10,000 people, 100 people (1%) are truly positive and 9,900 are truly negative.

Of the 100 true positives, the testing is 99% accurate, so 99 people will show up as positive, and 1 person will show up as negative, which is a false negative. Of the 9,900 true negatives, 1% are incorrect in the test results, and show up as positive (false positive) = 99 people. The remainder are true negative = 9,801.

"[A]ny HIV+ test finding should be taken with not a grain of salt but a rock of salt - it's more likely to be a false positive (assuming a 99% accurate test)."

Putting all this together, the negatives seen in the tests are 1 (false) + 9,801 (true) = 9,802, which is overwhelmingly correct, with only 1 out of 9,802 as a false negative. However, the positives seen in tests are 99 (false) + 99 (true) = 198, of which 50% (!) are false positive.

So how can the results be even more misleading? If the underlying prevalence is less than the error rate of the test, then the false positives will outweigh the true positives. HIV is estimated to be prevalent in only 0.3% of India's population (on average), so any HIV+ test finding should be taken with not a grain of salt but a rock of salt - it's more likely to be a false positive (assuming a 99% accurate test). Congratulations - you've now understood Bayes' Law of Statistics, a key component of college courses on statistics. You would also realise that there are intended limitations of statistics.

Addressing the spread of misinformation

So what can be done to prevent future tragedies? After all, if we all only knew much more, wouldn't the world already be a better place? Leaving aside individuals, we have three sets of actors where we can change things, doctors, diagnostic labs and the government.

"All testing facilities and labs should provide a handout (designed by external experts) and basic counselling about the result accuracy (and not just test accuracy)."

What doctors and labs can do: Repeating tests helps cut down both types of false results (and many tests, not necessarily for HIV, have different error rates for negative versus positive). For HIV, doctors normally ask for a second test, often with a more accurate (but expensive) test. While they might be able to convince the individual, a lab may not, since people might just think that the lab wants them to spend more money on a repeat or alternate test. Unfortunately, with the rise of standalone labs, many individuals go in for blood tests on their own, and attempt not just self-diagnosis, but even self-medication, which can be dangerous.

All testing facilities and labs should provide a handout (designed by external experts) and basic counselling about the result accuracy (and not just test accuracy). Because of the unique nature of infectious and stigma-ridden diseases like HIV/AIDS, we should consider regulating at least portions of this process. Who knows how many people have been traumatised, stigmatised, ostracised, or even killed due to a faulty diagnosis of HIV+?

Harnessing the power of the internet:The Internet, which played a role in the person's faulty information, should be harnessed to raise awareness and prevent misinformation from spreading. Before longer term efforts on prevention, management, etc., the reality of testing accuracy should be the first result one gets when searching for symptoms or diagnostics. This needs to be easy to follow, in all languages.

"The Internet and social media can spread rumours and lies. The answer to such problems isn't censorship, but letting the truth rise to the top."

Just like in the US where people trust the CDC or the Mayo Clinic, we might have such a website hosted by AIIMS (and mirrored by state and central health departments). It should be set up or supported by the government to be the go-to website for such things, both purposely (people know about it and try and find it) as well as organically (the first hit on a Google search).

There is lots of misinformation out there, not just about HIV but many other diseases, not to mention best practices for all aspects of life. In Africa, there was a widespread myth that having sex with a virgin could cure AIDS! Without effort, the Internet and social media can spread rumours and lies. The answer to such problems isn't censorship, but letting the truth rise to the top. HIV cannot be cured, but it can be managed through drugs. It need not be a death sentence. A simpler starting place should be accurate diagnosis for HIV and other diseases.

Dr. Rahul Tongia is not a medical doctor, but a scholar of technology and policy, and a Fellow at Brookings India and Adj. Professor at Carnegie Mellon University. All views are personal.

Like Us On Facebook |
Follow Us On Twitter |
Contact HuffPost India

MORE:

More On This Topic