Better Evaluate COVID-19 Test Performance using Bayes Factors

Over the past couple weeks, friends and family have asked me questions about the “accuracy” of different COVID-19 tests. This video by #3Blue1Brown explains why accuracy isn’t really what you’re wanting to know. A better question would be: “Given a positive test result, what’s the probability I actually have COVID-19?” or, in his terms, you want to update your assumed likelihood of having COVID-19 once you get a test result. Test accuracy only determines how your chances of having a disease are updated but does not by itself determine your chances.

LumiraDx Antigen-Based Rapid Test

Let’s take LumiraDx’s SARS-CoV-2 Antigen test as an example. (This is a common antigen-based rapid test being offered at CVS Minute Clinics near me. It offers results in about 15 minutes using a nasal swab.) They report a sensitivity of 97.6% and a false positive rate of 3.4%, thus the Bayes Factor (aka Likelihood Ratio) is about 28.7.

Note: LumiraDx reports its Sensitivity as Percent Positive Agreement and Specificity as Percent Negative Agreement. This verbiage is due to their rapid test results being validated in a lab as part of the development of their system.

With a prevalence of COVID-19 in North Carolina at about 10% (plug in a more accurate number here for your local situation), the probability of having COVID-19 with a positive result from this LumiraDx antigen-based rapid test example would be about 76%.

10% prevalence can be rewritten as a 10:90 ratio between people who have COVID-19 vs. not. Then, multiplying this ratio by the Bayes Factor from before, we get an updated ratio of 287:90 or an updated percentage of 287/(287+90) or 287/377 ≈ 76.1%.

What about Negative Results?

Alternatively, one might ask the question, “Given a negative test result, what’s the probability I actually have COVID-19?” We can recalculate the Bayes Factor accordingly.

Then, multiplying this ratio by the Bayes Factor from above, we get an updated ratio of 0.25:90 or an updated percentage of 0.25/(0.25+90) or 0.25/90.25 ≈ 0.3%. So, the probability of having COVID-19 with a negative result from this LumiraDx antigen-based rapid test example would be <1%.

What are My Odds…Generally?

The odds of having COVID-19 given a positive test result can be written as follows:

Alternatively, the odds of having COVID-19 given a negative result can be written similarly:

Where the odds of having COVID-19 (normally called the “prior” in Bayesian probability) may be simple prevalence or may be a more informed guess based on other factors like an exposure or symptoms.

While this was a long discussion into wrapping our heads around how “good” a test is for COVID-19 or other conditions, I think this idea that tests update our chances of having the disease really helps to better frame the question at hand. Give this calculation a try with your own local prevalence statistics to make a more informed decision between your COVID-19 testing options.

Stay safe, stay home, and wear your mask!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Colby T. Ford, PhD

Colby T. Ford, PhD

Cloud genomics and AI guy and aspiring polymath. I am a recovering academic from machine learning and bioinformatics and I sometimes write things here.