Better Evaluate COVID-19 Test Performance using Bayes Factors

Over the past couple weeks, friends and family have asked me questions about the “accuracy” of different COVID-19 tests. This video by #3Blue1Brown explains why accuracy isn’t really what you’re wanting to know. A better question would be: “Given a positive test result, what’s the probability I actually have COVID-19?” or, in his terms, you want to update your assumed likelihood of having COVID-19 once you get a test result. Test accuracy only determines how your chances of having a disease are updated but does not by itself determine your chances.

LumiraDx Antigen-Based Rapid Test

Let’s take LumiraDx’s SARS-CoV-2 Antigen test as an example. (This is a common antigen-based rapid test being offered at CVS Minute Clinics near me. It offers results in about 15 minutes using a nasal swab.) They report a sensitivity of 97.6% and a false positive rate of 3.4%, thus the Bayes Factor (aka Likelihood Ratio) is about 28.7.

Note: LumiraDx reports its Sensitivity as Percent Positive Agreement and Specificity as Percent Negative Agreement. This verbiage is due to their rapid test results being validated in a lab as part of the development of their system.

With a prevalence of COVID-19 in North Carolina at about 10% (plug in a more accurate number here for your local situation), the probability of having COVID-19 with a positive result from this LumiraDx antigen-based rapid test example would be about 76%.

10% prevalence can be rewritten as a 10:90 ratio between people who have COVID-19 vs. not. Then, multiplying this ratio by the Bayes Factor from before, we get an updated ratio of 287:90 or an updated percentage of 287/(287+90) or 287/377 ≈ 76.1%.

What about Negative Results?

Alternatively, one might ask the question, “Given a negative test result, what’s the probability I actually have COVID-19?” We can recalculate the Bayes Factor accordingly.

Then, multiplying this ratio by the Bayes Factor from above, we get an updated ratio of 0.25:90 or an updated percentage of 0.25/(0.25+90) or 0.25/90.25 ≈ 0.3%. So, the probability of having COVID-19 with a negative result from this LumiraDx antigen-based rapid test example would be <1%.

What are My Odds…Generally?

The odds of having COVID-19 given a positive test result can be written as follows:

Alternatively, the odds of having COVID-19 given a negative result can be written similarly:

Where the odds of having COVID-19 (normally called the “prior” in Bayesian probability) may be simple prevalence or may be a more informed guess based on other factors like an exposure or symptoms.

While this was a long discussion into wrapping our heads around how “good” a test is for COVID-19 or other conditions, I think this idea that tests update our chances of having the disease really helps to better frame the question at hand. Give this calculation a try with your own local prevalence statistics to make a more informed decision between your COVID-19 testing options.

Stay safe, stay home, and wear your mask!

Computational Biomathematician and Cloud AI Guy. I research machine learning and genomics and I sometimes write things here.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store