Better Evaluate COVID-19 Test Performance using Bayes Factors
Over the past couple weeks, friends and family have asked me questions about the “accuracy” of different COVID-19 tests. This video by #3Blue1Brown explains why accuracy isn’t really what you’re wanting to know. A better question would be: “Given a positive test result, what’s the probability I actually have COVID-19?” or, in his terms, you want to update your assumed likelihood of having COVID-19 once you get a test result. Test accuracy only determines how your chances of having a disease are updated but does not by itself determine your chances.
LumiraDx Antigen-Based Rapid Test
Let’s take LumiraDx’s SARS-CoV-2 Antigen test as an example. (This is a common antigen-based rapid test being offered at CVS Minute Clinics near me. It offers results in about 15 minutes using a nasal swab.) They report a sensitivity of 97.6% and a false positive rate of 3.4%, thus the Bayes Factor (aka Likelihood Ratio) is about 28.7.
Note: LumiraDx reports its Sensitivity as Percent Positive Agreement and Specificity as Percent Negative Agreement. This verbiage is due to their rapid test results being validated in a lab as part of the development of their system.
With a prevalence of COVID-19 in North Carolina at about 10% (plug in a more accurate number here for your local situation), the probability of having COVID-19 with a positive result from this LumiraDx antigen-based rapid test example would be about 76%.
10% prevalence can be rewritten as a 10:90 ratio between people who have COVID-19 vs. not. Then, multiplying this ratio by the Bayes Factor from before, we get an updated ratio of 287:90 or an updated percentage of 287/(287+90) or 287/377 ≈ 76.1%.
What about Negative Results?
Alternatively, one might ask the question, “Given a negative test result, what’s the probability I actually have COVID-19?” We can recalculate the Bayes Factor accordingly.
Then, multiplying this ratio by the Bayes Factor from above, we get an updated ratio of 0.25:90 or an updated percentage of 0.25/(0.25+90) or 0.25/90.25 ≈ 0.3%. So, the probability of having COVID-19 with a negative result from this LumiraDx antigen-based rapid test example would be <1%.
What are My Odds…Generally?
The odds of having COVID-19 given a positive test result can be written as follows:
Alternatively, the odds of having COVID-19 given a negative result can be written similarly:
Where the odds of having COVID-19 (normally called the “prior” in Bayesian probability) may be simple prevalence or may be a more informed guess based on other factors like an exposure or symptoms.
While this was a long discussion into wrapping our heads around how “good” a test is for COVID-19 or other conditions, I think this idea that tests update our chances of having the disease really helps to better frame the question at hand. Give this calculation a try with your own local prevalence statistics to make a more informed decision between your COVID-19 testing options.
Stay safe, stay home, and wear your mask!