COVID ‘Immunity Passport’ No More Reliable Than a Coin Flip

But what if it’s 95% specific, meaning a 5% false-positive rate? Doesn’t seem too bad, right?

Well, that means that 5% of the 95,000 people who aren’t immune—4750 people—will nevertheless test positive.

Okay. Now we have a total of 9750 people who tested positive, of whom only 5000—just over 50%—are actually immune.

That immunity passport you got is no better than a coin flip.

The positive predictive value is what it is despite the low false-positive rate because the underlying prevalence of the disease is still low.

This misunderstanding of the difference between false-positive rate and positive predictive value may seriously endanger people.

This is my wife. She’s a brilliant surgeon.

On Friday, she got an antibody test as part of a clinical study.

She’s interested in the result, particularly if it’s positive. But she’s never had typical symptoms, and the prevalence of coronavirus in Connecticut is probably still south of 5%. For her, like most people, a positive result won’t be particularly reassuring.

“Okay,” you say, “but what if the false-positive rate of the test is even lower than 5%? What about 1%, or as Abbott claims with its antibody test, 0.4%?” I made this helpful graph since there is some favorable exchange rate between pictures and words.

As you can see, even with a really good test, if the prevalence in your area is low, you still have a good chance of not being protected. And, of course, there are a lot of tests going around.

Alexander Marson of UCSF and his team have been doing great work independently testing these antibody kits. Their results—it won’t surprise you—tend to be worse than what manufacturers report, with false-positive rates ranging from 8.4% for the Decombio test and actual 0% for the Innovita test, though small sample sizes limit the precision of these estimates.

So here’s the bottom line: Until there is good evidence that your local prevalence of coronavirus infections is somewhere north of 20%, using a positive antibody test as your ticket to not take the common-sense precautions we’ve all been taking is a recipe for disaster. It’s just the math. Please spread the word before widespread availability of antibody testing leads to a bunch of bad decisions.