2

In the Czech Republic there is now mandatory testing for factory employees.

Anecdotally I am now being told by people I am acquainted with, who run medium sized firms here, that their employees are getting about 40 per 1000 rapid antigen positive test results.

These are then sent for PCR tests.

About 1 or 2 of those are coming back with PCR positive results.

So it's looking like about 20 or 40 to 1 antigen tests are false positives.

What's going on here?

0

2 Answers 2

4

The ratio of false positives to true positives (as well as overall "accuracy") is a function of the underlying rate in the population.

Let's imagine you have a test that has 95% specificity. Specificity is the number of negatives that you correctly identify as negative. If you test 1000 negative samples with a 95% specificity test, you will find 95% of them correctly return negative, and the other 5% (=50 tests) return positive. In this case, because we've tested an entirely negative population, 100% of the positive tests are false positives. You will always get this result when you test an entirely negative population, even if your test is 99.999% specific: if you test on entirely negative samples, 100% of the positives are false positives by definition.

Let's instead consider testing a combined population of 500 positive and 500 negative. For simplicity let's assume the test is 100% sensitive while still being 95% specific. 100% of 500=500 positives will test positive, and 5% of 500=25 negatives will test positive. You have a total of 525 positive tests, and only 25/525 (<5%) of these are false positive.


Sensitivity and specificity are a trade-off according to your decision threshold, so if you want to have fewer false negatives it's going to cost you in more false positives. For antigen screening followed up by PCR, the goal is to set a threshold for the antigen test that results in few false negatives, to avoid infected people from spreading to their coworkers. However, you expect you are going to get a lot of false positives that way, which is why the strategy is to then follow up on these tests with PCR. A combined positive on the antigen test + follow-up PCR test is much less likely to be a false positive.

The purpose of the antigen test here is less to identify true positive cases, and instead it is to add a rapid way to isolate potential positives and to make the most efficient use of the PCR test.

You can use Bayesian statistics to better interpret results after a test if you start out with a prior understanding of expected ratios of positive to negative cases in the population you're testing.

3
  • Why are you "expected" to get more false positives from an antigen test than from PCR when most literature on the matter says that antigen tests are more liable to produce more false negatives relative to PCR? As far as I can tell, the only way what the OP says happens happens when the antigen tests are badly done (cross-contamination of samples etc., which can of course boost the false positive rate of any kind of test).
    – Fizz
    Mar 21, 2021 at 8:38
  • Also, as far as I can tell the whole discussion of prevalence in the population is a red herring when comparing two tests against each other. I can't see any scenario in which prevalence (alone) inverts the apparent sensitivity of two tests relative to each other.
    – Fizz
    Mar 21, 2021 at 8:54
  • 1
    @Fizz It's definitely not a red herring, it's absolutely critical to understanding what's going on here. The reason it's critical is because the PCR tests are only being used here after a positive antigen test.
    – Bryan Krause
    Mar 21, 2021 at 18:32
1

Bryan is correct, especially in his comment in the case of 0% true/prior prevalence

40/1000 = 4% positive. 1-2/40 = 2.5-5%.

But even in such a conditioned test scenario, in which the 2nd test is only applied to the positive-reporting sample of the first test, a decent test (e.g. 90% sensitivity, 95% specificity) will actually converge pretty quickly (i.e. "disagree with itself" a lot less).

E.g. if the true prevalence is 1% (instead of 0%) instead of t2:t1 positive reports ratio of 1:20 (i.e. 5%), you'll actually get (in the 1% prevalence case) just a 1:5 (t2:t1) ratio (more precisely 18.85%). Also this t2:t1 positive report ratio further drops to just 1:3 (29.18%) on just 2% true prevalence. Here's a table of such calculations below, in which I assume the same test is repeated (as not to complicate this example with two sets of sensitivity/specificity.)

enter image description here

Furthermore, for a highly specific test, i.e. 99% (instead of 95%) specificity, the convergence is even faster, e.g. for just 1% true prevalence, the t2:t1 positive report ratio is now almost 1:2 (43.86%; up from 18.85% when the specificity was just 95%).

enter image description here

Ironically however, if the true prevalence is 0%, a more specific test (99% vs 95% specificity) will "disagree with itself" on a retest even more, i.e. 1:100 vs 1:20 concurrence.

Considering the specificities and sensitivities of the tests known (which is alas a bit iffy in "real world" situations as opposed to lab settings) you can basically find out what the true prevalence is, e.g. assuming 60% sensitivity and 95% specificity for the antigen test and 95% sensitivity and 99% specificity for the (t2) PCR test, from a range of t2:t1 like you reported (2:20 to 1:40), you can get somewhere between 0.1% and 1% true prevalence in the starting population... which is still a fairly large range. (Of some note, this doesn't depend on the initial population size anymore, i.e. having two tests allows us to "factor out" that.)

enter image description here

In reality (even for lab-settings tests) you only have confidence intervals for the sensitivity and specificity as opposed to their true number, which makes the above exercise a bit more involved... and as Bryan's numerical example showed, 0% (true) prevalence could well be in the solution interval. The specificity of the 2nd test (PCR) is actually critical in this regard, if you relax it to 95% (from 99%), then 0% true prevalence becomes a (realistic) solution.

2
  • Thanks again, Fizz. It's helpful and informative. I wish both answers were presented to the mainstream in a way that was easily understood. Yes, a lot of what is going on here is profiteering and incompetence, but that does not excuse the banal, magical thinking forming the majority of public opinion.
    – Frank
    Mar 23, 2021 at 12:11
  • @Frank: well, as I could not find any literature on Covid specifically that deals with these "nested tests", I asked a separate question here. I'm sure there is other (perhaps more general) literature on screening tests that is relevant, just that it doesn't seem to have been invoked in a Covid context yet...
    – Fizz
    Mar 23, 2021 at 12:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.