As of March 13, 45/301 people have tested positive/negative for COVID-19 in Florida. What is the expected false positives/negatives here?
UPDATE (March 14): 71/478 from what I consider the best source.
-
I didn't downvote but I would expect the reason someone did is the lack of prior research. How would anyone know the expected false positives/negatives so early with a new disease?– Carey Gregory ♦Commented Mar 14, 2020 at 0:48
-
1Honestly, I think questions which are difficult to answer deserve more upvotes (assuming the answer is useful), but ok. I could have researched this more, maybe for past flus, but am open to other theoretical approaches. I'd first like to know if those numbers could be off by even 50%.– bobuhitoCommented Mar 14, 2020 at 1:34
-
1Hard to answer and impossible to answer are different things, and I think your question falls into the latter category. Ask it a year from now and I think it will be more answerable.– Carey Gregory ♦Commented Mar 14, 2020 at 3:52
-
1Related: medicalsciences.stackexchange.com/q/21337/7951– Chris RogersCommented Mar 14, 2020 at 11:39
-
@CareyGregory: from what I've read in newspapers, the tests underwent fast-track regulatory approval - so I imagine someone does have an idea at least of their sensitivity and specificity (from which we could at least calculate some guesstimates with some information about testing regimes). I can very well imagine, of course, that those people right now don't have any time to hang out on SX...– cbeleitesCommented Mar 14, 2020 at 19:16
|
Show 1 more comment
2 Answers
See long answer for How accurate are coronavirus tests?
With the "worst-case numbers" from there which I take from the minimum performance requirements the FDA currently uses with an emergency validation to allow labs to quickly implement Covid-19 tests without undergoing the full validation procedure they normally take, we have LR+ ≈ 11 and LR- ≈ 1/20.
The tests may be (and probably are) actually much better.
If we take 71 positive : 478 negative tests as a surrogate for the prevalence of Covid-19 infected among the tested population (14.5 %), the post-test probabilities of having Covid-19 are
71 : 478 * 11 = 781 : 478 ≈ 5 : 3 for those who tested positive, i.e. ≈ 62 % PPV or post-test probability of having Covid-19.
Thus, as many as 38 % of the 71 or 27 could be false positives.71 : 478 * 1/20 = 71 : 9560 ≈ 1 : 135 or 0.7 % post-test probability of nevertheless really having Covid-19.
I.e. up to maybe 1 false negative case.
Update: I've updated the linked answer since I've meanwhile found more detailed data on the actual validation performed for several tests. Most of them used more than the minimum required sample size, but it's not that 1000s of validation samples were run. (The infamous CDC test got emergency approval after only 13 positive validation cases, though, so even less. But that was beginning of Feb, and they may not have had more test samples available at that time)
If we want to calculate with expected instead of worst-possible performance for e.g. the Thermo Fisher test, LR+ and LR- would be 61 and 1/61, respectively.
PPV would then have been 90 % (7 false positives) and NPV 0.25 % (0 false negatives).
-
Thanks, but I'd rather not have "worst-case numbers". Your other answer says all lab tests have been 100% correct, so one should bet Florida results are more accurate than 62%, right?– bobuhitoCommented Mar 19, 2020 at 21:34
-
@bobuhito: noone likes having worst-case numbers :-). Depending on what practical questions you ask, worst-case or e.g. expected would be more appropriate. The thing is, even though 30 correct out of 30 are 100 % observed hit rate just like 1000 out of 1000 are, there is a difference between the two situations. And right now, we do not have sufficient validation results to be sure that the tests work much better than I did outline. Since I wrote the answer, the FDA has emergency-approved some more tests, for which the validation used more cases (50 - 100). I've updated the linked question Commented Mar 19, 2020 at 21:41
-
with that information, and also updated with the expected performance. Your numbers are as far as I can tell from before those tests became available, though. The other thing is: in practical terms, nothing changes. Whether someone has 62 % PPV or 90 % PPV, they go to quarantine - and in both cases we don't expect false negatives from the test performance (my guesstimate is that a false negative due to sampling error could be more probable). The other thing is that additional "tests" can further increase PPV or decrease NPV. E.g. if you are positive and tested because you are coughing and have Commented Mar 19, 2020 at 21:51
-
fever, you can be sufficiently sure for most practical purposes that you got COVID-19. If you are tested because you are a contact person but do not have symptoms and get a negative result, the remaining uncertainty would rather come from the possibility that you could meanwhile have contracted the virus rather than from the uncertainty of the test. Commented Mar 19, 2020 at 21:53
The published numbers probably based on the PCR test. About the PCR I has found some information few time ago.
At first it is to consider that there is only a certain period in which a sample from a certain area shows results in the PCR. For throat swab samples was found that it can take 4 to 8 days for an existing infection to be displayed with PCR. The time period in which the test remains positive is limited to 4 to 15 days, although the disease continued to develop. Now nasal samples are recommended before taking a throat swab, but there will also be a suitable period for sampling.
In expansion, how accurate are the PCR tests in the period in which the tests should reliable be positive. There is a study in which PCR tests were carried out daily. There is shown that inside of period with positive results, some tests produced a negative result. The days only from first day with a positive result to the last day with a positive result gives a total period of 84 days over all patients. During this time, also 11 tests with negative results were produced. This gives a value of 13.7% negative test results, even though the patient was infected with COVID-19.
It must be noted that PCR testing is not a simple yes/no test. The result first depends whether a sufficient number of viruses is obtained.
- As written above, it is important to choose the correct region of body.
- It must be the relevant time at which the viruses are in the selected region.
- The sample must be obtained in the correct manner.
- The viral load in the patient himself has to be suffice (medrxiv.org)
Here is described that for a number of 484 copies of SARS-CoV-2 RNA per milliliter some PCR test products detect SARS-CoV-2 with 100% but one other of the approved products detects 0% of virus.
So there is not the one accuracy. It is to be asked which type of test kit was used by which manufacturer, who took how the sample, etc. Every single item can significantly impact the accuracy of a test.
