Why are Novel Coronavirus anitbody tests disparaged?

Question

The Novel Coronavirus antibody test has been around since February 2020 when it was invented at a Chinese University. It can measure both IgM to show infection in progress and IgG to indicate full immunity. It is disparaged because it is only 94.4% accurate.

The Detroit News however reports the "gold standard" PCR test is only 90% effective. People walked around town with false negatives when they were really positive. So why only use PCR tests and never IgM / IgG tests?

I would feel more comfortable with both tests. First the daily 15 minute finger prick blood testto see if IgM develops followed by hospital / lab PCR test. Then after positive PCR self isolate. Then after negative PCR test continue to self isolate until daily 15 minute finger prick blood test shows IgG. I think it's too early in the war to disparage either test but would point out the 15 minute finger prick blood test is cheap and doesn't drain limited healthcare resources.

It would be interesting to run the same blood sample on all the IgM / IgG test devices and compare results. It would also be interesting to send swabs from same suspect to multiple PCR test labs see how they stack up. Ones that don't agree with the majority either have a better test or failed and could be subject to government oversight.

IgM results will show up (I think I read) whilst suspect is asymptomatic. If we want to win this war in three months instead of three years like the Spanish Flu, we need to pull out all the stops. But the question remains, why are IgM / IgG tests (antibody tests) frowned upon?

Answer 1

Summary: they are not yet sufficiently well validated (and many are not working sufficiently well) to be used for mass testing.

For mass tests,

• of the kind you describe, we need high sensitivity very early into the disease
• of any kind, we need false positive rate (1 - specificity) to be well below the prevalence of SARS-CoV-2 positive people, otherwise we'll mostly find false positives the positive predcitive value (i.e. probability that you are really positive given that the test said so) would be too low to be of practical use: we'd want this to eventually conclude who is immune.

See also: Why are people with COVID-19 symptoms being denied tests in the US?

---

It would be interesting to run the same blood sample on all the IgM / IgG test devices and compare results

This is what https://covidtestingproject.org/ does, comparing 10 commercially available rapid antibody tests for SARS-CoV-2 and 2 ELISA tests: an in-house one of the group and a commercial one.

Manuscript (pre peer-review)

I would feel more comfortable with both tests. First the daily 15 minute finger prick blood testto see if IgM develops followed by hospital / lab PCR test. Then after positive PCR self isolate. Then after negative PCR test continue to self isolate until daily 15 minute finger prick blood test shows IgG.

Unfortunately, the covid testing project data shows IGM (like IGG) not being very sensitive at the onset of symptoms: IGM sensitivity reaches 60 - 85 % (point estimates for the various tests) around day 11 - 15 after onset of symptoms for all but one test that never gets above 38 % sensitivity.

So the antibody tests including IGM kick in basically only after PCR testing for current infection is done: PCR for virus RNA and antibody testing are thus for different time windows.

The scenario that you could do a rapid test to check whether you can safely visit the grandparents or other vulnerable persons is thus not going to work with these rapid tests.

A glance at the individual test results suggests:

• that (as one may expect) there is a trade-off between sensitivity and specificity, at least to some extent.
  This may not be a big problem, since we could choose more sensitive or more specific antibody tests according to the purpose of testing. (E.g. for a Corona-Antibody Passport, a highly specific test would be appropriate. For epidemiologic studies one may choose a different trade-off, but one would still need a sensible relationship between expected prevalence and specificity.)

• high correlation between the misclassified cases, i.e. the false negative and false positive samples tend to be the same across the tests.
  This has the important consequence that the possibility to improve overall accuracy by combining tests from different manufacturers is limited.

One important concern with the antibody tests is whether they show cross-sensitivity to the corona viruses that circulate since years and cause only normal harmless colds (I did not find studies on antibody prevalence for those). Unfortunately, the data so far includes only 4 such samples (of which some show low non-zero scores for some of the tests). Even for a test that correctly recognizes all four of them as negative for SARS-CoV-2 antibodies, the 95 % credible interval for specificity in this subgroup is only "better than 55 %". In other words, we cannot really rely [yet] on the tests in this particular respect.
Also, since local prevalence of such corona virus strains may vary, it is not clear whether a test specificity can be directly used for populations on other continents.

Answer 2

The antibody tests, at least in the USA, have only just received fast track approval so at least in the USA were not available in an FDA approved form until recently. The Cellex test has a sensitity of 93.8% and specificity of 95.6% when tested in China but may be better in the USA. So, the antibody tests are to demonstrate exposure, either in the past or current. The test itself only takes some minutes.

A rt-PCR test checks for the presence of virus from a swab, preferentially in the nasopharynx but also anterior nasal, and oropharynx where the rates of detection are likely to be lower. This demonstrates current infection. A positive antibody test may be followed by a rt-PCR test to see if the positive antibody test actually indicates current infection.

However, there has been a world wide shortage of testing equipment so inadequate testing has been performed. The Abbott M2000 system is supposed to be able to perform millions of tests but the test kits that go with them are in short supply

Abbott’s machines, a model called the m2000, should have been capable of running about a million Covid-19 tests in the last three weeks, Deborah Birx, a member of the White House’s coronavirus task force, said at a briefing in Washington Wednesday night. But they have run less than 10% of that amount.

“It’s a very high throughput machine, we have a million tests out there,” Birx said. “They’re not running. We’ve only run 88,000 tests in three weeks off of those machines, with a million test kits.”

U.S. attempts to contain the coronavirus have been stymied by repeated problems and failures with diagnostic tests, which are necessary to identify patients and isolate them, or to help protect health workers treating them. Heath workers and officials around the U.S. have cited long delays in getting results and limited availability of tests from multiple different sources.

In an ideal world the tests would be run as per their clinical indication. The reality is different due to supply constraints.

https://www.bloomberg.com/news/articles/2020-04-09/abbott-tests-to-detect-covid-19-are-falling-short-birx

Answer 3

The source you linked to says

In a clinical evaluation, Zhejiang Orient Gene Biotech reported the tests to be 61.8% to 94.4% sensitive, depending on the type of antibody that shows up in the test results. [...] For IgM antibodies, Zhejiang Orient Gene Biotech reported an overall sensitivity of 61.8%, meaning that it may only accurately report a positive prior case of COVID-19 61.8% of the time. For IgG antibodies, Zhejiang Orient Gene Biotech reported an overall sensitivity of 94.4%, meaning that these tests should accurately report a positive case of COVID-19 most of the time.

If it were 94.4% (all the time) as you claimed in your post, it would be a different matter.

Furthermore, per recent news, the antibodies tests are "disparaged" in the US because the FDA let anyone make them and didn't request data from the manufacturers as a prerequisite for approving them:

On CNN Tuesday, senior medical correspondent Elizabeth Cohen warned that many of the antibody tests on the market for COVID-19 — the tests that show whether you are immune to the virus — are barely tested under the Food and Drug Administration’s new standards, and possibly ineffective. [...]

“The problem is, the FDA lowered the standards last month, and basically pretty much anyone can sell an antibody test,” continued Cohen. “They don’t even have to show that it works. All they have to do is say, hey, FDA, I want to sell this test and I validated it here in my city where I am. That’s all they have to do. They don’t have to show their data.”

And after the backlash, there's some sort of review announced:

[Scott Becker, CEO of the Association of Public Health Laboratories] said FDA Commissioner Stephen Hahn told him and members of his association that the National Cancer Institute (NCI) would start reviewing tests that are on the market to see if they obtain correct results. [...]

The FDA did not respond to questions for this story, referring CNN to the NCI.

A lot of the media has incorrectly headlined these news as the FDA tightening the (approval) standards, but that doesn't seem to be actually happening yet.

The WHO has now gone to some length to explain their skepticism of the (rushed) antibodies tests.

“These antibody tests will be able to measure that level of serology presence, that level of antibodies, but that does not mean that somebody with antibodies” is immune, said Dr. Maria Van Kerkhove, head of WHO’s emerging diseases and zoonosis unit. [...]

Dr. Mike Ryan, executive director of WHO’s emergencies program, said scientists are also still determining the length of protection antibodies might give a person who has been infected with the coronavirus.

“Plus some of the tests have issues with sensitivity,” he added. “They may give a false negative result.”

Earlier this week, WHO officials said not all people who recover from the coronavirus have the antibodies to fight a second infection, raising concern that patients may not develop immunity after surviving Covid-19.

And from that earlier coverage:

A preliminary study of patients in Shanghai found that some patients had “no detectable antibody response” while others had a very high response, said Dr. Maria Van Kerkhove, WHO’s lead scientist on Covid-19. Whether the patients who had a strong antibody response were immune to a second infection is “a separate question,” she added. [...]

Ryan said there are questions about whether the virus can reactivate after a patient recovers and tests negative for Covid-19.

So it's not the case that all such tests are flawed but that governments seem to pin a lot of hopes on them in terms of reopening their countries etc., which may or may not turn out justified given the present level of knowledge.

Answer 4

There is a group of research organisations that call themselves the Covid-19 testing project, that has (and are) producing comparisons of various antibody tests.

It sounds like because the FDA relaxed their rules on marketing Covid-19 tests without the normal rigour that would insist in mid April 2020, but have since back tracked on this slightly in early May, allowing 10 days to produce the evidence of clinical correctness, before the FDA pulling the test off the market.

Looking at data in the covid testing project (link above) it shows the date of onset of symptoms will (obviously) give different levels of antibody detection. It is really only until >20 days that this seems to settle down. But even then with the 12 tests that were compared, there were a surprising variation in the false negatives between tests. False positives were not looked at in this analysis.

So to answer the question more directly, it seems that because may tests have come out that have had poor clinical testing, this has caused a general negative impression of antibody tests for Covid-19.