BIDMC researchers develop model to estimate false-negative rate for COVID-19 tests

Boston, Mass. – Even with more than 1.5 million Americans receiving a COVID vaccine each day, officials estimate it will take many more months before enough people are protected from the deadly virus. Until then, and potentially beyond, experts agree that opening up schools, restaurants and other public places as safely as possible will rely on widespread testing for SARS-CoV-2, the virus that causes COVID-19.

As of June 2020, the U.S. Food and Drug Administration (FDA) had granted emergency use authorization for more than 85 different viral DNA test kits — or assays — each with widely varying degrees of sensitivity and unknown rates of accuracy. However, with no existing gold standard test for the novel coronavirus, there’s little data on which to judge these various tests’ usefulness to municipalities’ efforts to safely re-open for business.

A team of researchers at Beth Israel Deaconess Medical Center (BIDMC) has developed a mathematical means of assessing tests’ false-negative rate. The team’s methodology, which allows an apples-to-apples comparison of the various assays’ clinical sensitivity, is published in the journal Clinical Infectious Diseases.

“For getting back to business as usual, we all agree we’ve got to massively ramp up testing to figure out who’s negative and who’s infectious — but that’s only going to work optimally if you can catch all the positive cases,” said co-corresponding author James E. Kirby, MD, Director of the Clinical Microbiology Laboratories at BIDMC. “We found that clinical sensitivities vary widely, which has clear implications for patient care, epidemiology and the social and economic management of the ongoing pandemic.”

“These results are especially important as we transition from testing mostly symptomatic individuals to more regular screening across the community,” said co-corresponding author Ramy Arnaout, MD, DPhil, Associate Director of the Clinical Microbiology Laboratories at BIDMC. “How many people will be missed—the false negative rate—depends on which test is used. With our model, we are better informed to ask how likely these people are to be infectious.”

COVID test results are usually reported as simply positive or negative. However, positive individuals can harbor radically different amounts of virus, or viral load, depending on how long they’ve been infected or how severe their symptoms are. In fact, viral load can vary as much as a hundred million-fold among individuals, said Kirby.

Using data from more than 27,000 tests for COVID-19 performed at Beth Israel Lahey Health hospital sites from March 26 to May 2, 2020, Kirby, Arnaout, and colleagues first demonstrated that viral loads can be dependably reported. “This helps distinguish potential superspreaders, at one extreme, from convalescent people, with almost no virus, and therefore low likelihood of spreading the infection,” Arnaout said.

Next, the researchers estimated the clinical sensitivity and the false-negative rate first for the in-house test — which was among the first to be implemented nationwide and considered among the best in class. Analyzing repeat test results for the nearly 5,000 patients who tested positive allowed the researchers to determine that the in-house test provided a false negative in about 10 percent of cases, giving the assay a clinical sensitivity of about 90 percent.

To estimate the accuracy of other assays, the team based their calculations on each tests’ limit of detection, or LoD, defined as the smallest amount of viral DNA detectable that a test will catch 95 percent or more of the time.

Arnaout, Kirby, and colleagues demonstrated that the limit of detection can be used as a proxy to estimate a given assay’s clinical sensitivity. By the team’s calculations, an assay with a limit of detection of 1,000 copies viral DNA per mL is expected to detect just 75 percent of patients with COVID-19, providing one out of every four people with a false-negative. The team also showed that one test available today misses as many as one in three infected individuals, while another may miss up to 60 percent of positive cases.

While not every COVID positive patient missed by sensitive PCR and antigen detection tests will be infectious to others, some will, the researchers note.

“These misses will undermine public health efforts and put patients and their contacts at risk,” said Arnaout. “This must give us pause, and we really need to benchmark each new test even in our rush to increase testing capacity to understand how well they support our testing goals.”

Co-authors included Rose A. Lee, Christina F. Yen, Kenneth P. Smith, Cody Callahan, Ghee Rye Lee, and Rohit Arora, all of BIDMC.

This work was supported in part by a grant from the National Institute of Allergy and Infectious Diseases, NIH (F32 AI124590). Dr. Arnaout is principal investigator on clinical study sponsored by Abbott molecular, MitoLab, and E25Bio, unrelated to the current work. All other authors report no potential conflicts of interest.

About Beth Israel Deaconess Medical Center Beth Israel Deaconess Medical Center is a patient care, teaching and research affiliate of Harvard Medical School and consistently ranks as a national leader among independent hospitals in National Institutes of Health funding.

BIDMC is the official hospital of the Boston Red Sox. For more information, visit www.bidmc.org.

Beth Israel Deaconess Medical Center is a part of Beth Israel Lahey Health, a health care system that brings together academic medical centers and teaching hospitals, community and specialty hospitals, more than 4,000 physicians and 35,000 employees in a shared mission to expand access to great care and advance the science and practice of medicine through groundbreaking research and education.

###