Since January 2020 Elsevier has created a COVID-19 resource centre with

free information in English and Mandarin on the novel coronavirus COVID-

19. The COVID-19 resource centre is hosted on Elsevier Connect, the
company's public news and information website.

Elsevier hereby grants permission to make all its COVID-19-related

research that is available on the COVID-19 resource centre - including this
research content - immediately available in PubMed Central and other
publicly funded repositories, such as the WHO COVID database with rights
for unrestricted research re-use and analyses in any form or by any means
with acknowledgement of the original source. These permissions are
granted for free by Elsevier for as long as the COVID-19 resource centre
remains active.
 braz j infect dis 2 0 2 0;2 4(2):180–187

The Brazilian Journal of

INFECTIOUS DISEASES
www.elsevier.com/locate/bjid

Brief communication

COVID-19: a meta-analysis of diagnostic test

accuracy of commercial assays registered in Brazil

Rodolfo Castro a,b , Paula M. Luz c , Mayumi D. Wakimoto c , Valdilea G. Veloso c ,

Beatriz Grinsztejn c , Hugo Perazzo c,∗
a Fundação Oswaldo Cruz (FIOCRUZ), Escola Nacional de Saúde Pública Sergio Arouca (ENSP), Rio de Janeiro, RJ, Brazil
b Universidade Federal do Estado do Rio de Janeiro (UNIRIO), Instituto de Saúde Coletiva (ISC), Rio de Janeiro, RJ, Brazil
c Fundação Oswaldo Cruz (FIOCRUZ), Instituto Nacional de Infectologia Evandro Chagas (INI), Rio de Janeiro, RJ, Brazil

a r t i c l e i n f o a b s t r a c t

Article history: The accuracy of commercially available tests for COVID-19 in Brazil remains unclear. We
Received 3 April 2020 aimed to perform a meta-analysis to describe the accuracy of available tests to detect COVID-
Accepted 5 April 2020 19 in Brazil. We searched at the Brazilian Health Regulatory Agency (ANVISA) online platform
Available online 18 April 2020 to describe the pooled sensitivity (Se), specificity (Sp), diagnostic odds ratio (DOR) and sum-
mary receiver operating characteristic curves (SROC) for detection of IgM/IgG antibodies
Keywords: and for tests using naso/oropharyngeal swabs in the random-effects models. We identified
SARS-CoV-2 16 tests registered, mostly rapid-tests. Pooled diagnostic accuracy measures [95%CI] were:
Coronavirus (i) for IgM antibodies Se = 82% [76–87]; Sp = 97% [96–98]; DOR = 168 [92–305] and SROC = 0.98
Diagnostic accuracy [0.96–0.99]; (ii) for IgG antibodies Se = 97% [90–99]; Sp = 98% [97–99]; DOR = 1994 [385–10334]
and SROC = 0.99 [0.98–1.00]; and (iii) for detection of SARS-CoV-2 by antigen or molec-
ular assays in naso/oropharyngeal swabs Se = 97% [85–99]; Sp = 99% [77–100]; DOR = 2649
[30–233056] and SROC = 0.99 [0.98–1.00]. These tests can be helpful for emergency testing
during the COVID-19 pandemic in Brazil. However, it is important to highlight the high rate
of false negative results from tests which detect SARS-CoV-2 IgM antibodies in the initial
course of the disease and the scarce evidence-based validation results published in Brazil.
Future studies addressing the diagnostic performance of tests for COVID-19 in the Brazilian
population are urgently needed.
© 2020 Sociedade Brasileira de Infectologia. Published by Elsevier España, S.L.U. This is
an open access article under the CC BY-NC-ND license (http://creativecommons.org/
licenses/by-nc-nd/4.0/).

The World Health Organization (WHO) declared Coronavirus with suppression strategies can drastically reduce the num-
disease 2019 (COVID-19) as pandemic on March 11, 2020. Due ber of hospitalizations and deaths.2 Large-scale testing, rapid
to the rapid spread of severe acute respiratory syndrome diagnosis and immediate isolation of cases coupled with rig-
coronavirus 2 (SARS-CoV-2) viruses, we are currently facing orous tracking and preventive self-isolation of close contacts
a scenario of sustained community transmission of COVID- are essential measures to reduce the burden of the COVID-19
19 worldwide.1 Early implementation of mitigation associated pandemic.3 Therefore, tests for COVID-19 should be point-of

∗
Corresponding author.
E-mail addresses: hugo.perazzo@ini.fiocruz.br, [email protected] (H. Perazzo).
https://doi.org/10.1016/j.bjid.2020.04.003
1413-8670/© 2020 Sociedade Brasileira de Infectologia. Published by Elsevier España, S.L.U. This is an open access article under the CC
BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).
 b r a z j i n f e c t d i s . 2 0 2 0;2 4(2):180–187 181

-care, widely available, and implemented outside of hospital gold standard. In addition, 11 tests are considered as point-of-
settings to prevent overwhelming the health care system as care (POC) tests: nine detecting IgM/IgG antibodies in finger
well as the risk of nosocomial transmission to other patients prick sample and two detecting SARS-CoV-2 Ag in nasopha-
and healthcare workers. Results from the mathematical mod- ryngeal and/or oropharyngeal swabs. A total of seven tests are
eling study performed by the Imperial College of London for imported from the following countries: China (n = 4), United
Brazil, the implementation of suppression strategies could States of America (n = 2), and Spain (n = 1) tests (Supplemen-
save up to a million lives and prevent the collapse of the tary Table 1). Table 1 summarizes the diagnostic performance
Brazilian Unified Health System (Sistema Único de Saúde, SUS) of each test as reported by the manufacturer. Data of posi-
compared to an unmitigated strategy.4 Though massive test- tive/negative samples were not divided by IgM/IgG antibodies
ing is a cornerstone to reduce the burden of COVID-19, the analysis for two tests, and the manufacturer did not report the
accuracy of commercially available tests for COVID-19 in Brazil number of TP, FP, TN and FN for a swab test [these tests were
remains unclear. The aim of this study was to perform a meta- excluded from the pooled diagnostic analysis].
analysis to describe the accuracy of available tests to detect For detection of IgM antibodies (eight tests; 951 samples),
COVID-19 in Brazil. pooled diagnostic accuracy measures [95%CI] were: Se = 82%
To identify the registered tests for COVID-19 diagnosis in [76–87]; Sp = 97% [96–98]; DOR = 168 [92–305]; LR+ = 31.3
Brazil, we performed searches on March 30, 2020 using the [19.7–49.7]; LR− = 0.19 [0.14–0.25] and SROC = 0.98 [0.96–0.99]
following terms “COVID-19” OR “SARS-CoV-2” OR “2019-nCoV” (Fig. 1A and B). For detection of IgG antibodies (eight tests; 1503
OR “coronavirus” at the Brazilian Health Regulatory Agency samples), pooled diagnostic accuracy measures [95%CI] were:
(Agência Nacional de Vigilância Sanitária, ANVISA, website: Se = 97% [90–99]; Sp = 98% [97–99]; DOR = 1994 [385–10334];
https://consultas.anvisa.gov.br/#/saude/). Data were extracted LR+ = 56.6 [30.6–104.7]; LR− = 0.03 [0.01–0.11]; and SROC = 0.99
by two independent researchers to an electronic database. [0.98–1.00] (Fig. 2A and B). Finally, for detection of SARS-
The identified tests were stratified according to those which CoV-2 by antigen or molecular assays in nasopharyngeal
detect SARS-CoV-2 immunoglobulin antibodies (IgM and/or and/or oropharyngeal swabs (four tests; 464 samples),
IgG) and nucleic acid (RNA) or antigen (Ag) from SARS-CoV-2. pooled diagnostic accuracy measures [95%CI] were: Se = 97%
The following data were extracted from documents uploaded [85–99]; Sp = 99% [77–100]; DOR = 2649 [30–233056]; LR+ = 89.5
by manufacturers for each registered test: test name, ANVISA [3.3–2400.8]; LR− = 0.03 [0.01–0.17]; and SROC = 0.99 [0.98–1.00]
registry number, type of sample, type of analysis, type of assay (Fig. 3A and B).
and data of the diagnostic value of each test as reported by the This meta-analysis highlighted the accuracy of tests for
manufacturer [number of true positive (TP), false positive (FP), COVID-19 diagnosis registered by the Brazilian health author-
true negative (TN) and false negative (FN)]. ities (ANVISA). To the best of our knowledge, this is the first
Diagnostic performance for the detection of IgM and IgG study to provide pooled diagnostic accuracy of tests for COVID-
antibodies was analyzed separately for each test. Detection 19 available for clinical use in Brazil. Large scale testing is
of IgM antibodies is often interpreted as an indicator of essential to tackle COVID-19 because accurate knowledge of
acute infection while the detection of IgG antibodies repre- the number of confirmed cases provides information on the
sents previous infection/immunity. Sensitivity (Se), specificity spread of the virus within a population and thus on the evo-
(Sp), positive predictive value (PPV), and negative predictive lution of the pandemic. As of April 3rd 2020, the Brazilian
value (NPV) were described. Data synthesis was performed Ministry of Health had confirmed around 9000 COVID-19 con-
using univariate mixed-effects logistic regression models firmed cases.6 However, the actual number of cases is likely
with maximum likelihood estimation based on adaptive much higher because of limited testing. Furthermore, while
Gaussian quadrature using xtmelogit (midas package) from we wait for effective vaccines, the knowledge of the extent
STATA for Windows (2017; StataCorp LP, College Station, TX, of immunity in the population after the pandemic ceases
USA).5 Pooled Se, Sp, positive likelihood ratio (LR+), nega- depends on accurate diagnostics which will later guide the
tive likelihood ratio (LR−) and diagnostic odds ratio (DOR) use of appropriate strategies when facing a potential second
are described for IgM antibodies, IgG antibodies and for tests wave of COVID-19.
using nasopharyngeal and/or oropharyngeal swabs for SARS- Since the beginning of the pandemic, medical companies
CoV-2 detection in the random-effects models. Forest plots and research institutes have been looking for developing and
with test-specific and overall point estimates and 95% con- approving tests to detect current viral infection and immu-
fidence intervals (CI) were provided with Cochran’s Q and I2 nity to SARS-CoV-2.7 The diagnosis of SARS-CoV-2 infection
heterogeneity statistics. Summary receiver operating charac- involves collecting the correct specimen from the patient at
teristic curves (SROC) were plotted with the presentation of the right time. SARS-CoV-2 detection using real-time poly-
a summary operating point, 95% confidence and prediction merase chain reaction (PCR) test kits can be considered as the
contours. gold-standard for the diagnosis of COVID-19. However, this
A total of 16 tests for detection of COVID-19 registered in technique requires certified laboratories, expensive equip-
the ANVISA’s online platform were identified. Out of these, ment and trained technicians. Rapid tests for detection of
11 tests detect SARS-CoV-2 N-Protein IgM and/or IgG antibod- specific antibodies of SARS-CoV-2 in blood samples remain
ies [nine for IgM/IgG; one for IgM and one for IgG detection] a good choice for diagnosing COVID-19. It is estimated that
in human serum, plasma, whole blood (or finger prick sam- SARS-CoV-2 IgM antibodies can be detected in a blood sample
ples); three detect the nucleic acid (RNA) and two detect the after 3–6 days and IgG antibodies after eight days of symp-
antigen (Ag) of SARS-CoV-2 in nasopharyngeal and/or oropha- toms onset.8 Serological assays, detecting IgM/IgG antibodies,
ryngeal swabs. All tests considered molecular assays as the are important tests for diagnosing SARS-CoV-2 and can help
182 b r a z j i n f e c t d i s . 2 0 2 0;2 4(2):180–187

Table 1 – Characteristics of tests for COVID-19 registered at the Brazilian Health Regulatory Agency (ANVISA) up to March
30, 2020.
COVID-19 test characteristics Type POC-test Result (min) n Se (%) Sp (%) TP (n) FP (n) TN (n) FN (n) PPV (%) NPV (%)

CORONAVIRUS IgG/IgM IgG Yes 20 70 100 98 20 1 49 0 95 100

(COVID-19)
ANVISA registry number:
10159820239
Samples: fingertip blood, serum
or plasma
Type: immunochromatographic
assay
IgM Yes 20 70 85 96 17 2 48 3 89 94
ECO F COVID-19 Ag Ag Yes 30 100 86 95 30 5 62 3 86 95
ANVISA registry number:
80954880131
Samples: nasopharyngeal and
oropharyngeal swabs
Type: immunofluorescence assay
COVID-19 IgG/IgM ECO Test IgG Yes 10 70 95 99 20 1 49 0 95 100
ANVISA registry number:
80954880132
Samples: fingertip blood, serum
or plasma
Type: immunochromatographic
assay
IgM Yes 10 70 90 94 17 2 48 3 89 94
COVID-19 Ag ECO Test Ag Yes 15 80 70 97 17 8 53 2 68 96
ANVISA registry number:
80954880133
Samples: nasopharyngeal and
oropharyngeal swabs
Type: immunochromatographic
assay
Anti-COVID-19 IgG/IgM Rapid Test IgG Yes 10 70 100 98 20 1 49 0 95 100
ANVISA registry number:
10009010356
Samples: fingertip blood, serum
or plasma
Type: immunochromatographic
assay
IgM Yes 10 70 85 96 17 3 48 2 85 96
LUMIRATEK COVID-19 (IgG/IgM) IgG Yes 10 181 97.4 98.9 37 1 142 1 97 99
ANVISA registry number:
81327670112
Samples: fingertip blood, serum
or plasma
Type: immunochromatographic
assay
IgM Yes 10 181 86.8 98.6 33 2 141 5 94 97
MedTeste Coronavirus (COVID-19) IgG Yes 10 181 97.4 99.3 37 1 142 1 97 99
IgG/IgM
ANVISA registry number:
80560310056
Samples: fingertip blood, serum
or plasma
Type: immunochromatographic
assay
IgM Yes 10 181 86.8 98.6 33 2 141 5 94 97
FAMÍLIA KIT XGEN MASTER RT-PCR No NA NR 95 100 NR NR NR NR NR NR
COVID-19 – SARS-CoV-2
ANVISA registry number:
80502070088
Samples: nasopharyngeal and
oropharyngeal swabs
Type: RT-PCR
 b r a z j i n f e c t d i s . 2 0 2 0;2 4(2):180–187 183

– Table 1 (Continued)
COVID-19 test characteristics Type POC-test Result (min) n Se (%) Sp (%) TP (n) FP (n) TN (n) FN (n) PPV (%) NPV (%)

DPP® COVID-19 IgM/IgG System IgG Yes 5 20 77.8 100 14 0 2 4 100 33

ANVISA registry number:
80535240052
Samples: fingertip blood, serum
or plasma
Type: immunochromatographic
assay
IgM Yes 5 20 55.6 100 10 0 2 8 100 20
Smart Test Covid-19 Vyttra IgM/IgG Yes 10 10 100 99.5 4 0 6 0 100 100
ANVISA registry number:
81692610175
Samples: fingertip blood, serum
or plasma
Type: immunochromatographic
assay
VIASURE SARS-CoV-2 RT-PCR No NA 134 100 100 35 0 98 1 100 99
ANVISA registry number:
10355870373
Samples: nasopharyngeal and
oropharyngeal swabs
Type: RT-PCR
Cobas SARS-CoV-2 RT-PCR No NA 150 100 100 50 0 100 0 100 100
ANVISA registry number:
10287411491
Samples: nasopharyngeal and
oropharyngeal swabs
Type: RT-PCR
Teste Rápido em Cassete IgG Yes 10 70 100 98 20 1 49 0 95 100
2019-nCoV IgG/IgM
ANVISA registry number:
81325990117
Samples: fingertip, serum or
plasma
Type: immunochromatographic
assay
IgM Yes 10 70 85 96 17 2 48 3 89 94
MAGLUMI IgG de 2019-nCoV IgG No NA 841 91.2 97.3 83 20 730 8 81 99
(CLIA)
ANVISA registry number:
80102512430
Samples: serum or plasma
Type: chemiluminescence assay
MAGLUMI IgM de 2019-nCoV IgM No NA 289 78.6 97.5 70 5 195 19 93 91
(CLIA)
ANVISA registry number:
80102512431
Samples: serum or plasma
Type: chemiluminescence assay
One Step COVID-2019 Test; IgM/IgG Yes 15 596 86.4 99.6 312 1 234 49 100 83
ANVISA registry number:
80537410048
Samples: whole blood (fingertip
not reported), serum or plasma
Type: immunochromatographic
assay

Ag, antigen; NA, not applicable; FN, false negative; FP, false positive; NPV, negative predictive value; NR, not reported; PPV, positive predictive
value; POC, point-of-care; RT-PCR, real-time polymerase chain reaction; Se, sensitivity; Sp, specificity.

understand the burden and role of asymptomatic infections. same time being limited due to higher rates of false-negative
The present study showed that 10 serological tests for IgM/IgG results when collected in the early-phase of symptoms onset.
antibodies, including nine point-of-care tests, are currently Our study reports that antigen testing and/or molecular assays
available in Brazil. These tests are simple and can provide using nasopharyngeal and oropharyngeal specimens had high
rapid confirmation of COVID-19 confirmation while at the accuracy for SARS-CoV-2 detection. In China, the rate of
184 b r a z j i n f e c t d i s . 2 0 2 0;2 4(2):180–187

Fig. 1 – Pooled diagnostic accuracy analysis (A) and summary receiver operating characteristic curve (B) of tests (n = 8) for
detection of IgM antibodies tests against SARS-CoV-2.

SARS-CoV-2 detection was higher in oropharyngeal compared the SARS-CoV-2 being present in the nasopharynx increases
to nasopharyngeal swabs during the COVID-19 outbreak.9 with time.8 In the present study, the pooled sensitivity of
Naso/oropharyngeal tests might miss early infection lead- tests using the detection of COVID-19 IgM antibodies in blood
ing to a strategy of repeated testing since the likelihood of was lower compared to antigen/molecular assay detection in
 b r a z j i n f e c t d i s . 2 0 2 0;2 4(2):180–187 185

Fig. 2 – Pooled diagnostic accuracy analysis (A) and summary receiver operating characteristic curve (B) of tests (n = 8) for
detection of IgG antibodies tests against SARS-CoV-2.

nasopharyngeal/oropharyngeal swabs (82% [95%CI 76–87] vs Moreover, data of clinical significance with regards to the diag-
97% [95%CI 85–99]). nostic validity of each test, such as patients’ characteristics
It is worth noting that for this analysis we relied on the or time of sample collection after the onset of symptoms,
accuracy of available tests for COVID-19 as per information were not provided. Literature searches that included test
provided by the manufacturers during the test registration names, dealers, or manufacturers, yielded a single paper that
process at ANVISA. There were no peer-reviewed publications. described the analytical performance of a molecular assay
186 b r a z j i n f e c t d i s . 2 0 2 0;2 4(2):180–187

Fig. 3 – Pooled diagnostic accuracy analysis (A) and summary receiver operating characteristic curve (B) of tests (n = 4) using
nasopharyngeal and/or oropharyngeal swabs for detection of antigen or nuclei acid of SARS-CoV-2.

to diagnose COVID-19 in nasopharyngeal and oropharyngeal the One Step COVID-2019 Test is said to have been tested with
specimens.10 Moreover, few tests have been validated with a one of the highest study sample, N = 596. However, the man-
limited number of samples (≤20), and only half of the tests ufacturers did not describe accuracy for IgM and IgG assays
included more than 150 samples in validation studies. In addi- separately in the ANVISA document leading to the exclusion
tion, these tests might present cross-reactivity with other of the test from the forest plot. Finally, there were few tests that
coronavirus that cause respiratory diseases. Among all tests, clearly used similar samples for validation of different tests.
 b r a z j i n f e c t d i s . 2 0 2 0;2 4(2):180–187 187

Recently, the U.S. Food and Drug Administration has granted

Cellex an emergency use authorization to market a rapid anti-
Appendix A. Supplementary data
body test for COVID-19 (Cellex qSARS-CoV-2 IgG/IgM Rapid
Test), the first antibody test released during the pandemic Supplementary data associated with this article can be
(https://www.fda.gov/media/136622/download). Of 128 sam- found, in the online version, at https://doi.org/10.1016/
ples confirmed positive by reverse transcription PCR in pre- j.bjid.2020.04.003.
market testing, 120 tested positive by IgG, IgM, or both. Of 250
confirmed negative, 239 were negative by the rapid test. More- references
over, the numbers translated to a positive percent agreement
with RT-PCR of 93.8% (95% CI: 88.06–97.26%) and a negative
percent agreement of 96.4% (95% CI: 92.26–97.78%), according 1. European Centre for Disease Prevention and Control. Novel
to labeling (https://www.fda.gov/media/136625/download). coronavirus disease 2019 (COVID-19) pandemic: increased
In conclusion, we have reviewed the details and reported on transmission in the EU/EEA and the UK – sixth update:
European Centre for Disease Prevention and Control; 2020.
the pooled diagnostic accuracy of different types of tests cur-
Available from: https://www.ecdc.europa.eu/en/publications-
rently available to tackle COVID-19 in Brazil. A total of 16 tests
data/rapid-risk-assessment-novel-coronavirus-disease-2019-
currently available and registered at the Brazilian Health Reg- covid-19-pandemic-increased
ulatory Agency (ANVISA) were identified, mostly rapid tests to 2. Walker P, Whittaker C, Watson O, et al. The Global Impact of
detect IgM and/or IgG antibodies. The pooled diagnostic accu- COVID-19 and Strategies for Mitigation and Suppression:
racy of tests available in Brazil was satisfactory, and they can Imperial College
be helpful for emergency testing during the COVID-19 pan- London; 2020, http://dx.doi.org/10.25561/77735. Available from:
https://www.imperial.ac.uk/media/imperial-college/medicine/
demic in Brazil. However, it is important to highlight that the
sph/ide/gida-fellowships/Imperial-College-COVID19-Global-
rate of false negative results from tests which detect SARS- Impact-26-03-2020v2.pdf
CoV-2 IgM antibodies, used for detection of COVID-19 in the 3. Salathe M, Althaus CL, Neher R, et al. COVID-19 epidemic in
acute phase, ranged from 10 to 44%. Furthermore, there is Switzerland: on the importance of testing, contact tracing
scarce evidence-base validation results published in Brazil. and isolation. Swiss Med Wkly. 2020;150:w20225.
Future studies addressing the diagnostic performance of a 4. Imperial College London. The Global Impact of COVID-19 and
Strategies for Mitigation and Suppression; 2020.
panel of tests for COVID-19 in the Brazilian population are
http://www.imperial-college-covid19-global-unmitigated-
urgently needed.
mitigated-suppression-scenarios.xlsx/ [accessed 30.3.20].
5. Dwamena B. MIDAS: Stata module for meta-analytical
integration of diagnostic test accuracy studies. Department of
Authors’ contributions
Economics, Boston College; 2007. Available from:
https://ideas.repec.org/c/boc/bocode/s456880.html
Rodolfo Castro: study concept and design, data collection, 6. COVID-19 in Brazil: update of confirmed cases and deaths;
analysis and interpretation of data and drafting and criti- 2020. https://covid.saude.gov.br/ [accessed 3.4.20].
cal revision of the manuscript; Paula M Luz: data collection, 7. Petherick A. Developing antibody tests for SARS-CoV-2.
interpretation of data and critical revision of the manuscript; Lancet. 2020;395:1101–2.
Mayumi D Wakimoto: interpretation of data and critical revi- 8. Li Z, Yi Y, Luo X, et al. Development and clinical application of
a rapid IgM-IgG combined antibody test for SARS-CoV-2
sion of the manuscript; Valdilea G Veloso: interpretation of
infection diagnosis. J Med Virol. 2020,
data and critical revision of the manuscript; Beatriz Grin- http://dx.doi.org/10.1002/jmv.25727 [Epub ahead of print].
sztejn: study supervision, interpretation of data and critical 9. Wang W, Xu Y, Gao R, et al. Detection of SARS-CoV-2 in
revision of the manuscript, Hugo Perazzo: study concept and different types of clinical specimens. JAMA. 2020,
design, study supervision, data collection, interpretation of http://dx.doi.org/10.1001/jama.2020.3786.
data, statistical analysis, drafting and critical revision of the 10. Pfefferle S, Reucher S, Norz D, Lutgehetmann M. Evaluation of
a quantitative RT-PCR assay for the detection of the emerging
manuscript.
coronavirus SARS-CoV-2 using a high throughput system.
Euro Surveill. 2020;25,
Conflicts of interest doi:10.2807/1560-7917.ES.2020.25.9.2000152.

The authors declare no conflict of interest.