https://orcid.org/0000-0001-6825-3803
Figures
Abstract
COVID-19 is the name of the acute respiratory disease caused by the new coronavirus SARS-CoV-2, a close relative of those that caused the severe outbreaks of SARS and MERS several years ago. Since first appearance on December of 2019, the COVID-19 pandemic has cause extremely high levels of mortality, morbidity, global economic breakdown, and the consequent human suffering. The main diagnostic test for the confirmation of symptomatic individuals is the detection of viral RNA by reverse transcriptase–quantitative real time PCR (RT-PCR). Additionally, serology techniques, such as ELISA are useful to measure the antibodies produced in humans after contact with the virus, as well as the direct presence of viral antigens. In this study we aim to assemble and evaluate four ELISA assays to measure the presence of IgG or IgM specific for the viral Spike protein in COVID-19 patients, using either the full recombinant SARS-CoV-2 Spike protein or the fragment corresponding to the receptor binding domain. As a control, we analyzed a group of pre-pandemic serum samples obtained before 2017. Strong reactivity was observed against both antigens. A few pre-pandemic samples displayed high OD values, suggesting the possibility of some cross reactivity. All four assays show very good repeatability, both intra- and inter-assay. Receiver operating characteristic analysis allowed the definition of cutoffs and evaluation of performance for each ELISA by estimation of the area under the curve. This performance parameter was high for all tests (AUC range: 0.98–0.99). Multiple comparisons between tests revealed no significant difference between each other (P values: 0.24–0.95). Our results show that both antigens are effective to detect both specific IgG and IgM antibodies, with high sensitivity (range 0.92–0.99), specificity (range 0.93–0.97) and congruence with the RT-PCR test (Cohen´s Kappa range 0.87–0.93). These assays will allow health authorities to have a new tool to estimate seroprevalence, in order to manage and improve the severe sanitary situation caused by this virus.
Citation: de la Guardia C, Rangel G, Villarreal A, Goodridge A, Fernández PL, Lleonart R (2021) Development of in-house, indirect ELISAs for the detection of SARS-CoV-2 spike protein-associated serology in COVID-19 patients in Panama. PLoS ONE 16(9): e0257351. https://doi.org/10.1371/journal.pone.0257351
Editor: Etsuro Ito, Waseda University: Waseda Daigaku, JAPAN
Received: May 20, 2021; Accepted: August 27, 2021; Published: September 14, 2021
Copyright: © 2021 de la Guardia et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: RL, AV, AG, GS and PLF were funded by grants COVID19-233, IDR10-067, ITE10-020, from National Secretary of Science, Technology and Innovation (SENACYT, Panama). RL, PLF, AV, AG were funded by grants No. 206-2017, 167-2017, 09-2020, 142-2018, and 22-2020 from Sistema Nacional de Investigación (SENACYT, Panama; https://www.senacyt.gob.pa/). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The new coronavirus disease 2019 (COVID-19) is a highly infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that has become a world-wide pandemic with huge human and economic costs [1]. Most patients with COVID-19 develop a mild to severe respiratory illness, while others develop minimally- or asymptomatic infection [2]. Several investigations have shown that asymptomatic patients can also spread the disease [3]. Since COVID-19 was declared a global pandemic, the world has suffered more that 130 million confirmed cases and 2.8 million deaths [4]. The Americas have also been strongly affected, with more than 58 million confirmed cases and 1.4 million deceased patients (as of April 11, 2021) [5, 6]. The last available situation report for Panama, dated April 6, 2021, shows 356,373 confirmed cases and 6,131 deceased [7].
Virus specific reverse transcriptase–polymerase chain reaction (RT-PCR) has become the assay of choice to rapidly detect viral RNA sequences in acute infection of either symptomatic subjects or their contacts [8]. For symptomatic individuals, the gold standard for testing is still the RT-PCR on nasopharyngeal swab samples [9, 10]. However, RT-PCR is not devoid of important limitations, such as false-negative results [11–15], variability in accuracy for different types of specimens [10], hazardous sample collection procedure [16] and sensitivity issues [11, 16].
Besides RT-PCR, the use of serological techniques has become an important tool for the management of the COVID-19 pandemic. In symptomatic individuals, serology testing may also help understand important aspects of the disease, such as the duration of virus-specific antibodies [17]. For asymptomatic individuals, serology testing contributes to answer epidemiological questions, including virus exposure in general population or in high-risk groups, to plan public health interventions, and to monitor vaccine applications and performance [17]. In fact, we recently evaluated the performance of a rapid lateral flow immunoassay to detect IgM/IgG antibodies against SARS-CoV-2 [18]. However, this type of rapid tests may present some limitations, such as not allowing quantitation of antibody titers or suboptimal sensitivity [19]. In contrast, better results may be obtained by tests based on the enzyme-linked immunosorbent assay (ELISA), which are easy and convenient to implement [20].
The humoral response of humans to the SARS-CoV-2 is apparently dominated by the production of antibodies specific for several structural proteins of the virion, mainly the Spike (S) and the Nucleocapsid (N) proteins [21]. Here we report the use of recombinant S protein, either as a full-length antigen, or just the fragment corresponding to the receptor binding domain (RBD) to assemble in-house, indirect enzyme-linked immunosorbent assays (ELISAs) for the detection of SARS-CoV-2-specific IgG and IgM antibodies. We also report the performance of the tests using a group of local COVID-19 patients and pre-pandemic serum samples collected before 2017. Our results confirm the usefulness of these antigens for the ELISA, opening the possibility for conducting new local studies for better knowledge and management of the disease in the country.
Materials and methods
Patients and samples
The study population represents a sub cohort of a cross-sectional study conducted between April and December 2020 [18]. Clinical serum samples were obtained from 102 hospitalized patients confirmed to be positive for SARS-CoV-2 viral infection by reverse transcriptase polymerase chain reaction (RT-PCR, done by a national reference lab) on nasopharyngeal swab testing. All participants develop mild to moderate clinical symptoms and were treated in public hospitals located in Panama and Colon cities. As negative control samples, 45 pre-pandemic serum samples were used from de-identified, bio-banked sera stored at INDICASAT-AIP. These samples correspond to research projects on tuberculosis serosurveillance, including 15 patients with active pulmonary tuberculosis, 15 patients with latent TB infections and 15 healthy individuals from blood bank. Venous blood samples were collected using Serum separator tubes (SST™ II advance, BD Vacutainer, VWR, No. BDAM368879). Immediately after collection, tubes were inverted six times, left at room temperature for 30 min, then centrifuged at room temperature for 10 min at 1300 x g. Serum samples were aliquoted and frozen at -80°C until testing. Prior to analysis, all serum samples were heat-inactivated at 56°C for one hour [22, 23].
Ethics statement
Clinical serum samples from COVID-19 patients were collected as part of a cross-sectional study registered with the Panama Ministry of Health (No.1462) and was approved by the National Research Bioethics Committee (CNBI; No. EC-CNBI-2020-03-43). Pre-pandemic serum samples were taken in Colon province, with ethical clearance No. 1131/CNBI/ICGES/11 and No. 125/CBI/ICGES/14. All study participants were enrolled after written informed consent was given.
Antigens
Recombinant antigens used for ELISA were donated by Dr Florian Krammer, and consisted of 1) recombinant RBD fragment, corresponding to amino acids 319–541 plus a His tail, and 2) the full spike protein, modified as described for stabilization and multimerization (aa 1–1213) [24]. Before use, recombinant protein batches were verified for concentration and integrity by SDS-PAGE (S1 Fig).
ELISAs
The ELISA assays were implemented to detect SARS-CoV-2-specific IgG and IgM antibodies specific to S or RBD antigens. Protocol was adapted from Stadlbauer et al. (2020) [22]. Briefly, 96-well high binding microtiter plates (Corning Costar no. 3361) were coated with 50 μL of RBD or S recombinant protein (both at 2 μg/mL) in 1X Phosphate Buffered Saline (PBS), overnight at 4°C. The next day, wells were washed three times with 250 μL of PBST (PBS containing 0.1% Tween 20) and blocked for 2 h at room temperature (RT) with 200 μL of 1.5% non-fat dry milk (NFM) in PBST. Plates were washed as before and serum samples added in 100 μL, prediluted 1:50 in PBST with 0.5% NFM, and incubated 2 h at RT. After similar washing, secondary conjugated antibody was added at 1:3,000 dilution in PBST with 0.5% NFM and incubated for 1 h at RT. Specific IgG and IgM antibodies were detected with anti-human HRP conjugates: goat anti-human IgG (H+L), Thermo Fisher No. H10307, or goat anti-human IgM (Heavy chain), Thermo Fisher No. A18835, respectively. Plates were then washed and 100 μL of substrate TMB (3, 3’, 5, 5’–Tetramethylbenzidine Liquid substrate, Super Sensitive, Sigma) added and incubated for 5 min. Reaction was stopped with 50 μL of 1M HCl and immediately read at 450 nm (SynergyHT, BioTek).
Statistical analysis
All statistical analyses were carried out using Prism v9.1.0 (GraphPad, USA) and easyROC online calculator (http://www.biosoft.hacettepe.edu.tr/easyROC/). Results of data analysis are presented as estimated statistics and their 95% confidence intervals (CI), or coefficients of variation (%CV). Normality of the optical density data was tested using Kolmogorov-Smirnov procedure. Correlation between OD data and evolution time (time elapsed between the positive RT-PCR result and serum sampling) was assessed using nonparametric Spearman’s rank-order correlation test. Receiver-operating characteristic (ROC) analysis was performed on ELISA OD data to calculate area under the curve (AUC) and optimal cut-off, based on procedures implemented in online calculator easyROC [25, 26]. Pairwise comparisons between the AUCs of described ELISA tests were done using the multiple comparison procedure implemented in easyROC. The performance of the tests was assessed by calculating sensitivity, specificity, and positive- and negative predictive values using contingency table analysis procedure as implemented in GraphPad Prism. Confidence intervals for each estimate was calculated following Wilson/Brown method as implemented in the same software. Based on cutoff calculated in the ROC analysis for each test, samples were classified as positive or negative, and agreement between RT-PCR test and ELISAs was assessed using Cohen`s kappa statistic, calculated as described by Fleiss et al. 2003 [27], using an online calculator (GraphPad QuickCalcs, USA). For all statistical analyses, P values < 0.05 were considered statistically significant.
Repeatability assessment was done by randomly selecting two positive serum samples, including one strongly positive (OD ~ 2) and one weakly reactive (OD ~ 0.6), and one negative. With these samples, ELISAs were repeated, in quadruplicates, over at least five different days. Then, coefficients of variation were calculated both intra- and inter-assay using OD data.
Results
We implemented and evaluated in-house, indirect ELISAs to detect SARS-CoV-2 specific IgG or IgM antibodies in serum samples, using two recombinant viral proteins, the receptor binding domain of spike (RBD), or the full-length trimeric spike as previously reported [24]. For this study, we used serum samples from two groups, 102 COVID-19 patients and 45 pre-pandemic serums obtained from patients enrolled in previous studies (Table 1). COVID-19 patients were enrolled from May 4 to December 27, 2020 at two hospital wards including: Complejo Hospitalario Dr. Arnulfo Arias Madrid in Panama City and Hospital Manuel Amador Guerrero, Colon City. All these patients had moderate symptoms and a positive RT-PCR test (median time of RT-PCR to serology, 14 days). Main clinical signs at admission were fever (76%), dysnea (72%) and cough (68%). Additionally, 73% reported to had at least one chronic disease, including hypertension (47%), type II diabetes (36%), and renal failure (11%). Pre-pandemic samples were obtained from subjects participating in tuberculosis (TB) research and surveillance programs at the Colon province, Panama.
Some COVID-19 samples were negative for all ELISA tests assayed, irrespective of their time of sampling after the RT-PCR positive result. These samples also tested negative previously in a rapid immunochromatography serology test measuring IgG and IgM [18], therefore were considered non-responsive and not considered for the rest of the study. When using 1:50 dilution of samples, strong reactivity was observed in most COVID-19 samples, while vast majority of pre-pandemic ones remain negatives (Fig 1). Some overlapping of OD values between COVID-19 and pre-pandemic samples was observed for all assays. IgG-detecting tests showed a more concentration of OD values in a strongly positive or negative zone, while OD values produced by IgMs were less defined regardless of the antigen used. A few samples (5) in the pre-pandemic group showed higher than expected values in some of the assays, particularly in the RBD-IgG ELISA (Fig 1), suggesting potential crossreactivity with pre-existing antibodies.
RBD: recombinant spike receptor binding domain. Spike: recombinant full Spike protein. Samples were processed as described in Materials and Methods. Dotted lines show the optimal cutoff for each ELISA test as determined by ROC analysis (Table 2).
To evaluate ELISA performance, we used receiver-operating characteristic (ROC) analysis using OD data (Table 2) and the area under the curve (AUC) graphs were constructed to graphically depict performance of the systems (Fig 2). The area under the curve was high for all ELISA systems, with highest value for the RBD-IgG ELISA (range 0.98–0.99) suggesting an excellent performance in all cases. By performing all possible pairwise multiple comparisons of AUC for all ELISA tests, we could not observe significant differences, with P values ranging from 0.24 (RBD-IgG vs Spike-IgM) to 0.95 (RBD-IgM vs Spike-IgM), suggesting a high performance yet very similar for all tests. Using Youden approach for selecting cutoffs based on both sensitivity and specificity, the best values were estimated in a range of 0.48 to 0.93 units of absorbance (Table 2). Using these calculated cut-off values, samples were classified as positive and negative for subsequent analyses.
RBD: recombinant spike receptor binding domain. Spike: recombinant full Spike protein. Dotted lines represent the theoretical performance of a test with no discriminatory ability, corresponding to an area under the curve (AUC) of 0.5.
All four assays showed good estimated values for sensitivity, specificity, positive and negative predictive values, and Kappa statistics. Sensitivity values were higher for the RBD-IgG ELISA (0.99, range 0.92–0.99), while specificity was better for the RBD-IgM and Spike-IgG ELISAs (both 0.97, range 0.93–0.97). Positive predictive values showed highest results for RBD-IgM (0.99, range 0.97–0.99), while negative predictive value was best for the RBD-IgG ELISA (0.97, range 0.89–0.97). Although the RT-PCR and ELISA tests would probably have very different purposes, strengths, and performances, we measured the agreement by estimating Kappa statistic. The calculated values of Cohen`s kappa (range 0.87–0.93) indicated an “almost perfect agreement” between both types of assays (Table 2), as per the interpretation scale of Landis and Koch (1977) [28].
We attempted to evaluate if there was a relationship between the time of evolution (time after the positive RT-PCR result) and optical density data for each ELISA. Normality of the OD data sets was tested using Kolmogorov-Smirnov test, resulting in rejection of normality. Therefore, correlation between OD and time variables was assessed using nonparametric Spearman’s rank-order correlation test. Optical densities did not show significant correlation with evolution time for any of the described ELISAs (RBD-IgG: P = 0.19; RBD-IgM: P = 0.80; Spike-IgG: P = 0.13; Spike-IgM: P = 0.43; S2 Fig). The same was observed by looking at the positivity rate for time period ranges (Table 3).
To assess the repeatability of the assays we also measure the coefficient of variation (%CV) both intra- and inter-assays, by selecting several negative, weak positive and strong positive sera and measure the dispersion of the resulting ODs at several replicates within a plate and among several plates in different days (Table 4). The values of CV ranged from 4% to 9%, well below the maximum tolerated of 15% for ELISA tests, indicating good performance in terms of repeatability.
Discussion
In this study, we have evaluated four in-house ELISAs for the detection of antibodies against SARS-CoV-2 proteins. RBD- and full S-based ELISAs were used to detect IgM and IgG antibodies in serum samples of RT-PCR confirmed COVID-19 patients and pre-pandemic samples. As expected, most COVID-19 patients showed a strong reactivity of both types of antibodies against both the full Spike protein and the receptor binding domain. As well, most of the pre-pandemic serum samples remains unreactive. However, some overlap was observed in the OD values of both groups of samples, for all four assays. Some samples in the COVID-19 group remain negative, a fact already observed by some authors that suggests a lack of- or a weak seroconversion [29–31].
In terms of the temporal behavior of the seroconversion, we did not observe a clear pattern of increase positivity in time. In fact, most of our COVID-19 group samples show very early seroconversion, while some remain negative irrespective of their time since RT-PCR confirmation. This unusual trend is consistent with other reports indicating a much earlier seroconversion in COVID-19 patients [32–34] and a non-typical temporal pattern of responses for IgG and IgM [35].
Interestingly, some of the samples of the pre-pandemic group showed high levels of reactivity, particularly in the RBD-IgG ELISA, suggesting possible crossreactivity with pre-existing antibodies. Several authors have shown antibody crossreactivity in the immune response to several coronaviruses in human samples [29, 36–38].
Apart from highly pathogenic SARS-CoV, MERS-CoV and SARS-CoV-2, infections by other human coronaviruses are seldom reported, particularly in Panama and other countries of the region [39]. These viruses, however, are also able to cause severe disease in children, elderly patients and immunocompromised individuals [40]. Although heavily underreported, these coronaviruses have been circulating in several countries of the region, with estimated prevalence of about 5–7% of all influenza-like infections in Latin American countries including Brazil, Costa Rica and Colombia [41]. Although we could not assess the contact status of our pre-pandemic samples with other human coronaviruses, it is conceivable that some of them may have been exposed to these endemic coronaviruses and potentially developed crossreactive antibodies.
Our study presents however some limitations. The number of samples analyzed is still low. Additionally, to assess the relationship between positivity and evolution time we used the available date of RT-PCR confirmation, and this may not be a good surrogate of infection time. When available, studies should try to use the date of symptoms onset for better estimation of seroconversion time. It is also important to note that our COVID-19 cohort only include mild to moderate patients, and this fact may have influenced the strength of the immune response observed in our study. It has been shown that severity correlates with a more intense immune response and production of specific antibodies [42–46]. A more complete picture of the serology would have been obtained with patients from all the spectrum of the disease. Next steps should include testing of RT-PCR positive, asymptomatic subjects to characterize whether the tests are able to detect lower antibody levels likely expected in such situation.
Nevertheless, our results confirm previous reports indicating the usefulness of these antigens for assays to measure seroconversion after SARS-CoV-2 infections [24, 47–49]. The ELISA tests are simple to perform and very robust once optimized. Besides, since only a very small amount of sample is required, they are very convenient. Our results are consistent with those shown by others, suggesting that these serology assays have a very good potential for studies measuring virus exposure in large groups, as well as seroconversion and seroprevalence studies in Panama. Additionally, since current vaccines being applied in Panama are based on Spike protein as immunogen, these assays may be a useful asset to study their effect and monitor the general immunological status of the population on the way to the much-needed herd immunity.
Supporting information
S1 Fig. SDS-PAGE showing approximate size and integrity of antigens used for the ELISA assays.
M, Protein size standards in kilodalton (kD); S, recombinant SARS-CoV-2 spike protein; R, recombinant SARS-CoV-2 spike fragment, corresponding to the Receptor Binding Domain (RBD).
https://doi.org/10.1371/journal.pone.0257351.s001
(TIF)
S2 Fig. Scatter plots showing the reactivity of each sample in ELISA as optical density at 450 nm, with their corresponding sampling time (since the positive RT-PCR result), for those samples for which data was available.
RBD: recombinant spike receptor binding domain. Spike: recombinant full Spike protein. Dotted lines represent the cutoff value for each ELISA.
https://doi.org/10.1371/journal.pone.0257351.s002
(TIF)
S1 Table. Minimal set of primary data, including time (time since positive RT-PCR, days), and optical density for each ELISA assay.
1, de-identified sample number; 2, time since positive RT-PCR, days; nd, not determined.
https://doi.org/10.1371/journal.pone.0257351.s003
(XLSX)
Acknowledgments
We are very thankful to Dr. Florian Krammer (Department of Microbiology, Icahn School of Medicine at Mount Sinai, New York, NY, USA) for donation of the SARS-CoV-2 recombinant proteins used in this study. We thank all enrolled patients for their willingness to participate in the study. We are very grateful to Dr. Eduardo Ortega and Dr. Jagannatha Rao for continued support and useful advice during this study. We also thank MSc. Dilcia Sambrano, Dr. Carlos Restrepo, Dr. Diana Oviedo, BSc. Ambar Pérez-Lao, Dr. Odemaris Luque, Dr. Ana de Chavez, Dr. Julio Jurado, Dr. Nicolas Hurtado, Dr. Azael Gonzalez and BSc. Laiss Mudarra at Caja de Seguro Social and Ministerio de Salud in Colon City for their support during recruitment and sampling. We also thank support of Sistema Nacional de Investigación (SNI, SENACYT, Panama) for the authors A.V., A.G., P.L., and R.L.
References
- 1.
WHO. Novel Coronavirus Situation Report -10 [Internet]. World Health Organization. 2020 [cited 2021 May 10]. Available from: https://www.who.int/docs/default-source/coronaviruse/situation-reports/20200130-sitrep-10-ncov.pdf?sfvrsn=d0b2e480_2
- 2. Grant MC, Geoghegan L, Arbyn M, Mohammed Z, McGuinness L, Clarke EL, et al. The prevalence of symptoms in 24,410 adults infected by the novel coronavirus (SARS-CoV-2; COVID-19): A systematic review and meta-analysis of 148 studies from 9 countries. PLoS One [Internet]. 2020 Jun 1 [cited 2021 May 10];15(6 June). Available from: https://pubmed.ncbi.nlm.nih.gov/32574165/ pmid:32574165
- 3. Lee S, Kim T, Lee E, Lee C, Kim H, Rhee H, et al. Clinical Course and Molecular Viral Shedding among Asymptomatic and Symptomatic Patients with SARS-CoV-2 Infection in a Community Treatment Center in the Republic of Korea. JAMA Intern Med [Internet]. 2020 [cited 2021 May 10];180(11). Available from: https://pubmed.ncbi.nlm.nih.gov/32780793/
- 4. Cucinotta D, Vanelli M. WHO declares COVID-19 a pandemic [Internet]. Vol. 91, Acta Biomedica. Mattioli 1885; 2020 [cited 2021 May 10]. p. 157–60. Available from: https://pubmed.ncbi.nlm.nih.gov/32191675/
- 5.
WHO. WHO Weekly epidemiological update on COVID-19–6 April 2021. [Internet]. World Health Organization. 2021 [cited 2021 May 10]. Available from: https://www.who.int/publications/m/item/weekly-epidemiological-update-on-covid-19—6-april-2021
- 6.
PAHO. PAHO COVID-19 Informe de situación de Panamá No.51 - Abril 6, 2021 [Internet]. PAHO. 2021 [cited 2021 May 10]. Available from: https://www.paho.org/es/documentos/covid-19-informe-situacion-panama-ndeg51-abril-6-2021
- 7.
PAHO. PAHO COVID-19 Daily Update: 11 April 2021 [Internet]. PAHO. 2021 [cited 2021 May 10]. Available from: https://www.paho.org/en/covid-19-global-and-regional-daily-update
- 8. Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DKW, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Eurosurveillance [Internet]. 2020 Jan 23 [cited 2021 May 10];25(3). Available from: https://pubmed.ncbi.nlm.nih.gov/31992387/
- 9.
Loeffelholz MJ, Tang YW. Laboratory diagnosis of emerging human coronavirus infections–the state of the art [Internet]. Vol. 9, Emerging Microbes and Infections. Taylor and Francis Ltd.; 2020 [cited 2021 May 10]. p. 747–56. Available from: https://pubmed.ncbi.nlm.nih.gov/32196430/ https://doi.org/10.1080/22221751.2020.1745095 pmid:32196430
- 10. Wang W, Xu Y, Gao R, Lu R, Han K, Wu G, et al. Detection of SARS-CoV-2 in Different Types of Clinical Specimens [Internet]. Vol. 323, JAMA—Journal of the American Medical Association. American Medical Association; 2020 [cited 2021 May 10]. p. 1843–4. Available from: https://pubmed.ncbi.nlm.nih.gov/32159775/
- 11. Hase R, Kurita T, Muranaka E, Sasazawa H, Mito H, Yano Y. A case of imported COVID-19 diagnosed by PCR-positive lower respiratory specimen but with PCR-negative throat swabs. Infect Dis (Auckl) [Internet]. 2020 Jun 2 [cited 2021 May 10];52(6):423–6. Available from: https://pubmed.ncbi.nlm.nih.gov/32238024/ pmid:32238024
- 12. Liu R, Han H, Liu F, Lv Z, Wu K, Liu Y, et al. Positive rate of RT-PCR detection of SARS-CoV-2 infection in 4880 cases from one hospital in Wuhan, China, from Jan to Feb 2020. Clin Chim Acta [Internet]. 2020 Jun 1 [cited 2021 May 10];505:172–5. Available from: https://pubmed.ncbi.nlm.nih.gov/32156607/ pmid:32156607
- 13. Baron RC, Risch L, Weber M, Thiel S, Grossmann K, Wohlwend N, et al. Frequency of serological non-responders and false-negative RT-PCR results in SARS-CoV-2 testing: A population-based study. Clin Chem Lab Med [Internet]. 2020 Nov 1 [cited 2021 May 10];58(12):2131–40. Available from: https://pubmed.ncbi.nlm.nih.gov/32866113/ pmid:32866113
- 14. Kucirka LM, Lauer SA, Laeyendecker O, Boon D, Lessler J. Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure [Internet]. Vol. 173, Annals of internal medicine. NLM (Medline); 2020 [cited 2021 May 10]. p. 262–7. Available from: https://pubmed.ncbi.nlm.nih.gov/32422057/
- 15. Long C, Xu H, Shen Q, Zhang X, Fan B, Wang C, et al. Diagnosis of the Coronavirus disease (COVID-19): rRT-PCR or CT? Eur J Radiol [Internet]. 2020 May 1 [cited 2021 May 10];126. Available from: https://pubmed.ncbi.nlm.nih.gov/32229322/
- 16. Shen M, Zhou Y, Ye J, Abdullah AL-maskri AA, Kang Y, Zeng S, et al. Recent advances and perspectives of nucleic acid detection for coronavirus [Internet]. Vol. 10, Journal of Pharmaceutical Analysis. Xi’an Jiaotong University; 2020 [cited 2021 May 10]. p. 97–101. Available from: https://pubmed.ncbi.nlm.nih.gov/32292623/
- 17.
Ferreira CE, Bonvehi PE, de la Torre JCG, Sáenz-Flor KV, Condino-Neto A. Algorithms for testing COVID-19 focused on use of RT-PCR and high-affinity serological testing: A consensus statement from a panel of Latin American experts [Internet]. Vol. 103, International Journal of Infectious Diseases. Elsevier B.V.; 2021 [cited 2021 May 10]. p. 260–7. Available from: https://pubmed.ncbi.nlm.nih.gov/33232832/ https://doi.org/10.1016/j.ijid.2020.11.173 pmid:33232832
- 18. Villarreal A, Rangel G, Zhang X, Wong D, Britton G, Fernandez PL, et al. Performance of a Point of Care Test for Detecting IgM and IgG Antibodies Against SARS-CoV-2 and Seroprevalence in Blood Donors and Health Care Workers in Panama. Front Med [Internet]. 2021 Mar 2 [cited 2021 May 10];8. Available from: https://pubmed.ncbi.nlm.nih.gov/33748157/ pmid:33748157
- 19. Lisboa Bastos M, Tavaziva G, Abidi SK, Campbell JR, Haraoui LP, Johnston JC, et al. Diagnostic accuracy of serological tests for covid-19: Systematic review and meta-analysis. BMJ [Internet]. 2020 Jul 1 [cited 2021 May 10];370. Available from: https://pubmed.ncbi.nlm.nih.gov/32611558/ pmid:32611558
- 20. Engvall E. The ELISA, enzyme-linked immunosorbent assay [Internet]. Vol. 56, Clinical Chemistry. Clin Chem; 2010 [cited 2021 May 10]. p. 319–20. Available from: https://pubmed.ncbi.nlm.nih.gov/19850633/ pmid:19850633
- 21.
Liu A, Li Y, peng J, Huang Y, Xu D. Antibody responses against SARS-CoV-2 in COVID-19 patients [Internet]. Vol. 93, Journal of Medical Virology. John Wiley and Sons Inc; 2021 [cited 2021 May 10]. p. 144–8. Available from: https://pubmed.ncbi.nlm.nih.gov/32603501/ https://doi.org/10.1002/jmv.26241 pmid:32603501
- 22. Stadlbauer D, Amanat F, Chromikova V, Jiang K, Strohmeier S, Arunkumar GA, et al. SARS-CoV-2 Seroconversion in Humans: A Detailed Protocol for a Serological Assay, Antigen Production, and Test Setup. Curr Protoc Microbiol [Internet]. 2020 Jun 1 [cited 2021 May 10];57(1). Available from: https://pubmed.ncbi.nlm.nih.gov/32302069/
- 23. Stadlbauer D, Tan J, Jiang K, Hernandez MM, Fabre S, Amanat F, et al. Repeated cross-sectional sero-monitoring of SARS-CoV-2 in New York City. Nature [Internet]. 2021 Feb 4 [cited 2021 Aug 19];590(7844):146–50. Available from: https://pubmed.ncbi.nlm.nih.gov/33142304/ pmid:33142304
- 24. Amanat F, Stadlbauer D, Strohmeier S, Nguyen THO, Chromikova V, McMahon M, et al. A serological assay to detect SARS-CoV-2 seroconversion in humans. Nat Med [Internet]. 2020 Jul 1 [cited 2021 May 10];26(7):1033–6. Available from: https://pubmed.ncbi.nlm.nih.gov/32398876/
- 25. Lopez-Raton M, Rodriguez-Alvarez M, Cadarso-Suárez C, Gude-Sampedro F. OptimalCutpoints: An R package for selecting optimal cutpoints in diagnostic tests. J Stat Softw. 2014;61(8):1–36.
- 26. Goksuluk D, Korkmaz S, Zararsiz G, Ergun Karaagaoglu A. easyROC: An Interactive Web-tool for ROC Curve Analysis Using R Language Environment. R J. 2016;8(2):213–30.
- 27.
Fleiss JL, Levin B, Paik MC. Statistical methods for rates and proportions. 3rd ed. Hoboken, New Jersey: John Wiley & Sons, Inc.; 2003.
- 28. Landis JR, Koch GG. The Measurement of Observer Agreement for Categorical Data. Biometrics [Internet]. 1977 Mar [cited 2021 May 10];33(1):159. Available from: https://pubmed.ncbi.nlm.nih.gov/843571/ pmid:843571
- 29. Faccini-Martínez ÁA, Rivero R, Garay E, García A, Mattar S, Botero Y, et al. Serological cross-reactivity using a SARS-CoV-2 ELISA test in acute Zika virus infection, Colombia. Int J Infect Dis [Internet]. 2020 Dec 1 [cited 2021 May 10];101:191–3. Available from: https://pubmed.ncbi.nlm.nih.gov/33002616/
- 30. Ng KW, Faulkner N, Cornish GH, Rosa A, Harvey R, Hussain S, et al. Preexisting and de novo humoral immunity to SARS-CoV-2 in humans. Science (80-) [Internet]. 2020 Dec 11 [cited 2021 May 10];370(6522):1339–43. Available from: https://pubmed.ncbi.nlm.nih.gov/33159009/ pmid:33159009
- 31. Oved K, Olmer L, Shemer-Avni Y, Wolf T, Supino-Rosin L, Prajgrod G, et al. Multi-center nationwide comparison of seven serology assays reveals a SARS-CoV-2 non-responding seronegative subpopulation. EClinicalMedicine [Internet]. 2020 Dec 1 [cited 2021 May 10];29–30. Available from: https://pubmed.ncbi.nlm.nih.gov/33235985/ pmid:33235985
- 32. Tan W, Lu Y, Zhang J, Wang J, Dan Y, Tan Z, et al. Viral Kinetics and Antibody Responses in Patients with COVID-19. medRxiv [Internet]. 2020 Jan 1;2020.03.24.20042382. Available from: http://medrxiv.org/content/early/2020/03/26/2020.03.24.20042382.abstract pmid:32511587
- 33. Pan Y, Li X, Yang G, Fan J, Tang Y, Zhao J, et al. Serological immunochromatographic approach in diagnosis with SARS-CoV-2 infected COVID-19 patients. J Infect [Internet]. 2020 Jul 1 [cited 2021 May 10];81(1):e28–32. Available from: https://pubmed.ncbi.nlm.nih.gov/32283141/ pmid:32283141
- 34. Suthar MS, Zimmerman MG, Kauffman RC, Mantus G, Linderman SL, Hudson WH, et al. Rapid Generation of Neutralizing Antibody Responses in COVID-19 Patients. Cell Reports Med [Internet]. 2020 Jun 23 [cited 2021 May 10];1(3). Available from: https://pubmed.ncbi.nlm.nih.gov/32835303/
- 35. Long QX, Liu BZ, Deng HJ, Wu GC, Deng K, Chen YK, et al. Antibody responses to SARS-CoV-2 in patients with COVID-19. Nat Med [Internet]. 2020 Jun 1 [cited 2021 May 10];26(6):845–8. Available from: https://pubmed.ncbi.nlm.nih.gov/32350462/ pmid:32350462
- 36. Nguyen-Contant P, Embong AK, Kanagaiah P, Chaves FA, Yang H, Branche AR, et al. S protein-reactive IGG and memory B cell production after human SARS-CoV-2 infection includes broad reactivity to the S2 subunit. MBio [Internet]. 2020 Sep 1 [cited 2021 May 10];11(5):1–11. Available from: https://pubmed.ncbi.nlm.nih.gov/32978311/
- 37. Yonker LM, Neilan AM, Bartsch Y, Patel AB, Regan J, Arya P, et al. Pediatric Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2): Clinical Presentation, Infectivity, and Immune Responses. J Pediatr [Internet]. 2020 Dec 1 [cited 2021 May 10];227:45–52.e5. Available from: https://pubmed.ncbi.nlm.nih.gov/32827525/
- 38. Ladner JT, Henson SN, Boyle AS, Engelbrektson AL, Fink ZW, Rahee F, et al. Epitope-resolved profiling of the SARS-CoV-2 antibody response identifies cross-reactivity with endemic human coronaviruses. Cell Reports Med [Internet]. 2021 Jan 19 [cited 2021 May 10];2(1). Available from: https://pubmed.ncbi.nlm.nih.gov/33495758/ pmid:33495758
- 39. Bonilla-Aldana DK, Holguin-Rivera Y, Cortes-Bonilla I, Cardona-Trujillo MC, García-Barco A, Bedoya-Arias HA, et al. Coronavirus infections reported by ProMED, February 2000–January 2020. Travel Med Infect Dis [Internet]. 2020 May 1 [cited 2021 May 10];35. Available from: https://pubmed.ncbi.nlm.nih.gov/32036011/ pmid:32036011
- 40.
Cui J, Li F, Shi ZL. Origin and evolution of pathogenic coronaviruses [Internet]. Vol. 17, Nature Reviews Microbiology. Nature Publishing Group; 2019 [cited 2021 May 10]. p. 181–92. Available from: https://pubmed.ncbi.nlm.nih.gov/30531947/ https://doi.org/10.1038/s41579-018-0118-9 pmid:30531947
- 41. Taylor S, Lopez P, Weckx L, Borja-Tabora C, Ulloa-Gutierrez R, Lazcano-Ponce E, et al. Respiratory viruses and influenza-like illness: Epidemiology and outcomes in children aged 6 months to 10 years in a multi-country population sample. J Infect [Internet]. 2017 Jan 1 [cited 2021 May 10];74(1):29–41. Available from: https://pubmed.ncbi.nlm.nih.gov/27667752/
- 42. Liu L, To KKW, Chan KH, Wong YC, Zhou R, Kwan KY, et al. High neutralizing antibody titer in intensive care unit patients with COVID-19. Emerg Microbes Infect [Internet]. 2020 [cited 2021 May 10];9(1):1–30. Available from: https://pubmed.ncbi.nlm.nih.gov/32618497/ pmid:32618497
- 43. Wellinghausen N, Plonné D, Voss M, Ivanova R, Frodl R, Deininger S. SARS-CoV-2-IgG response is different in COVID-19 outpatients and asymptomatic contact persons. J Clin Virol [Internet]. 2020 Sep 1 [cited 2021 May 10];130. Available from: https://pubmed.ncbi.nlm.nih.gov/32707511/ pmid:32707511
- 44. Lynch KL, Whitman JD, Lacanienta NP, Beckerdite EW, Kastner SA, Shy BR, et al. Magnitude and Kinetics of Anti-Severe Acute Respiratory Syndrome Coronavirus 2 Antibody Responses and Their Relationship to Disease Severity. Clin Infect Dis [Internet]. 2021 Jan 15 [cited 2021 May 10];72(2):301–8. Available from: https://pubmed.ncbi.nlm.nih.gov/33501951/ pmid:33501951
- 45. Hansen CB, Jarlhelt I, Pérez-Alós L, Hummelshøj Landsy L, Loftager M, Rosbjerg A, et al. SARS-CoV-2 Antibody Responses Are Correlated to Disease Severity in COVID-19 Convalescent Individuals. J Immunol [Internet]. 2021 Jan 1 [cited 2021 May 10];206(1):109–17. Available from: https://pubmed.ncbi.nlm.nih.gov/33208457/ pmid:33208457
- 46. Dogan M, Kozhaya L, Placek L, Gunter C, Yigit M, Hardy R, et al. SARS-CoV-2 specific antibody and neutralization assays reveal the wide range of the humoral immune response to virus. Commun Biol [Internet]. 2021 Dec 1 [cited 2021 May 10];4(1). Available from: https://pubmed.ncbi.nlm.nih.gov/33514825/ pmid:33514825
- 47. Okba NMA, Müller MA, Li W, Wang C, Geurtsvankessel CH, Corman VM, et al. Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Responses in Coronavirus Disease Patients. Emerg Infect Dis [Internet]. 2020 Jul 1 [cited 2021 May 10];26(7):1478–88. Available from: https://pubmed.ncbi.nlm.nih.gov/32267220/ pmid:32267220
- 48. Perera RAPM, Mok CKP, Tsang OTY, Lv H, Ko RLW, Wu NC, et al. Serological assays for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), March 2020. Eurosurveillance [Internet]. 2020 [cited 2021 May 10];25(16). Available from: https://pubmed.ncbi.nlm.nih.gov/32347204/ pmid:32347204
- 49. Peterhoff D, Glück V, Vogel M, Schuster P, Schütz A, Neubert P, et al. A highly specific and sensitive serological assay detects SARS-CoV-2 antibody levels in COVID-19 patients that correlate with neutralization. Infection [Internet]. 2021 Feb 1 [cited 2021 May 10];49(1):75–82. Available from: https://pubmed.ncbi.nlm.nih.gov/32827125/ pmid:32827125