Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Real-life clinical sensitivity of SARS-CoV-2 RT-PCR test in symptomatic patients

  • Elisa Kortela ,

    Contributed equally to this work with: Elisa Kortela, Vesa Kirjavainen, Maarit J. Ahava, Suvi T. Jokiranta

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Writing – original draft, Writing – review & editing

    Affiliation Division of Infectious Diseases, Inflammation Center, Helsinki University Hospital and University of Helsinki, Helsinki, Finland

  • Vesa Kirjavainen ,

    Contributed equally to this work with: Elisa Kortela, Vesa Kirjavainen, Maarit J. Ahava, Suvi T. Jokiranta

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – review & editing

    Affiliation HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  • Maarit J. Ahava ,

    Contributed equally to this work with: Elisa Kortela, Vesa Kirjavainen, Maarit J. Ahava, Suvi T. Jokiranta

    Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Visualization, Writing – review & editing

    Affiliation HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  • Suvi T. Jokiranta ,

    Contributed equally to this work with: Elisa Kortela, Vesa Kirjavainen, Maarit J. Ahava, Suvi T. Jokiranta

    Roles Data curation, Formal analysis, Investigation, Validation, Visualization, Writing – review & editing

    Affiliation Translational Immunology Research Program and Department of Bacteriology and Immunology, University of Helsinki, Helsinki, Finland

  • Anna But,

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – review & editing

    Affiliation Biostatistics Consulting, Department of Public Health, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  • Anna Lindahl,

    Roles Data curation, Formal analysis, Investigation, Writing – review & editing

    Affiliation Department of Respiratory Medicine, Heart and Lung Center, Helsinki University Hospital, University of Helsinki, Helsinki, Finland

  • Anu E. Jääskeläinen,

    Roles Investigation, Methodology, Validation, Writing – review & editing

    Affiliation HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  • Annemarjut J. Jääskeläinen,

    Roles Investigation, Methodology, Validation, Writing – review & editing

    Affiliation HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  • Asko Järvinen,

    Roles Investigation, Methodology, Validation, Writing – review & editing

    Affiliation Division of Infectious Diseases, Inflammation Center, Helsinki University Hospital and University of Helsinki, Helsinki, Finland

  • Pia Jokela,

    Roles Investigation, Methodology, Validation, Writing – review & editing

    Affiliation HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  • Hannimari Kallio-Kokko,

    Roles Investigation, Methodology, Validation, Writing – review & editing

    Affiliation HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  • Raisa Loginov,

    Roles Investigation, Methodology, Validation, Writing – review & editing

    Affiliation HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  • Laura Mannonen,

    Roles Investigation, Methodology, Visualization, Writing – review & editing

    Affiliation HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  • Eeva Ruotsalainen,

    Roles Investigation, Methodology, Validation, Writing – review & editing

    Affiliation Division of Infectious Diseases, Inflammation Center, Helsinki University Hospital and University of Helsinki, Helsinki, Finland

  • Tarja Sironen,

    Roles Investigation, Methodology, Validation, Writing – review & editing

    Affiliations Department of Virology, Faculty of Medicine, University of Helsinki, Helsinki, Finland, Department of Veterinary Biosciences, Faculty of Veterinary Medicine, University of Helsinki, Helsinki, Finland

  • Olli Vapalahti,

    Roles Investigation, Methodology, Validation, Writing – review & editing

    Affiliations HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland, Department of Virology, Faculty of Medicine, University of Helsinki, Helsinki, Finland, Department of Veterinary Biosciences, Faculty of Veterinary Medicine, University of Helsinki, Helsinki, Finland

  • Maija Lappalainen,

    Roles Funding acquisition, Investigation, Methodology, Resources, Supervision, Validation, Writing – review & editing

    Affiliation HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  • Hanna-Riikka Kreivi,

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Validation, Writing – review & editing

    Affiliation Department of Respiratory Medicine, Heart and Lung Center, Helsinki University Hospital, University of Helsinki, Helsinki, Finland

  • Hanna Jarva,

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Supervision, Validation, Writing – review & editing

    Affiliations HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland, Translational Immunology Research Program and Department of Bacteriology and Immunology, University of Helsinki, Helsinki, Finland

  • Satu Kurkela ,

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Project administration, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    satu.kurkela@helsinki.fi

    ‡ SK and ELKE are Joint Senior Authors.

    Affiliation HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland

  •  [ ... ],
  • Eliisa Kekäläinen

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

    ‡ SK and ELKE are Joint Senior Authors.

    Affiliations HUSLAB Clinical Microbiology, HUS Diagnostic Center, University of Helsinki and Helsinki University Hospital, Helsinki, Finland, Translational Immunology Research Program and Department of Bacteriology and Immunology, University of Helsinki, Helsinki, Finland

  • [ view all ]
  • [ view less ]

Abstract

Background

Understanding the false negative rates of SARS-CoV-2 RT-PCR testing is pivotal for the management of the COVID-19 pandemic and it has implications for patient management. Our aim was to determine the real-life clinical sensitivity of SARS-CoV-2 RT-PCR.

Methods

This population-based retrospective study was conducted in March–April 2020 in the Helsinki Capital Region, Finland. Adults who were clinically suspected of SARS-CoV-2 infection and underwent SARS-CoV-2 RT-PCR testing, with sufficient data in their medical records for grading of clinical suspicion were eligible. In addition to examining the first RT-PCR test of repeat-tested individuals, we also used high clinical suspicion for COVID-19 as the reference standard for calculating the sensitivity of SARS-CoV-2 RT-PCR.

Results

All 1,194 inpatients (mean [SD] age, 63.2 [18.3] years; 45.2% women) admitted to COVID-19 cohort wards during the study period were included. The outpatient cohort of 1,814 individuals (mean [SD] age, 45.4 [17.2] years; 69.1% women) was sampled from epidemiological line lists by systematic quasi-random sampling. The sensitivity (95% CI) for laboratory confirmed cases (repeat-tested patients) was 85.7% (81.5–89.1%) inpatients; 95.5% (92.2–97.5%) outpatients, 89.9% (88.2–92.1%) all. When also patients that were graded as high suspicion but never tested positive were included in the denominator, the sensitivity (95% CI) was: 67.5% (62.9–71.9%) inpatients; 34.9% (31.4–38.5%) outpatients; 47.3% (44.4–50.3%) all.

Conclusions

The clinical sensitivity of SARS-CoV-2 RT-PCR testing was only moderate at best. The relatively high false negative rates of SARS-CoV-2 RT-PCR testing need to be accounted for in clinical decision making, epidemiological interpretations, and when using RT-PCR as a reference for other tests.

Introduction

During the COVID-19 pandemic a central method for limiting the spread of SARS-CoV-2 has been the so-called “Test, Trace, Isolate” (TTI) approach promoted by the World Health Organization [1, 2]. A key feature of any laboratory test is its efficacy in detecting true positive cases. Evidence suggests a fair analytical sensitivity for the SARS-CoV-2 RT-PCR tests available on the market [3, 4]. However, reports suggest that clinically evident COVID-19 infections often go undetected by SARS-CoV-2 RT-PCR testing [59], as estimated by e.g. networked dynamic metapopulation models [6], and repeat-testing of patients [9].

A number of pivotal factors may decrease the overall sensitivity of testing and its usefulness in the TTI strategy. Preanalytical pitfalls such as suboptimal specimen collection may affect sample quality and hamper test sensitivity. Variation in viral shedding in different anatomical locations, and temporal variation in relation to disease onset can influence detection rates [10, 11].

High false negative rate complicates controlling the epidemic but it also has implications for healthcare settings [12]. Removal of infection control precautions in hospitalized patients due to a false negative test causes an occupational hazard for healthcare workers and can lead to nosocomial spread of the disease. Real-life sensitivity estimates in the initial reports [13, 14] were limited by small sample sizes and variable testing methods and reference standards.

We decided to evaluate the clinical sensitivity of SARS-CoV-2 RT-PCR in a population-based setting in the beginning of the epidemic with low level of transmission. To avoid bias created by the high pretest probability of inpatients, we included in our analysis also outpatients. We used manually curated clinical characteristics from a cohort of 3,008 individuals as the gold standard for the RT-PCR test. We focused on the sensitivity of the first SARS-CoV-2 RT-PCR test since for outpatients repeated testing is not often feasible. Our analysis that uses only non-dependent samples aims to avoid the bias created from repeated sampling of the same individuals. Our data can be directly used to inform the practicing clinicians and epidemiologists how well the RT-PCR performs in a low prevalence setting.

Materials and methods

Study design and participants

We present data from a retrospective study conducted from electronic, comprehensive medical records. The data were fully anonymized for the study. The study complies with the STARD reporting guidelines [15], and it was approved by the review board of the Helsinki University Hospital, Finland (HUS/157/2020-29), which waived the requirement for informed consent.

During the study period 4 March– 15 April 2020, 22,821 individuals underwent SARS-CoV-2 RT-PCR testing with a total of 1,938 test positive specimens at HUSLAB laboratory, Helsinki University Hospital, Finland, which serves the Helsinki Capital Area in Finland. S1 Fig shows the number of daily specimens and proportion of positive specimens at HUSLAB during the study period.

We reasoned that the pretest probability, i.e. the probability for testing positive, would be different for inpatients on the COVID-19 cohort wards and outpatients and decided to study these two populations separately.

Outpatient cohort.

During the study period, the tested patients were generally symptomatic but the criteria which prompted testing varied slightly over time (S1 Table). Initially, persons returning from recognized epidemic areas and exhibiting respiratory symptoms within 14 days of return were primarily tested. The criteria were soon expanded to include symptomatic persons with risk factors, and all symptomatic healthcare workers. Outpatients fulfilling testing criteria were recorded manually with some clinical details on a line list. These lists were the most systematically collected dataset for the tested outpatients so we chose to sample our outpatient cohort from these lists. We performed systematic (quasi-random) sampling by including every fifth individual from the line lists. Along with practical advantages, this approach decreased the probability of sampling dependent individuals, such as members of the same family. Besides the clinical details on the line lists, we checked electronic medical records for comorbidities and other demographic details. Other exclusion criteria for outpatients were age below 18 years and residence outside of the Helsinki and Uusimaa Hospital District. Altogether 1,814 eligible outpatients (mean [SD] age, 45.4 [17.2] years; 69.1% women; 41.2% healthcare workers) were included in the study (Fig 1, Table 1).

thumbnail
Fig 1. Selection of the outpatient cohort presented as a flowchart.

https://doi.org/10.1371/journal.pone.0251661.g001

thumbnail
Table 1. Patient demographic and clinical characteristics.

https://doi.org/10.1371/journal.pone.0251661.t001

Inpatient cohort.

Patients with fever, respiratory or gastrointestinal symptoms, and/or difficulty in breathing were suspected for COVID-19 and treated in designated cohort wards: 11 wards and 6 ICUs in eight hospitals (list of wards in S2 Table). All patients aged >18 years admitted to one of the cohort wards were eligible for the study and only patients without a SARS-CoV-2 RT-PCR performed at the HUSLAB laboratory were excluded. These inpatients formed a consecutive case series of 1,194 individuals (mean [SD] age, 63.2 [18.3] years; 45.2% women) (Fig 2, Table 1).

thumbnail
Fig 2. Selection of the inpatient cohort presented as a flowchart.

https://doi.org/10.1371/journal.pone.0251661.g002

Altogether 12.3% (147/1194) of inpatients and 14.1% (256/1814) of outpatients were sampled for SARS-CoV-2 RT-PCR testing more than once during a specific disease episode (Table 1). The descriptive statistics of the tested individuals and subgroups are presented in Tables 1 and 2 and S2 Fig.

thumbnail
Table 2. Patient demographic and clinical characteristics by clinical suspicion of COVID-19 in inpatients (A) and outpatients (B).

Three patients in the inpatient cohort could not be classified. ‘Not suspected’: deemed by the clinician as non-compatible with COVID-19 disease or were diagnosed with another acute disease; ‘Not excluded’: no other diagnosis recorded explaining their current symptoms, and COVID-19 disease could not be excluded. ‘High suspicion’: physician in charge of the treatment recorded the suspicion on clinical grounds to the electronic patient record, OR the patient fulfilled a set of pre-defined clinical and exposure criteria (see Methods); ‘Laboratory confirmed’: tested positive from the first sample taken or with repeated testing with SARS-CoV-2 RT-PCR.

https://doi.org/10.1371/journal.pone.0251661.t002

Index testing

SARS-CoV-2 RT-PCR testing was conducted by one of the following methods (gene targets): Cobas® SARS-CoV-2 test kit on the CobasC® 6800 system (orf1ab and E) (Roche Diagnostics, Basel, Switzerland), Amplidiag® COVID-19 test (orf1ab and N) (Mobidiag, Espoo, Finland,) and a laboratory-developed test based on a protocol recommended by WHO (N) (non-exponential amplification curves and amplification with cycle treshold values >34 in the laboratory-developed test were reanalysed with either Cobas or Amplidiag) [16]. The specifics and analytical performance of these methods in our laboratory setting have been described previously [4]. Samples were collected with nasopharyngeal swabs (FLOQSwab, Copan, Brescia, Italy) (proportion of nasopharyngeal samples: 60% inpatients, 58% outpatients) but oropharyngeal swabs were used in a proportion of patients (11.1% inpatients, 16.4% outpatients) due to global shortage of nasopharyngeal swabs. Other specimens types (0.7% inpatients, 0.2% outpatients) were tracheal, brochial, and sputum specimens, as well as sinus and lung biopsies The specimen type was unknown in 28.2% of inpatients and 25.5% of outpatients.

Samples were analysed in median 24 hours after collection. As per our laboratory’s standard operating procedure, samples with failed results were reanalysed and only qualified results were included. 13 weak positive results which became positive in >35 PCR cycles (7 outpatients and 6 inpatients) were included.

Reference standard used in the study

Since no gold standard for COVID-19 diagnosis exists, we decided to use high clinical suspicion for COVID-19 as the reference standard for the RT-PCR test. We systemically graded the clinical suspicion for COVID-19 based on a combination of symptoms, clinical findings, and recorded exposure to laboratory confirmed COVID-19 cases or travel history to epidemic areas. The criteria were based on CDC’s and ECDC’s case definitions for COVID-19 in April 2020. Electronic patient records or line lists were reviewed by a team consisting of senior residents in Infectious Diseases and Clinical Microbiology, medical students, and research nurses. Patients’ medical history, symptoms, and epidemiological information were collected into a Microsoft Access® database according to the pre-defined criteria. Chest X-ray and CT findings indicating chest infection were recorded according to radiologist’s interpretation. The team collecting the data were aware of the SARS-CoV-2 RT-PCR test result when collecting the data.

The clinical suspicion for COVID-19 disease was graded as follows:

  1. ‘Not suspected’ patients were deemed by the clinician as non-compatible with COVID-19 disease or were diagnosed with another acute disease.
  2. ‘Not excluded’ patients had no other diagnosis recorded explaining their current symptoms, and COVID-19 disease could not be excluded.
  3. ‘High suspicion’ patients were considered to suffer from a probable COVID-19 if the physician in charge of the treatment recorded the suspicion on clinical grounds to the electronic patient record, OR the patient fulfilled at least one of the following criteria:
    1. respiratory symptoms and/or fever and/or diagnostic finding for infection in chest X-ray/CT and travel history to epidemic regions at the time of the study i.e. Tirol/Austria, Northern Italy, Spain, Iran, South Korea, or China during the preceding 14 days.
    2. respiratory symptoms and fever and diagnostic finding in chest X-ray/CT during April 2020 (time criterion based on the changed epidemiological situation).
    3. respiratory or gastrointestinal symptoms or fever or diagnostic finding in chest X-ray/CT and a close contact with a laboratory confirmed COVID-19 patient during the preceding 14 days prior to disease onset.
  4. ‘Laboratory confirmed’ patients (regardless of their clinical presentation) were those individuals that tested positive for SARS-CoV-2 RT-PCR during the study period. At the time of the study, only symptomatic individuals were tested, so the laboratory confirmed group does not include any asymptomatic cases.

Sample size calculation

We estimated the minimum sample size needed for outpatients based on Bujang et al. [17], with a minimal statistical power of 80% and type I error <0.05. Sample size calculation for sensitivity requires a prevalence estimation in the target population. During the study period, the median positivity rate was 9.6% (S1 Fig) so we estimated a 10% prevalence for the tested population. Published estimates from small cohorts [18, 19] available at the time reported sensitivity of the SARS-CoV-2 RT-PCR to be on average 70%. Based on these estimates the minimum sample size of outpatients for null hypothesis of sensitivity of 70% was 1,550. We performed another sample size calculation by using the nomogram described by Carley et al. [20] which accounts for confidence intervals (CI): with CI of 93% and prevalence 10%, 70% sensitivity would require a minimum sample size of 1,600.

Group comparisons

To detect if the high suspicion and the laboratory confirmed groups were comparable and if there would be significant confounding factors between the groups that were used for sensitivity calculations, we compared demographic and clinical characteristics between them (Tables 1 and 2). To compare these two groups with respect to the categorical variables, we used the Pearson’s Chi-squared test without or with Yate’s correction for continuity or the Fisher’s exact test, as appropriate. For the extensive contingency tables with the excess of small (expected) frequencies, we assessed the simulated p-value of the Fisher’s test based on 20,000 replicates. The differences in the age distribution were assessed using the Mann-Whitney U-test. These comparisons were performed separately within the inpatients and outpatients.

Analysis of sensitivity

Two approaches were deployed to calculate SARS-CoV-2 RT-PCR sensitivity: repeat-tested laboratory confirmed patients, and patients with a high clinical suspicion of COVID-19.

For the laboratory confirmed patients, the sensitivity values were calculated based on the first RT-PCR test of each patient. All patients who tested RT-PCR positive during a specific disease episode were considered laboratory confirmed. Of these, the first samples with a negative RT-PCR test result were considered false negatives, while the first samples with a positive result were considered true positives. The same disease episode would include samples taken ≤14 days apart.

For the high clinical suspicion group, the sensitivity was calculated by considering those patients that were graded as high suspicion but never tested positive as false negative cases.

The 95% CIs for (binomial) sensitivity were calculated by using the Wilson-Score method, which is based on inverting the z-test for a single proportion and provides more reliable coverage than the alternatives. We performed comparisons of sensitivity between the subgroups by using the independent sample tests for binomial proportions, including Chi-squared test without or with Yate’s correction for continuity or the Fisher’s exact test, as appropriate. For the extensive contingency tables with the excess of small (expected) frequencies, we assessed the simulated p-value of the Fisher’s test based on 20,000 replicates. We set the confidence level at 5%. All calculations were performed using the R software.

Results

Demographics of the study population and clinical comparison between study groups

In all, 3,008 individuals were eligible for this study (Figs 1 and 2): 1,814 outpatients and 1,194 inpatients. Altogether 83 eligible outpatients (4.6%) and 3 inpatients (0.3%) were excluded from the final analysis due to insufficient data for the grading of clinical suspicion.

The inpatients were on average older than outpatients, comorbidities were more common, and the male sex was slightly overrepresented (Table 1). Healthcare workers and women were overrepresented in the outpatient population reflecting the distribution of the whole tested population, as reported before [21].

All patients were categorized by a clinical grade of suspicion (not suspected / not excluded / high suspicion / laboratory confirmed) for COVID-19 based on criteria described in Methods (Table 1). To detect if our grading created systematic bias or if there were significant confounding factors present between the groups, we compared test negative patients that were deemed as high suspicion to laboratory confirmed patients (Table 2). There were no significant differences in sex or age distribution between these groups, but patients treated in the intensive care unit were overrepresented in the laboratory confirmed hospitalized patients (Table 2A). Laboratory confirmed patients were also more often febrile and had had contact with laboratory confirmed symptomatic COVID-19 cases (Table 2A and 2B). Laboratory confirmed patients also had more often gastrointestinal symptoms than patients in the high suspicion group (Table 2A and 2B).

In the outpatients, the high suspicion group had a higher proportion of females and healthcare workers compared to laboratory confirmed cases. This was expected based on the overall higher testing rate of both [21]. Again, the laboratory confirmed cases were more often febrile. Since our grading criteria included exposure to symptomatic COVID-19 patients or travel to epidemic areas, these factors were more common in the high suspicion group that tested negative, than in the laboratory confirmed group (Table 2B).

Sensitivity of the first SARS-CoV-2 RT-PCR in inpatients and outpatients

The sensitivity of SARS-CoV-2 RT-PCR was calculated with two different denominators (Table 3). We first calculated the sensitivity with laboratory confirmed cases, i.e. repeat-tested patients as a denominator, yielding the highest sensitivity estimates in this study, as follows: 85.7% for inpatients; 95.5% for outpatients, and 89.9% for all. Due to low number of repeat-tested patients (N = 11), the calculation for outpatients here is unreliable.

thumbnail
Table 3. Sensitivity calculations for the first SARS-CoV-2 RT-PCR test.

https://doi.org/10.1371/journal.pone.0251661.t003

The sensitivity was then calculated by including in the denominator patients that were graded as high suspicion but never tested positive (from one or more tests conducted within the study period), yielding the following sensitivity values: 67.5% for inpatients, 34.9% for outpatients and 47.3% for all. Thus, the lowest calculated sensitivity estimate in this study was for outpatients with high suspicion.

The delay between disease onset and testing was longer for inpatients than outpatients (Table 2). We could not detect a significant difference in the delay to first test between the laboratory confirmed cases and the high suspicion group in either cohort (S3 Fig, Fisher’s Exact Test p = 1 when “No data” category excluded). However, for inpatients, information on the delay was missing more often in the high suspicion group (2.3%) compared to the laboratory confirmed (0.9%, p = 0.026) (Table 2). For outpatients, information on the delay was missing less often in the high suspicion group (10.5%) as compared to the laboratory confirmed (12.2% p = 0.026) (Table 2).

We could not detect a significant difference between the sensitivity of nasopharyngeal and oropharyngeal samples in the inpatients (p = 0.51, Chi-squared test), outpatients (p = 0.22) or all patients (p = 0.66) (S3 Table; S4 Fig). However, data on the specimen type was missing in 20.4% (inpatients) and 17.4% (outpatients) of the cases.

Delay between symptom onset and positive test result

To estimate the delay from disease onset for highest clinical sensitivity for SARS-CoV-2 RT-PCR, we calculated sensitivities for different time frames. To achieve reliable group sizes, both cohorts were pooled together. There was no significant difference in the test sensitivity according to delay from onset, calculated for the laboratory confirmed cases alone, and with the high clinical suspicion group included (P = 0.1013 Fisher’s Exact Test for Count Data with simulated p-value, based on 20000 replicates; Fig 3). Detailed sensitivity calculations per delay from disease onset are presented in S4 Table.

thumbnail
Fig 3. Clinical SARS-CoV-2 RT-PCR sensitivity estimates in the laboratory confirmed, and in the laboratory confirmed and high suspicion group combined, presented according to delay (days) from symptom onset to sampling.

https://doi.org/10.1371/journal.pone.0251661.g003

Discussion

Wide-spread testing and contact tracing together with social distancing has been promoted as the tool that prevents new lockdowns–without clear understanding of how well the SARS-CoV-2 RT-PCR test performs. Here, we used clinical suspicion as the gold standard to estimate the clinical sensitivity of the test. We also included a sensitivity calculation based on the repeat-tested individuals where RT-PCR acts as the gold standard for itself.

A previous large-scale sensitivity estimate for SARS-CoV-2 molecular testing was based only on repeat-tested individuals [9, 22]. This approach overestimates the sensitivity. Repeated testing is done mostly on inpatients who have a strong clinical suspicion, rendering high pre-test probability. We sought to overcome this limitation by including a large cohort of outpatients. From an epidemiological point-of-view, understanding the clinical sensitivity for mild cases is important. An RT-PCR sensitivity of 64% for exposed family members systemically tested with serology was recently reported [23]. This is in line with our sensitivity estimation for inpatients. Another small study found an 86.2% RT-PCR sensitivity in symptomatic COVID-19 patients in comparison with convalescent antibody [24]. Interestingly, a recent Cochrane meta-analysis on thoracic imaging of COVID-19 patients found an 87.9% pooled sensitivity for chest CT and 80.6% for chest X-ray [25]. However, diagnostic imaging is mainly performed on inpatients that can overestimate the sensitivity. Our analysis was done in a low prevalence setting. Thus, the negative predictive value for the RT-PCR test was high (89%) for the outpatients even though the clinical sensitivity was low (35%), assuming all COVID-19 excluded cases were true negatives. High false negative rates reduce the negative predictive value of testing. This is particularly problematic when the prevalence of the disease increases. In such settings, it will impair effective use of wide-spread testing.

For health-care facilities the message of our data is different: a single negative result cannot be trusted to rule out COVID-19 in patients with suitable symptoms. Our data show that the sensitivity of the repeat-tested inpatients was high (86%), and in line with previous reports on repeated testing [9, 22]. When the sensitivity of the COVID-19 PCR test was judged based on the laboratory confirmed and high clinical suspicion patients the estimated sensitivity of the test dropped to around 68%. Our results emphasize the importance of repeated sampling but it also highlights the importance to evaluate the patient’s clinical presentation carefully.

This study estimated test sensitivity both with repeat-tested patients and by using clinical suspicion as a gold standard. The estimated sensitivity (89.9%) for repeat-tested patients is probably an overestimation: samples from a single individual are not independent and there is often a clear clinical rationale, i.e. high clinical suspicion for COVID-19 that necessitates the repeated testing, leading to increased pretest probability. In addition, hospitalized patients often present with a more severe disease with higher viral loads and longer viral nucleic acid shedding than individuals with mild symptoms [26], again leading to overestimation of test sensitivity. Our data shows that laboratory confirmed cases in both outpatients and hospitalized patients were more often febrile than the high clinical suspicion cases even though almost all other symptoms were comparable between the groups. This could indicate that the more severe cases were more often detected with RT-PCR. In contrast, the estimate which included both laboratory confirmed and high suspicion outpatients (34.9%), is likely an underestimation as COVID-19 symptoms are shared with other respiratory infections. In all, we conclude that the group which included both laboratory confirmed and high suspicion inpatients likely yielded the most realistic sensitivity estimate (67.5%).

Generally the first symptomatic days are considered best for virus detection from the upper airways [27, 28]. Due to the limited sample size our analysis could not detect a definitive time-point for highest sensitivity. However, even with this under-powered estimation, we should have detected major trends.

The study had several limitations. All patients were considered symptomatic so the estimates cannot be generalized to asymptomatic patients. The clinical criteria were set based on the information available in April 2020. While the core symptoms have remained the same, understanding of COVID-19 presentations has since increased. The clinical diagnosis of COVID-19 is notoriously hard as the symptoms are variable and overlap with many other similar conditions. In the hospitalized patients the testing coverage for other viral pathogens was extensive, and circulation of influenza and RSV was very limited at the time. However, most outpatients in this study were not tested for other potential viral pathogens. Potential information bias was introduced by the sometimes undetailed clinical records of outpatients. Reporting bias for more detailed symptoms most likely exists, especially for the laboratory confirmed outpatient cases. The specimen types recorded in the sample referrals may have contained errors. While pre-defined clinical criteria were used for grading, the data were collected retrospectively and the data collectors were aware of the index test result.

Large scale molecular testing has permanently changed the practice of clinical microbiology. RT-PCR for SARS-CoV-2 detection has many limitations as a labor intensive test with a relatively slow throughput. This has led to unbearable delays in results. Multiple solutions are being developed: point-of-care viral antigen detection [29], sample pooling [30], and self-sampling [31]. All these approaches, which use RT-PCR as a reference, quite consistently report lower sensitivity than RT-PCR. It is thus evident that all our current testing options are far from optimal in detecting all COVID-19 cases. In controlling of the ongoing pandemic, we need focused research to find an appropriate balance in the tradeoff between test sensitivity, and speed and ease of testing in each epidemiological setting.

Supporting information

S1 Fig. Number of SARS-CoV-2 RT-PCR tests conducted at the HUSLAB laboratory, and positivity rate over the study period 4 March– 15 April 2020.

Median and mean positivity rates were 9.6% and 10%, respectively.

https://doi.org/10.1371/journal.pone.0251661.s001

(TIF)

S2 Fig. Age distribution of the study population as histograms.

A. Inpatients. B. Outpatients.

https://doi.org/10.1371/journal.pone.0251661.s002

(TIFF)

S3 Fig. Delay to first SARS-CoV2 RT-PCR test (days) in the laboratory confirmed cases and the high suspicion group.

A. Inpatients B. Outpatients.

https://doi.org/10.1371/journal.pone.0251661.s003

(TIFF)

S4 Fig. Visualisation of clinical sensitivity estimates for inpatients and outpatients according to specimen type.

https://doi.org/10.1371/journal.pone.0251661.s004

(TIFF)

S1 Table. List of SARS-CoV-2 RT-PCR testing criteria over the study period.

https://doi.org/10.1371/journal.pone.0251661.s005

(DOCX)

S2 Table. List of COVID-19 cohort wards.

Inpatients were admitted to these COVID-19 cohort wards during the study period. Both laboratory confirmed and suspected COVID-19 patients were admitted to the cohort wards, except for two wards (COV_KNKINF and COV_KITEHO) to which only laboratory confirmed cases were admitted.

https://doi.org/10.1371/journal.pone.0251661.s006

(DOCX)

S3 Table. Estimated SARS-CoV-2 RT-PCR sensitivity values in the laboratory confirmed and high suspicion group combined according to specimen type in the first SARS-CoV-2 RT-PCR test.

https://doi.org/10.1371/journal.pone.0251661.s007

(DOCX)

S4 Table. Estimated SARS-CoV-2 RT-PCR sensitivity values in the laboratory confirmed, and in the laboratory confirmed and high suspicion group combined according to delay (days) from symptom onset to first SARS-CoV-2 RT-PCR test.

https://doi.org/10.1371/journal.pone.0251661.s008

(DOCX)

Acknowledgments

We thank study nurses Kerstin Ahlskog and Kirsi Lindgren (Dept. of Respiratory Medicine, Helsinki University Hospital) for their help in data collection, and Dr. Lasse Lönnqvist and Prof. Marjukka Myllärniemi for their support during the study and valuable comments on the manuscript.

References

  1. 1. WHO Overview of public health and social measures in the context of COVID-19. Available from https://www.who.int/publications/i/item/overview-of-public-health-and-social-measures-in-the-context-of-covid-19.
  2. 2. Kucharski AJ, Klepac P, Conlan A, Kissler SM, Tang M, Fry H, et al. Effectiveness of isolation, testing, contact tracing and physical distancing on reducing transmission of SARS-CoV-2 in different settings. Lancet 2020 Apr 29,;20(10):1151–1160. pmid:32559451
  3. 3. Moore NM, Li H, Schejbal D, Lindsley J, Hayden MK. Comparison of two commercial molecular tests and a laboratory-developed modification of the CDC 2019-nCoV RT-PCR assay for the detection of SARS-CoV-2. J Clin Microbiol 2020 May 27;58(8):938. pmid:32461287
  4. 4. Mannonen L, Kallio-Kokko H, Loginov R, Jääskeläinen A, Jokela P, Antikainen J, et al. Comparison of two commercial platforms and a laboratory developed test for detection of SARS-CoV-2 RNA. J Mol Diagn. 2021;23(4):407–416. pmid:33486074
  5. 5. Arevalo-Rodriguez I, Buitrago-Garcia D, Simancas-Racines D, Zambrano-Achig P, Campo RD, Ciapponi A, et al. False-negative results of initial RT-PCR assays for COVID-19: A systematic review. PLoS One 2020 Dec 1,;15(12 December):e0242958. pmid:33301459
  6. 6. Li R, Pei S, Chen B, Song Y, Zhang T, Yang W, et al. Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV2). Science 2020;368(6490):eabb3221–493.
  7. 7. Lippi G, Simundic A, Plebani M. Potential preanalytical and analytical vulnerabilities in the laboratory diagnosis of coronavirus disease 2019 (COVID-19). Clin Chem Lab Med 2020; 58(7):1070–1076. pmid:32172228
  8. 8. Woloshin S, Patel N, Kesselheim AS. False Negative Tests for SARS-CoV-2 Infection—Challenges and Implications. N Engl J Med 2020;383(6):e38. pmid:32502334
  9. 9. Green DA, Zucker J, Westblade LF, Whittier S, Rennert H, Velu P, et al. Clinical Performance of SARS-CoV-2 Molecular Tests. J Clin Microbiol 2020;58(8):995. pmid:32513858
  10. 10. Wölfel R, Corman VM, Guggemos W, Seilmaier M, Zange S, Müller MA, et al. Virological assessment of hospitalized patients with COVID-2019. Nature 2020;581(7809):465–469. pmid:32235945
  11. 11. Zheng S, Fan J, Yu F, Feng B, Lou B, Zou Q, et al. Viral load dynamics and disease severity in patients infected with SARS-CoV-2 in Zhejiang province, China, January-March 2020: retrospective cohort study. BMJ 2020 Apr 21,;369(8245):m1443. pmid:32317267
  12. 12. Incorporating Test Characteristics Into SARS-CoV-2 Testing Policy—Sense and Sensitivity. JAMA Health Forum: American Medical Association; 2020. Available from: https://jamanetwork.com/channels/health-forum/fullarticle/2764750
  13. 13. Zhao J, Yuan Q, Wang H, Liu W, Liao X, Su Y, et al. Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease 2019. Clin Infect Dis. 2020;71(16):2027–2034.
  14. 14. Yang W, Sirajuddin A, Zhang X, Liu G, Teng Z, Zhao S, et al. The role of imaging in 2019 novel coronavirus pneumonia (COVID-19). Eur Radiol 2020;30(9):4874–4882. pmid:32296940
  15. 15. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. Clin Chem 2015;61(12):1446–1452. pmid:26510957
  16. 16. Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DK, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill 2020;25(3):01.
  17. 17. Bujang MA, Adnan TH. Requirements for minimum sample size for sensitivity and specificity analysis. Journal of clinical and diagnostic research: J Clin Diagn Res 2016;10(10):YE01. pmid:27891446
  18. 18. Wang Y, Kang H, Liu X, Tong Z. Combination of RT-qPCR testing and clinical features for diagnosis of COVID-19 facilitates management of SARS-CoV-2 outbreak. J Med Virol 2020;92(6):538–539. pmid:32096564
  19. 19. Xiao AT, Tong YX, Zhang S. False negative of RT-PCR and prolonged nucleic acid conversion in COVID-19: Rather than recurrence. J Med Virol 2020;92(10):1755–1756. pmid:32270882
  20. 20. Carley S, Dosman S, Jones SR, Harrison M. Simple nomograms to calculate sample size in diagnostic studies. Emerg Med J 2005 Mar;22(3):180–181. pmid:15735264
  21. 21. Jarva H, Lappalainen M, Luomala O, Jokela P, Jääskeläinen AE, Jääskeläinen AJ, et al. Laboratory-based surveillance of COVID-19 in the Greater Helsinki area, Finland, February–June 2020. Int J Infect Dis 2021;104:111–116. pmid:33352330
  22. 22. Williams TC, Wastnedge E, McAllister G, Bhatia R, Cuschieri K, Kefala K, et al. Sensitivity of RT-PCR testing of upper respiratory tract samples for SARS-CoV-2 in hospitalised patients: a retrospective cohort study. Wellcome Open Res 2020;5:254.
  23. 23. Zhang Z, Bi Q, Fang S, Wei L, Wang X, He J, et al. Insight into the practical performance of RT-PCR testing for SARS-CoV-2 using serological data: a cohort study. Lancet Microbe 2021;2(2):e79–e87. pmid:33495759
  24. 24. Holborow A, Asad H, Porter L, Tidswell P, Johnston C, Blyth I, et al. The clinical sensitivity of a single SARS-CoV-2 upper respiratory tract RT-PCR test for diagnosing COVID-19 using convalescent antibody as a comparator. Clin Med (Lond) 2020;20(6):e209–e211. pmid:32917746
  25. 25. Islam N, Ebrahimzadeh S, Salameh JP, Kazi S, Fabiano N, Treanor L, et al. Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev 2021;3:CD013639. pmid:33724443
  26. 26. Fu Y, Li Y, Guo E, He L, Liu J, Yang B, et al. Dynamics and Correlation Among Viral Positivity, Seroconversion, and Disease Severity in COVID-19: A Retrospective Study. Ann Intern Med 2020;M20–3337.
  27. 27. He X, Lau EHY, Wu P, Deng X, Wang J, Hao X, et al. Temporal dynamics in viral shedding and transmissibility of COVID-19. Nat Med 2020;26(5):672–675. pmid:32296168
  28. 28. Kucirka LM, Lauer SA, Laeyendecker O, Boon D, Lessler J. Variation in False-Negative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure. Ann Intern Med 2020;173(4):262–267. pmid:32422057
  29. 29. Dinnes J, Deeks JJ, Adriano A, Berhane S, Davenport C, Dittrich S, et al. Rapid, point‐of‐care antigen and molecular‐based tests for diagnosis of SARS‐CoV‐2 infection. Cochrane Database Syst Rev 2021 3:CD013705. pmid:33760236
  30. 30. Ben-Ami R, Klochendler A, Seidel M, Sido T, Gurel-Gurevich O, Yassour M, et al. Large-scale implementation of pooled RNA extraction and RT-PCR for SARS-CoV-2 detection. Clin Microbiol Infect 2020;26(9):1248–1253. pmid:32585353
  31. 31. Tu Y, Cangelosi GA, Jennings R, Wehber K, Verma P, Wood RC, et al. Swabs Collected by Patients or Health Care Workers for SARS-CoV-2 Testing. N Engl J Med 2020;383(5):494–496. pmid:32492294