Introduction

Seasonal influenza poses a significant annual disease burden [1–3] and is common in outpatient clinical settings. Although clinicians often diagnose possible influenza based on patient history and symptoms, studies indicate symptoms alone perform inadequately for influenza [4,5]. Rapid influenza diagnostic tests (RIDTs) have been shown to significantly improve the accuracy of physicians’ estimates of influenza during peak influenza season [6]. RIDTs allow laboratory confirmation of clinically-suspected influenza cases within a timeframe that is clinically meaningful in primary care and urgent care settings and therapeutically meaningful for prompt initiation of antiviral therapy and avoidance of inappropriate antibiotic prescribing [7].

Point-of-care RIDTs are easy to use, relatively inexpensive, and provide results in as little as 15 minutes. Most RIDTs are highly specific (>95%), but they exhibit varying and often low sensitivity [8–10] when compared to reverse transcription–polymerase chain reaction (RT-PCR). Thus, the Center for Disease Control and Prevention (CDC) advises caution when interpreting negative test results [11].

Clinicians are rarely provided adequate training on the performance characteristics of RIDTs and approaches for an informed interpretation of results. Accordingly, falsely negative results can lead to missed opportunities or errors in treatment and patient education; falsely positive results may lead to inappropriate treatment.

Understanding which factors influence sensitivity may enable clinicians to better select appropriate patients for testing and improve their ability to interpret RIDT results. Pragmatic RIDT operating characteristics, however, have not been well defined in primary care settings. Studies that attempt to identify potential factors often limit the number of assessed variables. Several studies examined only two potential factors [6,12–14]. Age and virus strain were assessed most frequently [6,12–19]. Other factors included timing of illness onset relative to testing and illness severity [16–18], specimen collection method [19], and viral load or influenza RT-PCR cycle threshold (Ct) value [16,20–22]. To our knowledge, no study has examined the effect of within-season influenza vaccination status on sensitivity, or combined multiple clinically-relevant factors simultaneously.

We performed a post-hoc assessment of a large surveillance dataset to evaluate multiple clinical and laboratory factors that may affect RIDT sensitivity using seven years of data amassed from five primary care clinics that sequentially employed Sofia® Influenza A+B Immunoassay (Sofia-FIA; Quidel Corporation) for point-of-care influenza testing. We focused primarily on factors that clinicians could take into consideration during a clinical encounter. Our a priori hypotheses were that sensitivity declined with increasing age and with increasing time from illness onset. We were also interested in the combined roles of sex, influenza vaccination status, influenza type, meeting influenza-like illness (ILI) criteria, and influenza RT-PCR Ct value.

Methods

The University of Wisconsin Health Sciences Minimal Risk Institutional Review Board deemed protocols exempt and classified the project as clinical care and public health surveillance. Thus, patients were not required to sign a consent form to be enrolled in the surveillance program.

Procedures

Surveillance program staff provided a brief initial training to all clinicians and trained all incoming family medicine residents. Clinicians collected extensive demographic, epidemiologic, and symptom data (Table 1) on each patient along with paired respiratory specimens: (a) anterior nasal specimen using a nasal swab (Pur-Wraps®) and (b) either a nasopharyngeal (NP) or a high oropharyngeal (OP) specimen using a flocked swab (Copan®). The anterior nasal swab was returned to its paper sheath and immediately transferred to the on-site clinical laboratory. Sofia-FIA was performed on the anterior nasal swab specimen at the time of the patient visit, thus allowing for clinical decision-making. Clinic laboratory technicians followed laboratory-approved procedures for Sofia-FIA, as detailed in the package insert [24]. The NP/OP swab was immediately placed into a labeled 3 ml viral transport medium (Remel MicroTest™ M4RT®) tube, kept at 4–8° Celsius, and shipped with a requisition form to the Wisconsin State Lab of Hygiene (WSLH) via courier, usually within 24 hours of collection. The WSLH requisition form contained the Sofia-FIA results, patient demographic information, and clinical information, including number of days from illness onset, symptoms, and whether patients received an influenza vaccine prior to their illness. Surveillance staff confirmed vaccine status through the Wisconsin Immunization Registry [25].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Routinely collected data for patients with acute respiratory infection selected for influenza surveillance at 5 primary care clinics.

https://doi.org/10.1371/journal.pone.0268279.t001

Results

A total of 5,989 respiratory specimens were collected from October 26, 2012 through June 30, 2019. We excluded 1,514 specimens due to one or more of the following criteria: (1) Sofia RIDT was not performed (n = 1,259 RIDT results were obtained with QuickVue); (2) specimen collected >7 days after illness onset (n = 222); and (3) RT-PCR was not performed (n = 33). We analyzed 4,475 paired specimens, of which one pair was missing symptom information so negatives were imputed for the symptoms. A detailed description of how samples were selected for data analysis is provided in Fig 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1.

https://doi.org/10.1371/journal.pone.0268279.g001

The demographics of the surveillance population were reflective of general primary care populations with a broad range in patient ages (0.0–98.8 years) and a majority of female patients (Table 2). The majority of patients evaluated for ARI met the ILI criteria (57.4%) and presented for care an average of 3.47 days after symptom onset. Patients presented most commonly with cough (83.7%), nasal discharge (75.6%), sore throat (63.4%), and fever (60.6%). The influenza vaccination rate of 46.2% was slightly above state and national averages [30].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Demographics and distribution of sample characteristics of 4,475 primary care patients presenting with acute respiratory infections and selected for influenza surveillance.

https://doi.org/10.1371/journal.pone.0268279.t002

RIDT performance characteristics

Sofia-FIA detected the presence of influenza in 874 specimens, 774 of which were confirmed influenza-positive by RT-PCR (PPV 88.6%). Overall sensitivity of Sofia-FIA for influenza A was 66.2% (95% confidence interval: 63.0–69.3), with a specificity of 97.9% (97.4–98.3). For influenza B, sensitivity was also 66.2% (60.6–71.4) while the specificity was 97.4% (96.8–97.8). A summary of Sofia-FIA performance statistics can be found in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Summary performance statistics for Sofia® Influenza A + B fluorescent immunoassay.

https://doi.org/10.1371/journal.pone.0268279.t003

Clinical and laboratory predictors

In an unadjusted analysis, greater sensitivity was associated with the following factors: illness meeting ILI criteria, no non-influenza virus co-detection (RPP), no seasonal influenza vaccination, younger age, lower Ct value, fewer days since illness onset, and presence of nasal discharge, nasal congestion, and fever (p < .05, Table 4A and 4B). Factors not significantly associated with sensitivity were sex, influenza type, illness severity, and seasonality (early, peak, and late influenza season) and all other recorded symptoms. None of the factors were significantly associated with specificity.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. a: Unadjusted analyses of the effects of clinical and laboratory factors on Sofia® Influenza A + B fluorescent immunoassay sensitivity as compared to RT-PCR.

Subjects were excluded from analysis if necessary data was missing (1 from sex analysis, 24 from severity analysis). Significant results indicated with an asterisk. b: Unadjusted analysis of the effect of age and Ct value and days from illness onset to specimen collection on sensitivity of Sofia® Influenza A + B fluorescent immunoassay. Significant results indicated with an asterisk.

https://doi.org/10.1371/journal.pone.0268279.t004

After adjustment, illness meeting ILI criteria, younger age, fewer days from onset, no co-detection, and presence of a nasal discharge maintained significance. Sensitivity was significantly improved by including a quadratic term for age (LRT p = 0.010). Adjusted odds ratios, confidence intervals, and p-values can be found in Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Odds ratios with 95% confidence intervals for clinical and laboratory factors used in a sensitivity model based on a referent sample from a female patient with an ILI, moderate severity, no flu vaccination, no non-influenza virus co-detection, during peak flu season, and with no symptoms.

Age estimates are per 5 years and centered on the median age for this set (36.1 years). Estimates for days from onset are per 1 additional day. Red triangles depict factors that lower sensitivity. Green triangles depict factors that increase sensitivity.

https://doi.org/10.1371/journal.pone.0268279.g002

Again, no factors were significantly associated with specificity in the adjusted analysis.

Discussion

We found that increased sensitivity of Sofia-RIDT was associated with four clinical factors that are readily identifiable at the time of a medical evaluation (presence of an ILI, younger age, fewer days from illness onset, and presence of nasal discharge), and one additional factor that would not be discernable by a clinician (no co-detection of additional viruses in a respiratory pathogen panel). All of these factors are likely to increase influenza antigenic load in the anterior nares. No factors significantly affected specificity in either unadjusted or adjusted analyses.

Compared with other studies, we had a considerably larger sample size over several consecutive years that included seven sequential influenza seasons. In a 2012 meta-analysis of 159 studies assessing RIDT accuracy, the average sample size was 131 confirmed cases of influenza [31]. Only two of the studies referenced had over 1,000 influenza positive samples, and all samples were collected during a single influenza season, possibly limiting generalizability [32,33]. In addition to incorporating multiple influenza seasons that differed in timing, intensity, and predominant types/subtypes, this study introduced additional variability by including dozens of clinicians who had received the level of training for respiratory specimen collection typical in primary care settings. Accordingly, the results from this pragmatic assessment are more likely to represent performance of Sofia-FIA in real-life settings.

Many studies have noted the effect of age on RIDT sensitivity [6,12–19], but few have considered other clinical factors that could contribute to RIDT performance. More commonly, studies examine laboratory factors such as Ct value and virus type and subtype, which are not generally available during a clinical session. Our study assessed simultaneously eight potential clinical factors (ILI status, sex, age, illness severity, common respiratory symptoms, seasonality, influenza vaccination status, and days from illness onset) and three laboratory factors (Ct value, influenza type, and detection of another virus).

The effect of Ct value, sex, age, and time from illness onset on RIDT performance are consistent with previous findings within the literature [17,34]. Seasonality is not well defined, but studies indicate that RIDTs are most useful when community prevalence of influenza is high because positive predictive value is greatest at that time [10]. We are unaware of any studies that assess the effect of vaccination status and individual symptoms.

Few studies have examined the relationship between sensitivity and the presence/absence of an ILI. During a performance assessment of QuickVue, Koul et al. reported no difference in sensitivity for patients with an ILI or a severe acute respiratory illness (SARI) [16]. ILI is well defined in the literature, but how severity is measured for sensitivity analysis varies greatly. SARI was defined as those who have ILI (fever accompanied by cough and/or sore throat) and are hospitalized. Another study that used hospitalization as a marker for severity found that sensitivity was especially poor among hospitalized adults (45%) compared with outpatient adults (75%), with a similar mean time from illness onset (2.7 days and 2.1 days, respectively) [35]. Hospitalized children had a higher sensitivity (84%), which was likely due to the higher viral loads commonly found in younger populations. The higher sensitivity among children may be due to a higher viral load in younger populations. In our study, we did not implement an age cutoff, and sensitivity was greater for those who had an ILI, but it was not influenced by severity. Our definition of severity was based on a clinician-reported three-point scale and may be subject to bias. A more uniform definition of severity may be needed.

As is commonly found in the scientific literature, Sofia-FIA did not perform as well as expected in real-world clinical settings. The Sofia package insert cites nasal swab sensitivity and specificity as 90% and 95% for influenza A and 89% and 96% for influenza B, respectively [24]. In contrast, we found an overall sensitivity for influenza A of 66.2% (95% confidence interval: 63.0–69.3), with a specificity of 97.9% (97.4–98.3). For influenza B, sensitivity was also 66.2% (60.6–71.4) while the specificity was 97.4% (96.8–97.8). Most studies have indicated lower sensitivity for influenza B, but our study showed 66.2% sensitivity for both influenza A and B [19,22,36]. Furthermore, our study found that detecting another virus could decrease sensitivity. Although research in this area is limited, a study comparing four RIDTs saw no cross reactivity in samples with co-infections [37].

This analysis had several limitations. First, we only analyzed the performance of one RIDT, but many of our findings are compatible to those of studies of other RIDTs. Second, our surveillance program encompasses a limited geographical area in Southcentral Wisconsin, and we defined seasonality based on the temperate climate. The distribution of influenza types and subtypes, however, are similar to distributions reported nationally in any given year. For the purpose of generalizability, we defined seasonality as early (July-November), peak (December-February), and late (March-June) for each year analyzed, but influenza outbreaks occur at varying times of the year, and this may have had an effect on the analysis. Third, by utilizing multiple clinicians to identify potential patients, collect clinical data, and obtain respiratory specimens, much more variability is introduced than would occur in other research settings. This study provides results obtained in real-world primary care settings in which patient selection, specimen collection, and testing occurred within routine clinical activities. Although this is an asset in determining whether RIDTs are appropriate for point-of-care diagnosis and treatment, we cannot guarantee rigor in all data elements to the same degree as a more traditionally controlled research study or randomized clinical trial. Moreover, variability in patient selection and specimen collection may contribute to lower estimates of sensitivity. Finally, despite the large sample size, there may not have been sufficient power to identify factors that may affect specificity. Their effects, however, would be trivial given the overall high specificity.

Several clinical and laboratory factors in this study appear to affect RIDT sensitivity. Awareness of these factors and their identification and consideration by clinicians at the point-of-care may aid in patient selection and the appropriate interpretation of negative influenza RIDT results. If a clinician suspects that a patient has influenza, but an RIDT test is negative, the clinician may be able to take age, symptoms, and days from illness onset into consideration when assessing whether or not the results are accurate.

Due to the ongoing SARS-CoV-2 pandemic, clinicians are relying more on diagnostic tests than symptom assessment, and nucleic acid amplification testing (NAAT) has become more widely available in multiple areas of the world. NAAT tests are sensitive and specific, but they can be costly and it can take hours to days to receive results. RIDTs, in contrast, are relatively inexpensive and produce results in the time that it takes a clinician to assess and treat patients. Thus, an RIDT can be used as an initial tool during point-of-care with the caveat that some factors my influence sensitivity and NAAT confirmation may be necessary.

Additional performance characteristic studies analyzing potential factors affecting sensitivity are necessary to explain broad ranges sensitivity across RIDT platforms and to establish standard guidelines for clinical interpretation of influenza RIDT results.

Acknowledgments

We would like to acknowledge Dr. Mindy Smith for her role in providing critical feedback on this manuscript.