Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

The Association between Serum Biomarkers and Disease Outcome in Influenza A(H1N1)pdm09 Virus Infection: Results of Two International Observational Cohort Studies

  • Richard T. Davey Jr ,

    rdavey@niaid.nih.gov

    Affiliation National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Maryland, United States of America

  • Ruth Lynfield,

    Affiliation Infectious Disease Division, Minnesota Department of Health, St. Paul, Minnesota, United States of America

  • Dominic E. Dwyer,

    Affiliation Department of Virology, Centre for Infectious Diseases and Microbiology, Westmead Hospital and University of Sydney, Westmead, New South Wales, Australia

  • Marcello H. Losso,

    Affiliation HIV Unit, Department of Medicine, Hospital José María Ramos Mejía, Buenos Aires, Argentina

  • Alessandro Cozzi-Lepri,

    Affiliation Research Department of Infection and Population Health, University College London, London, England, United Kingdom

  • Deborah Wentworth,

    Affiliation Division of Biostatistics, University of Minnesota, Minneapolis, Minnesota, United States of America

  • H. Clifford Lane,

    Affiliation National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Maryland, United States of America

  • Robin Dewar,

    Affiliation Virus Isolation and Serology Laboratory, SAIC-Frederick, Inc., Frederick National Laboratory, Frederick, Maryland, United States of America

  • Adam Rupert,

    Affiliation AIDS Monitoring Laboratory, SAIC-Frederick, Inc., Frederick, Maryland, United States of America

  • Julia A. Metcalf,

    Affiliation National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Maryland, United States of America

  • Sarah L. Pett,

    Affiliation Kirby Institute, Faculty of Medicine, University of New South Wales, Sydney, New South Wales, Australia

  • Timothy M. Uyeki,

    Affiliation Influenza Division, Centers for Disease Control and Prevention, Atlanta, Georgia, United States of America

  • Jose Maria Bruguera,

    Affiliation Servicio de Inmunocomprometidos, Hospital José María Ramos Mejía, Buenos Aires, Argentina

  • Brian Angus,

    Affiliation Nuffield Department of Medicine, University of Oxford, Oxford, England, United Kingdom

  • Nathan Cummins,

    Affiliation Division of Infectious Diseases, Mayo Clinic, Rochester, Minnesota, United States of America

  • Jens Lundgren,

    Affiliation Department of Infectious Diseases, Copenhagen University Hospital/Rigshospitalet & University of Copenhagen, Copenhagen, Denmark

  •  [ ... ],
  • James D. Neaton, on behalf of the INSIGHT FLU 002 & 003 Study Groups

    The Writing Group assumes responsibility for the manuscript. Membership of the INSIGHT FLU 002 & 003 Study Groups is included in the Acknowledgements.

    Affiliation Division of Biostatistics, University of Minnesota, Minneapolis, Minnesota, United States of America

  • [ view all ]
  • [ view less ]

Abstract

Background

Prospective studies establishing the temporal relationship between the degree of inflammation and human influenza disease progression are scarce. To assess predictors of disease progression among patients with influenza A(H1N1)pdm09 infection, 25 inflammatory biomarkers measured at enrollment were analyzed in two international observational cohort studies.

Methods

Among patients with RT-PCR-confirmed influenza A(H1N1)pdm09 virus infection, odds ratios (ORs) estimated by logistic regression were used to summarize the associations of biomarkers measured at enrollment with worsened disease outcome or death after 14 days of follow-up for those seeking outpatient care (FLU 002) or after 60 days for those hospitalized with influenza complications (FLU 003). Biomarkers that were significantly associated with progression in both studies (p<0.05) or only in one (p<0.002 after Bonferroni correction) were identified.

Results

In FLU 002 28/528 (5.3%) outpatients had influenza A(H1N1)pdm09 virus infection that progressed to a study endpoint of complications, hospitalization or death, whereas in FLU 003 28/170 (16.5%) inpatients enrolled from the general ward and 21/39 (53.8%) inpatients enrolled directly from the ICU experienced disease progression. Higher levels of 12 of the 25 markers were significantly associated with subsequent disease progression. Of these, 7 markers (IL-6, CD163, IL-10, LBP, IL-2, MCP-1, and IP-10), all with ORs for the 3rd versus 1st tertile of 2.5 or greater, were significant (p<0.05) in both outpatients and inpatients. In contrast, five markers (sICAM-1, IL-8, TNF-α, D-dimer, and sVCAM-1), all with ORs for the 3rd versus 1st tertile greater than 3.2, were significantly (p≤.002) associated with disease progression among hospitalized patients only.

Conclusions

In patients presenting with varying severities of influenza A(H1N1)pdm09 virus infection, a baseline elevation in several biomarkers associated with inflammation, coagulation, or immune function strongly predicted a higher risk of disease progression. It is conceivable that interventions designed to abrogate these baseline elevations might affect disease outcome.

Introduction

The sudden and unexpected emergence in 2009 and subsequent rapid global spread of a novel influenza virus, A(H1N1)pdm09, was yet another reminder of the ongoing challenges posed by this rapidly evolving class of respiratory viruses to world health [1], [2]. Both its seeming defiance early on of well-established patterns of seasonality, initial alarming reports of its heightened virulence in segments of the population (e.g. otherwise healthy non-elderly adults) not necessarily conforming to conventional risk groups, atypical clinical manifestations, and rapid emergence of A(H1N1)pdm09 virus as the major influenza virus causing human disease worldwide further augmented concerns about the potential threat of this novel virus to previous gains in prevention and management of this respiratory infection [3]. These concerns galvanized a global effort both to study and control A(H1N1)pdm09 virus infection through new and heightened surveillance, improved international communication, as well as the rapid development, testing, and deployment of vaccine strategies effective against the novel virus [4]-[6]. Fortunately, whether as a direct major consequence of these efforts alone or perhaps more as a combination of these efforts with still poorly understood elements of the biology of the virus itself, influenza A(H1N1)pdm09 virus has remained a major circulating human influenza virus but failed to reach the heights of morbidity and mortality previously feared. While largely still retaining its sensitivity to neuraminidase inhibitors, along the way it also assumed a more typical seasonal pattern of incidence and currently is considered a seasonal subtype of H1N1 [7]. Although recently supplanted as the predominant circulating virus in many parts of the Northern and Southern Hemispheres by influenza A (H3N2) as well as influenza B, A(H1N1)pdm09 remained detectable throughout 2011 influenza and 2012.

There has been a long-standing effort to identify and better characterize possible predictors of the severity of influenza virus infection in the human host [8]. In particular, it would be highly desirable for the treating clinician to have at hand a simple set of prognostic indicators, ideally both those that are readily obtainable from the patient and easily performed in the laboratory, that would help to guide therapy, as well as predict which subset of patients are potentially at greater risk for developing more serious sequelae from influenza infection following their initial diagnosis [9]. This is especially true if, as with A(H1N1)pdm09 virus infection, patient characteristics and the course of infection do not necessarily follow the typical clinical seasonal pattern for influenza. Furthermore, such prognostic markers may provide greater insight into which aspects of host defense are activated by the infection, and whether this activation is beneficial or harmful to the host. Ultimately, such insights may lead to better adjunctive therapy.

Both with A(H1N1)pdm09 virus infection and with disease due to other seasonal influenza subtypes, considerable interest has been generated in trying to characterize the potential role that cytokine dysregulation (so-called “cytokine storm”) triggered in the host may play in the pathogenesis and outcome of the viral infection [10]. This phenomenon includes the generation of markedly elevated levels of various cytokines and other pro-inflammatory mediators early in the course of natural infection that influence both the height and duration of the innate and adaptive immune response [11]. While overall the production of these mediators is felt to be protective, in some cases it is possible that an exaggerated or prolonged inflammatory response may actually contribute to a worsened disease outcome.

The INSIGHT H1N1v Outpatient study (FLU 002) and the INSIGHT H1N1v Hospitalization study (FLU 003) are two international observational cohort studies of influenza launched in 2009 whose purpose is to describe adult participants in geographically diverse locations who present for medical treatment due to influenza-like illness (ILI) and are documented to have laboratory-confirmed influenza and their outcomes over 14 days (FLU 002) and 60 days (FLU 003) of follow-up [12].

In this report we present the results of a comprehensive panel of serum biomarker determinations performed on blood specimens obtained at study entry in patients from these two cohorts with confirmed A(H1N1)pdm09 virus infection. Our goal was to study the association of the biomarkers with the risk of developing worsened disease outcomes as defined a priori in the two studies.

Methods

Ethics Statement

The FLU 002 and FLU 003 protocols were approved by the institutional review boards (IRB) or institutional ethics committees (IEC) at the University of Minnesota and at each of the other 63 participating clinical sites worldwide. Formal written documentation of IRB/IEC approval was required of each site Principal Investigator during the registration process that preceded site activation as a study center. Copies of these approval letters are filed with the central coordinating center at the University of Minnesota. All patients gave signed informed consent prior to enrollment.

Study Design

The two international studies, FLU 002 and FLU 003, were initiated by the National Institutes of Health in August 2009 under the auspices of the INSIGHT (International Network for Strategic Initiatives in Global HIV Trials) clinical trials network. The INSIGHT network conducts these ongoing studies through a central coordinating center at the University of Minnesota and four international coordinating centers located in the United States (Washington DC), Europe (London and Copenhagen), and Australasia (Sydney). The study design and statistical considerations underlying each study were described previously [12]. Briefly, the two studies initially focused on the global manifestations of A(H1N1)pdm09 infection and together were designed to cover a broad clinical spectrum ranging from outpatients presenting with mild symptomatology (FLU 002) to those with more serious disease requiring hospitalization due to complications of influenza (FLU 003). The purpose of these studies is to estimate the percent of adult patients presenting with influenza-like illness (ILI) that is due to laboratory-confirmed influenza and to identify clinical and virologic factors associated with the risk of a worsened disease outcome or death. Each study takes advantage of an established international network of community and hospital-based investigators already in place and supplemented by additional sites seeing large numbers of adults diagnosed with acute influenza.

Data Collection in FLU 002 and 003

At the time of enrollment the following information was collected: patient demographics, height, weight and vital signs; date of ILI onset, earliest contact with the health system for current illness; use of neuraminidase inhibitors to prevent or treat influenza in the preceding 14 days; medical history and underlying medical conditions, pregnancy status, smoking history, and current medications; influenza vaccination history since 2008 and pneumococcal vaccination history; and antiviral, antibacterial and other treatments prescribed at enrollment. Also recorded were local laboratory test results for influenza A; chest radiograph findings; and other local laboratory tests performed as part of standard of care.

In both studies respiratory (nasal and oropharyngeal) swabs were collected at enrollment and sent to one of two central laboratories to confirm by RT-PCR the local influenza diagnosis and to determine subtype. Identical methods were used by each laboratory. Serum samples were also collected from each participant at enrollment.

Patients enrolled in the outpatient study (FLU 002) with ILI were followed for 14 days for progression to hospitalization, the development of complications, or death. For the inpatient study (FLU 003) patients could be enrolled while in the general ward or in the intensive care unit (ICU). In either case patients were followed for 60 days. For those enrolled in the general ward, outcomes assessed included death, requirement for the ICU and/or mechanical ventilation, or prolonged hospitalization; the latter was defined as an inpatient stay exceeding 28 days of the 60-day follow-up period, not necessarily consecutively. For participants enrolled after already having been admitted to the ICU, death or prolonged hospitalization for >28 days were the primary outcomes.

Biomarker Study

Patients were included in this study if they had influenza A(H1N1)pdm09 virus infection confirmed by RT-PCR at a central laboratory and had completed ascertainment of outcome status at day 14 and day 60 in the two studies, respectively. Serum samples obtained at enrollment were analyzed for all such patients in the inpatient study. For efficiency a nested case-control sampling scheme was used in the outpatient study, where samples were analyzed for each patient who experienced an event (28 patients) within the 14 day follow-up period and for three controls (patients who survived the 14 day follow-up period without complications or hospitalization and who were declared to be symptoms-free at day 14) matched on country of enrollment, age (± 5 years) and duration of symptoms. For two cases only 1 suitable control and for one case 2 controls were found (thus, a total of 79 controls instead of 84).

The blood samples were centrifuged and stored at -80 degrees Celsius until analysis. Sera concentrations of multiple cytokines and chemokines were obtained using a Pro-inflammatory 9-plex (IL-1 beta, IL-2, IL-6, IL-8, IL10, IL-12p70, GM-CSF, IFN-gamma, and TNF-alpha), a Chemokine 9-plex (Eotaxin, MIP-1beta, Eotaxin-3, TARC, IP-10, IL-8, MCP-1, MDC, and MCP-4), a Vascular Injury II Panel (CRP, VCAM-1, ICAM-1, and SAA), and LBP (Meso Scale Diagnostics, Gaithersburg, MD). Additional ELISA results were assayed for CRP, sCD14, sCD163, IL-6 (R&D Systems Inc., MN, USA). D-Dimer measurements were made using the VIDAS assay system (BioMerieux, Durham, NC). For biomarkers determined by both multiplex and ELISA, ELISA test results are cited. IL-8 was measured using both a chemokine and cytokine platform, only the latter was cited. Altogether, we report the results for 25 biomarkers. The sera samples for these studies were analyzed blinded to event status. Samples from outpatients and hospitalized patients were analyzed at the same time by the central laboratory at SAIC-Frederick.

While acknowledging that there is considerable overlap in cellular origin, cascading immunological relationships, and downstream effects among many of these biomarkers, for ease of comparison the 25 markers are displayed by broadly grouping them into four functional categories: 1) macrophage pro-inflammatory activation response; 2) acute phase response; 3) T cell activation response; and 4) macrophage chemokine response.

Statistical analysis

The association of each of the 25 biomarkers with disease severity/progression was assessed in three separate analyses: 1) for disease severity, a cross-sectional comparison of biomarker levels for participants in FLU 002 versus FLU 003, and for those in FLU 003 enrolled in the general ward versus the ICU; 2) a comparison of biomarker levels for participants with disease progression after 14 days of follow-up versus those without progression in FLU 002 (case-control analysis); and 3) a comparison of biomarker levels for participants with disease progression after 60 days of follow-up versus those without progression in FLU 003 (cohort analysis). While power is greater for the cross-sectional comparisons than the follow-up comparisons within each study, a disadvantage of these comparisons is that the temporal association between the biomarker level and disease severity is not known (i.e., whether the biomarker is elevated as a consequence of disease severity or predicts disease progression is uncertain). This problem is overcome, at least in part, with the follow-up comparisons since in both FLU 002 and FLU 003 biomarker levels were determined prior to the disease progression outcome.

Simple summary statistics were use to describe the characteristics of patients in the two biomarker studies. To reduce the impact of outlying levels and to account for the positively skewed distribution of the biomarkers, results are categorized in tertiles and log10 transformed. The tertiles are defined separately for FLU 002 and FLU 003 [Table S1 in Supporting Information].

For the cross-sectional comparison, analysis of covariance with covariates corresponding to age and geographic location was used to compare the biomarker levels for patients in the two studies at the time of enrollment. Unlike the follow-up analyses within each study that are described below, we did not adjust for duration of symptoms as we considered it to be on the causal pathway for this cross-sectional comparison. For these comparisons, back-transformed levels (geometric means) are cited for each study. In FLU 003, further comparisons are made for the subgroups defined by location of enrollment (general ward versus ICU). For FLU 002, the log10 transformed biomarker levels for the controls and events were weighted to account for the larger FLU 002 group of patients from which the controls were selected for biomarker analyses.

For the follow-up comparisons in FLU 002 and FLU 003, logistic regression was used to summarize the association of each biomarker with the disease progression outcomes. For FLU 002 conditional logistic regression analyses for matched case–control studies were used. For FLU 003, unconditional logistic regression analyses stratified by location of enrollment (general ward versus ICU) and with covariates corresponding to the matching factors used in FLU 002 (age, duration of symptoms and geographic location) were carried out. Odds ratios (ORs) for the upper tertile versus the lowest tertile are cited along with 95% confidence intervals (CIs) and p-values. Analyses that used the log10 transformed biomarkers were also carried out; in the summary tables we refer to the p-values from this analysis as the p-value for linear trend. Analyses that further adjusted for race (black vs non-black), gender, current smoking, BMI and history of chronic conditions were also carried out and gave similar results. Kaplan-Meier survival curves are used to compare the patterns of mortality for tertiles of IL-6 in the FLU 003 study.

Two approaches were taken to minimize the risk of identifying false-positive associations between biomarkers and disease progression by highlighting subsets of individual biomarkers: 1) biomarkers that are significantly (p<0.05) related to disease progression based on the linear trend p-value in both studies; and 2) biomarkers which are significant (p≤ 0.002) in one study but not the other. The latter level of significance corresponds to a Bonferroni adjusted p-value (0.05/25).

To determine the relationship of multiple biomarkers with disease progression, we took advantage of the functional groupings that were identified. A global test procedure proposed by O’Brien for multiple endpoints is used [13]. With this approach, each marker in the raw scale within a functional grouping is ranked from lowest to highest, the ranks of the individual markers are summed for each patient. We refer to the sum of the ranks as the “biomarker score”. This biomarker score is then compared for patients who experienced an event versus those who did not with logistic regression models as described above (i.e., ORs for upper versus lowest tertile of the biomarker score are cited). Advantages of this procedure are simplicity and increased power if the biomarkers within a category all trend in the same direction. A disadvantage is that while the global test identifies biomarker groupings that are significant, it does not provide information on which markers are driving the statistical significance. Also, it is not an optimal test procedure if the association of some markers with disease outcome is positive and that of some other markers is negative. Thus, we also carried out a likelihood ratio test that tested the significance of adding all the markers (in the log10 scale) in a functional category to a base model that only included age, duration of symptoms and geographic region.

Statistical analyses were performed using SAS (Version 9.2).

Results

At the time of this biomarker investigation, 737 patients with influenza A(H1N1)pdm09 virus infection centrally confirmed by RT-PCR had completed ascertainment of outcome status in FLU 002 (528 patients) or FLU 003 (209 patients). Patients were enrolled from 63 sites in 14 countries. The risk of serious outcomes and the baseline characteristics varied considerably for participants in FLU 002 and FLU 003. Of the 528 patients with ILI seen at one of the outpatient clinics, 28 (5.3%) had disease progression requiring hospitalization (26 patients), one developed a severe complication (chronic obstructive pulmonary disease exacerbation), and one died within 14 days of enrollment (Figure 1A). These 528 patients were enrolled during two influenza seasons beginning in September 2009 in the Northern (n = 505) and Southern (n = 23) Hemispheres.

thumbnail
Figure 1. Biomarker study designs.

The respective study designs and distribution of patients with and without severe disease outcomes in the outpatient case-control study FLU 002 (A) and in the hospitalization cohort study FLU 003 (B).

https://doi.org/10.1371/journal.pone.0057121.g001

Of the 209 hospitalized patients enrolled in FLU 003, 170 had been enrolled while in the general ward and 39 from the ICU. These 209 patients were also enrolled during the same two influenza seasons in the Northern (n = 203) and Southern Hemispheres (n = 6). Overall, during the 60-day follow-up period, 49 (23.4%) of the 209 patients died, entered the ICU (if enrolled in the general ward), or had a hospital stay > 28 days. Figure 1B depicts these outcomes depending on whether patients were enrolled from the general hospital ward or from the ICU.

Table 1 summarizes the characteristics at study entry for participants enrolled in either FLU 002 or FLU 003 overall and, for FLU 003 participants, by location of enrollment (general ward or ICU). As expected, the median patient age of 30 years for the outpatient cohort was substantially lower than that of the hospitalized cohort, as was the duration of symptoms, the percentages of patients with an elevated BMI, who were pregnant, who had chronic medical co-morbidities, or who had received antiviral medication in the 14 day period prior to enrollment.

thumbnail
Table 1. Baseline Characteristics of the FLU 002 and FLU 003 Cohorts: Patients with Central Laboratory Confirmed A(H1N1)pdm09 Infection.

https://doi.org/10.1371/journal.pone.0057121.t001

Cross-Sectional Relationship of Individual Biomarkers with Disease Severity

Geometric means for each of the 25 biomarkers were compared for outpatients in FLU 002 and inpatients in FLU 003 (overall and according to location of enrollment) [Table 2]. There was a strong correlation between levels of the individual biomarkers and the likely severity of illness of each cohort at time of enrollment: that is, most biomarkers were significantly lower in FLU 002 patients compared to FLU 003 patients and, among FLU 003 patients, biomarkers were generally lower in those enrolled while in the general ward as compared to the ICU. Three exceptions were IFN-γ, MDC and TARC for which the means were significantly higher for patients in FLU 002 compared to FLU 003 and for which levels were higher for patients in FLU 003 who were enrolled in the general hospital ward compared to the ICU. The mean level of IP-10 was also significantly higher in patients in FLU 002 compared to the overall mean level in FLU 003 patients, although higher in the ICU subcohort of the latter population.

thumbnail
Table 2. Geometric Means of Biomarkers for Patients in FLU 002 and FLU 003.

https://doi.org/10.1371/journal.pone.0057121.t002

Relationship of Individual Biomarkers with Disease Progression during Follow-up

Table 3 compares characteristics for patients in each study who had disease progression with those who did not. In FLU 002 there was a trend for more cases than controls to be of black race (p = 0.06). However, none of the differences between cases and controls reached statistical significance. In FLU 003, patients with subsequent disease progression were more likely to have a longer duration of clinical symptoms prior to enrollment (p = 0.02) and to use antiviral medications in the 14 day period prior to enrollment (p = 0.03). As expected, the most striking difference between those who progressed and those who did not was the difference in risk for those enrolled in the general ward versus those enrolled from the ICU (p<0.001).

thumbnail
Table 3. Baseline Characteristics of Patients in the FLU 002 and FLU 003 Biomarker Cohorts According to Event Status.

https://doi.org/10.1371/journal.pone.0057121.t003

Tables 4 and 5 give median values at entry for those who progressed and those who did not in FLU 002 and FLU 003, respectively. Also cited are ORs for the upper versus lowest tertile of each marker, the p-value corresponding to this comparison, and the p-value for linear trend (from a model that includes the log-transformed level of the biomarker). Results were similar after further controlling for race (black vs non-black), gender, current smoking, BMI, and history of chronic conditions (data not shown).

thumbnail
Table 4. FLU 002 Patients: Odds Ratios of Progression for the Highest Versus Lowest Tertile of Each Biomarker.

https://doi.org/10.1371/journal.pone.0057121.t004

thumbnail
Table 5. FLU 003 Patients: Odds Ratios of Progression for the Highest Versus Lowest Tertile of Each Biomarker.

https://doi.org/10.1371/journal.pone.0057121.t005

Higher levels of 12 markers were significantly associated with disease progression. Seven markers were significant (p< 0.05) in both studies: IL-6, CD163, IL-10, LBP, IL-2, MCP-1, and IP-10. For ease of comparison, Figure 2A gives the ORs (3rd/1sttertile) and p-values for these 7 biomarkers for each study. All of the ORs in Figure 2A are > 2.4. Five other biomarkers were significant (p = 0.002 or smaller) in FLU 003 and not significant (p> 0.05) in FLU 002 (Figure 2B). Of note, for FLU 002 none of the 12 biomarkers were significant at a Bonferroni-adjusted level of significance of 0.002 or lower with the exception of IL-6 (OR = 4.4).

thumbnail
Figure 2. Biomarkers significantly related to disease progression.

The biomarkers found to be significantly associated with disease progression in both studies (A) and in FLU 003 only (B) are shown. Odds ratios (3rd/1st tertile) and 95% confidence intervals are depicted.

https://doi.org/10.1371/journal.pone.0057121.g002

For inpatients (FLU 003), we also examined the relationship of IL-6 with mortality (Figure 3). Only one death occurred among patients in the lowest tertile (<6.4 pg/mL) at enrollment; in contrast, for the 2nd and 3rdtertiles (6.4–17.3 and >17.3 pg/mL), 7 and 12 deaths occurred (p = 0.008 for difference in mortality among the tertiles).

thumbnail
Figure 3. Mortality by tertiles of IL-6 in the FLU 003 study.

A Kaplan-Meier graph of cumulative mortality (%) for FLU 003 participants according to baseline levels of IL-6 as grouped into tertiles.

https://doi.org/10.1371/journal.pone.0057121.g003

Relationship of Functional Categories of Biomarkers with Disease Progression

Figure 4 summarizes ORs (3rd/1sttertile) for the biomarker score for each functional grouping within each study (see Table 2 for biomarkers included in these four groupings). For both studies the first three functional categories showed stronger relationships (ORs were 2.7 or greater) with disease progression than did the last (macrophage chemokine response). In part, the weaker association of this last category with disease progression was due to some biomarkers in that grouping having a positive association with disease progression and others displaying an inverse association. To investigate this further, we carried out a logistic regression analysis for each study in which we compared the chi-square statistic resulting from adding the last category of biomarkers to a model that only included age, duration of symptoms and geographic region. For FLU 003, the last category of markers (macrophage chemokine response) contributed significantly (p<0.001) to model fit (data not shown) as did the first three categories, macrophage proinflammatory activation response (p<0.001), acute phase response (p<0.001), and T cell activation response (p = 0.03). For FLU 002, macrophage proinflammatory activation response (p = .003) and macrophage chemokine response (p = .02) were significant determinants of disease progression. The other two categories were of borderline significance (p-values  = 0.06 and 0.07).

thumbnail
Figure 4. Relationship of functional categories of biomarkers to disease progression in FLU 002 and FLU 003.

The odds ratios and 95% confidence intervals for the risk of disease progression in FLU 002 (in blue) and FLU 003 (in red) are depicted according to four functional categorizations of the 25 biomarkers analyzed in these two studies.

https://doi.org/10.1371/journal.pone.0057121.g004

Discussion

In vitro laboratory experiments, small animal model testing, and clinical studies of humans with acute influenza, including those infected with A(H1N1)pdm09 virus, have revealed a number of cytokines and other mediators that appear to play a significant role in the initial host response to influenza virus infection. In mouse or ferret studies experimental infections with virulent strains of influenza A viruses have been associated with higher elevations in select biomarkers such as IL-6, TNF-α, and IFN-α and have usually been associated with worsened disease outcomes in those animals [14], [15]. In an influenza challenge study reported in 1998 in which normal human volunteers were experimentally infected with a seasonal influenza A (H1N1 Texas/36/91) virus, Hayden et al. found that both IL-6 and IFN-alpha levels in nasal lavage fluids peaked early in the course of infection and correlated directly with viral titers, temperature, mucus production, and symptom scores [8]. Several other clinical studies in adults or children have been reported by investigators in which the addition of various serum or nasal biomarker level measurements to the standard clinical evaluation appeared to add to the diagnostic certainty of respiratory virus infection with seasonal influenza [16][19].

In a number of generally cross-sectional studies focusing either partially or exclusively upon confirmed cases of A(H1N1)pdm09 virus infection in specific geographic areas, potential correlations between various cytokine levels and disease severity have been reported. Investigators from Mexico reported that A(H1N1)pdm09 virus infection resulted in stronger in vitro upregulation of IL-6, CCL3, and CXCL8 in 72-hour cell cultures as well as elevated serum levels of IL-6, CXCL8, and certain other cytokines in individuals infected with this subtype compared to those with seasonal influenza virus infection [20]. In separate publications a group in Hong Kong reported that elevated levels of IL-6, CXCL8, CCL2, and sTNFR-1 correlated with severe cases of A(H1N1)pdm09 virus infection overall and, in particular, with the extent and severity of influenza-associated pneumonia [21], [22]. Similar findings concerning elevated IP-10 and IL-6 levels in cases of pediatric pneumonia were reported from Korea [23]. At least two groups from mainland China have described similar relationships for several of the pro-inflammatory cytokines withA(H1N1)pdm09 virus infection, and separate groups from Spain, Italy, and Romania have also described correlations between elevated levels of various cytokines such as IL-6, IL-15, and TNF-α and the severity of disease outcomes in patients with confirmed A(H1N1)pdm09 virus infection [24][28]. Most recently, a group from Canada has found IL-6 to be an important feature of the host response in both humans and mice infected with A(H1N1)pdm09 virus and, in the former case, found elevated IL-6 levels to be an important predictor of severe disease [29].

Biomarker analyses from our two large ongoing international studies of influenza described here have strengthened and extended these observations in important ways. A major virtue of the present studies is that these data were collected prospectively according to a common data-set and with defined periods of follow-up to assess disease progression, samples have been garnered from a relatively large number of patients living in geographically disparate regions of the world and analyzed through common central laboratory systems using standardized methodologies, and both studies included enrollments of patients with different severities of A(H1N1)pdm09 virus infection spanning over more than a single influenza season in those areas. Further, the samples were collected, shipped, and processed for analysis using identical procedures to minimize any potential artifactual effects on serum levels of the biomarkers measured.

Even after adjustments for multiple potentially confounding variables within each dataset according to different stringencies, there are several biomarkers whose baseline levels appear to be highly correlated with a worsened disease course according to the definitions of disease progression predefined for each study. Several could have been singled out in this context. However, the seven markers that were significantly correlated with disease progression in both studies were IL-6, CD163, IL-10, LBP, IL-2, MCP-1, and IP-10, a somewhat disparate set spanning all four functional groupings. While cross-sectional comparisons between studies must always be interpreted with caution, for six of these seven markers, the exception being MCP-1, the absolute levels of each at study entry also appeared to correlate with disease severity at time of enrollment, as reflected principally by their higher geometric mean values in hospitalized patients versus outpatients.

Two notable inconsistences between the cross-sectional and follow-up findings were the associations with IFN-γand IP-10. Both markers were lower in the cross-sectional comparison of FLU 003 participants compared to FLU 002 participants, but within the two studies higher, not lower, levels were associated with an increased risk of disease progression during follow-up. The reason for this discrepancy remains unclear at present; however, the difference may reflect a limitation of cross-sectional comparisons in which temporal relationships are uncertain.

As a biomarker with known involvement in the pro-inflammatory cascade associated with many different types of infections, as well as one that has featured prominently in earlier other published analyses of the potential role of biomarkers in predicting influenza disease outcomes, we also chose to validate the strong predictive potential of IL-6 in these two studies. For both outpatients and those requiring hospitalization, serum IL-6 was a strong predictor of disease progression. For the hospitalized patients in FLU 003, those with an IL-6 level in the upper two tertiles were also at an increased of mortality. This is similar to prior observations a decade earlier in a small number of fatal cases of H5N1 infection [30]. A causal explanation for this is not fully elucidated, although animal data do support an association of elevated levels of IL-6 production with enhanced lethality of the infecting virus [31], [32].

Although the strong statistical associations found in these two studies between select individual biomarkers and a worsened disease outcome are compelling, nonetheless these results present an obvious difficulty with extrapolation to the clinical arena at the present time. Most of the biomarkers described here are part of a multiplex testing array generally performed in a research setting and are not a routine part of the diagnostic work-up performed for a typical patient presenting with signs and symptoms of acute influenza. Hence, at present they may be of more value in providing insight into potential mechanisms of viral pathogenesis and host defense rather than in offering direct clinical benefit. There are some potential exceptions to this. D-dimer and CRP assays, for example, are generally available today in most acute care facilities as indicators of recent thrombotic events and abnormal systemic inflammation, respectively, and the test results are generally available in real time.

It is fair to say that, at present, there does not appear to be a single discrete biomarker readily available to the physician at the time of presentation that one can conclude adds unequivocably to the ability of the standard diagnostic assessment to predict the likelihood of disease progression in all patients. Nonetheless, as these multiplex assays become cheaper and more readily available beyond the research setting, this situation may improve. For example, assuming an exaggerated or dysfunctional cytokine response to acute influenza may actually contribute to disease severity in some cases, their additional value may be in pinpointing areas of this response that may be amenable to dampening or other forms of therapeutic intervention. Thus, at the same time that one is treating the virus in at-risk individuals, it is conceivable that adjunctive therapy directed at abrogating or redirecting an overly exuberant host response could also be introduced to further improve prognosis.

Although this leap from the research setting to the bedside still remains a challenge for the reasons cited, it is still reasonable to ask whether in the future even stronger prognostic utility or power might be found in analyzing select combinations or subsets of these markers, whether identified through statistical modeling or de novo as biologically plausible groupings. In addition to allowing for more intelligent utilization of available biomarkers, such an approach might further minimize justifiable concerns over potential over-interpretation of statistical associations seen in comparatively large batteries of single assay results not adjusted for multiple comparisons, such as those generated by the multiplex platforms in increasingly common use. In a limited look at this possibility, we found that combinations of multiple biomarkers from even four relatively indiscrete functional groupings did generate odds ratios for disease outcomes that were statistically robust, generalizable across both studies, and comparable to the most potent of the single biomarkers studied. The fact that elevated levels of certain biomarkers were found in each of the four groupings suggests a very broad and diverse immunologic response to acute influenza. But with further dissection of specific categories of markers it may be possible to delineate more precisely what role the disparate portions of the host response play in modulating the course of infection. Whether with further exposition such a combination approach might prove to be either more powerful or, ideally, disease-specific than any of the single biomarkers highlighted remains yet to be determined. In any case, these preliminary results are intriguing and provide a rationale for performing more detailed analyses of these particular biomarkers as additional patients are accrued in these studies and additional progression events recorded. It is certainly possible, for example, that combinations of other biomarkers, or even different logical groupings of the same biomarkers into other categories based upon such factors as common cellular origin or relative positioning within the pro-inflammatory cascade, might offer results of equal or superior prognostic value or provide even greater insight into potentially deleterious aspects of the host immune response.

The focus of this present analysis has been on the prognostic value of baseline biomarkers in predicting the subsequent course of disease specifically in A(H1N1)pdm09 virus-infected patients. Since both studies are ongoing on a multi-year basis in both the Northern and Southern Hemispheres, have been broadened to include patients presenting with all major subtypes of seasonal virus in circulation at the time, and now collect serial blood specimens for up to 60 days following enrollment, the intention is that these analyses can be expanded in at least two ways. One goal will be to map the kinetics in serum and plasma of each of the major biomarkers, singly and in groupings, from baseline through the course of acute infection and until the time of resolution and recovery. Surprisingly little is known about how these markers change differentially over time according to such factors as disease severity or the host’s baseline immune status, or how quickly they return to the pre-infection state. A second goal will be to compare the prognostic value of biomarkers in infection due to A(H1N1)pdm09 virus to that found in infection with other influenza viruses. It is unclear at present whether these correlations observed with A(H1N1)pdm09 virus infection reflect broad-based host response pathways that can be applied, for example, universally to influenza A virus infections as a whole, or whether more discrete differences in various cytokine response patterns will emerge as different subtypes are examined further. These and other types of analyses should be readily possible under the bounds of these ongoing studies.

Supporting Information

Table S1.

* The upper and lower tertiles for each of the biomarkers are defined separately for the FLU 002 and FLU 003 cohorts; these biomarkers are shown grouped into the four functional categories as defined in the main manuscript.

https://doi.org/10.1371/journal.pone.0057121.s001

(DOCX)

Acknowledgments

The INSIGHT FLU 002 and FLU 003 Study Groups wish to acknowledge and thank the many patients who have participated in these two observational trials.

Contributors

INSIGHT Influenza FLU 002 & 003 Study Groups

Community Representatives: David Munroe, Claire Rappoport, Siegfried Schwarze.

Coordinating Centers:

Copenhagen: Bitten Aagaard, Dejan Adzic, Jesper Grarup, Patricia Herrero, Per Jansson, Marie Louise Jakobsen, Birgitte Jensen, Karoline B. Jensen, Heidi Juncher, Jesper Kjær, Jens Lundgren, Paco Lopez, Amanda Mocroft, Mary Pearson, Begoña Portas, Caroline Sabin, Klaus Tillmann

London: Abdel Babiker, Nafisah Braimah, Yolanda Collaco-Moraes, Fleur Hudson, Ischa Kummeling, Filippo Pacciarini, Nick Paton

Statistical and Data Management Center – Minneapolis – Alain DuChene, Michelle George, Merrie Harrison, Kathy Herman, Eric Krum, Gregg Larson, Ray Nelson, Kien Quan, Siu-Fun Quan, Cavan Reilly, Terri Schultz, Greg Thompson, Nicole Wyman

Sydney: Anchalee Avihingsanon, Lara Cassar, Kanlaya Charoentonpuban, Sean Emery, Kobkeaw Laohajinda, Thidarat Jupimai, Isabel Lanusse, Alejandra Moricz, Ines Otegui, Kiat Ruxrungtham, Rose Robson

Washington: Elizabeth Finley, Fred Gordin, Adriana Sanchez, Michael Vjecha

Specimen Repositories and Laboratories:

John Baxter, Shawn Brown (SAIC Frederick, Inc.), Elodie Ghedin (JCVI), Rebecca Halpin (JCVI), Marie Hoover (ABML)

National Institute of Allergy and Infectious Disease: Beth Baseler (SAIC Frederick Inc.), Julia A. Metcalf

Other Experts:

Centers for Disease Control: Nancy Cox, Larisa Gubareva, Kathy Hancock, Jackie Katz, Alexander Klimov, Michael Shaw

Department of Health and Human Services: Lewis Rubinson

FLU 002 Clinical Site Investigators

Argentina: Laura Barcan, Jorge Alberto Corral, Daniel Omar David, Hector Enrique Laplume, Maria Beatriz Lasala, Gustavo Daniel Lopardo, Marcelo H. Losso, Sergio Lupo, Eduardo Warley

Australia: Mark Bloch, Dominic E. Dwyer, Richard Moore, Sarah L. Pett, Norman Roth, Tuck MengSoo, Emanuel Vlahakis

Austria: Heinz Burgmann

Belgium: Nathan Clumeck, Stephan De Wit, Eric Florence, KabambaKabeya, JozefWeckx

Chile: Carlos Perez, Marcelo J. Wolff

Denmark: Jan Gerstoft, Jens D. Lundgren, Lars ∅stergaard

Estonia: Kai Zilmer

Germany: Johannes R. Bogner, Norbert H. Brockmeyer, Gerd Faetkenheuer, Hartwig Klinker, Andreas Plettenberg, Juergen Rockstroh, Christoph Stephan

Greece: Anastasia Antoniadou, Georgios Koratzanis, Nikolaos Koulouris, Vlassis Polixronopoulos, Helen Sambatakou, Nikolaos Vasilopoulos

Lithuania: SauliusCaplinskas

Peru: Alberto La Rosa, Fernando Mendo, Raul Salazar, Jorge Valencia

Poland: ElzbietaBakowska, AndrzejHorban, BrygidaKnysz

Portugal: Francisco Antunes, Manuela Doroana

South Africa: NesriPadayatchi

Spain: David Dalmau, Eduardo Fernandez-Cruz, Jose Maria Gatell, Jesus SanzSanz, Vincent Soriano

Thailand: Ploenchan Chetchotisakd, Kiat Ruxrungtham, Gompol Suwanpimolkul

United Kingdom: Clifford L.S. Leen

United States: Calvin Cohen, David L. Cohn, Jack A. DeHovitz, Wafaa El-Sadr, Marshall Glesby, Fred M. Gordin, Sally Hodder, Norman Markowitz, Richard M. Novak, Robert Schooley, Gary L. Simon, Ellen Marie Tedaldi, ZelalemTemesgen, Joseph Timpone, Daniel Z. Uslan, Barbara Heeter Wade

FLU 003 Clinical Site Investigators

Argentina: Laura Barcan, Jorge Alberto Corral, Daniel Omar David, Hector Enrique Laplume, Maria Beatriz Lasala, Gustavo Daniel Lopardo, Marcelo H. Losso, Eduardo Warley

Australia: Dominic E. Dwyer, Julian Elliott, Pam Konecny, John McBride, Sarah L. Pett

Austria: Heinz Burgmann

Belgium: Nathan Clumeck, Stephan De Wit, Philippe Jorens, KabambaKabeya

Chile: Marcelo J. Wolff

China: Tak Chiu Wu

Denmark: Jan Gerstoft, Lars Mathiesen, Henrik Nielsen, Lars Østergaard, Svend Stenvang Pedersen

Germany: Frank Bergmann, Johannes R. Bogner, Norbert H. Brockmeyer, Gerd Faetkenheuer, Hartwig Klinker, Juergen Rockstroh, Christoph Stephan

Greece: Anastasia Antoniadou, Georgios Koratzanis, Nikolaos Koulouris, Vlassis Polixronopoulos, Helen Sambatakou, Nikolaos Vasilopoulos

Norway: Anne Maagaard

Peru: Fernando Mendo, Raul Salazar

Poland: ElzbietaBakowska, AndrzejHorban

South Africa: NesriPadayatchi

Spain: David Dalmau, Vicente Estrada, Eduardo Fernandez-Cruz, Hernando Knobel Freud, Rosa M BlazquezGarrido, Jose Maria Gatell, Jose Sanz Moreno, Jose Ramon Pano-Pardo, Jesus SanzSanz, Vincent Soriano

Thailand: PloenchanChetchotisakd, KiatRuxrungtham, GompolSuwanpimolkul

United Kingdom: Brian J. Angus, David R. Chadwick, David Dockrell, Clifford L.S. Leen, Melanie Newport, Ed Wilkins

United States: Harry Anderson III, Jason V. Baker, David L. Cohn, Jack A. DeHovitz, Wafaa El-Sadr, Matthew S. Freiberg, Fred M. Gordin, Roy Gulick, David Gurka, Sally Hodder, Norman Markowitz, Richard M. Novak, Armando Paez, NamrataPatil, Annette Reboli, Michael Sands, Robert Schooley, Gary L. Simon, ZelalemTemesgen, Joseph Timpone, Daniel Z. Uslan, Barbara Heeter Wade

Author Contributions

Edited the manuscript: HCL SLP TMU JMB BA NC JL. Conceived and designed the experiments: RD. Performed the experiments: AR JAM. Analyzed the data: DW JDN AC-L. Wrote the paper: RTD RL DED MHL.

References

  1. 1. Centers for Disease Control and Prevention Outbreak of swine-origin influenza A (H1N1) virus infection – Mexico, March–April 2009. (2009) MMWR Morb Mortal Wkly Rep 58: : 467–470.
  2. 2. Novel Swine-Origin Influenza A (H1N1) Virus Investigation Team (2009) Emergence of a novel swine-origin influenza A (H1N1) virus in humans. N Engl J Med 360: 2605–2615.
  3. 3. Writing Committee of the WHO Consultation on Clinical Aspects of Pandemic (H1N1) 2009 Influenza (2010) Clinical aspects of pandemic 2009 influenza A (H1N1) virus infection. N Engl J Med. 362: 1708–19.
  4. 4. Briand S, Mounts A, Chamberland M (2011) Challenges of global surveillance during an influenza pandemic. Public Health 125: 247–56.
  5. 5. Jain S, Kamimoto L, Bramley AM, Schmitz AM, Benoit SR, et al. (2009) Hospitalized patients with 2009 H1N1 influenza in the United States, April-June 2009. N Engl J Med 361: 1935–44.
  6. 6. Greenberg ME, Lai MH, Hartel GF, Wichems CH, Gittleson C, et al. (2009) Response to a monovalent 2009 influenza A (H1N1) vaccine. N Engl J Med 361: 2405–13.
  7. 7. World Health Organization website. Available: http://www.who.int/mediacentre/news/statements/2010/h1n1_vpc_20100810/en/index.html. Accessed 10 Aug 2010.
  8. 8. Hayden FG, Fritz R, Lobo MC, Alvord W, Strober W, et al. (1998) Local and systemic cytokine responses during experimental human influenza A virus infection. Relation to symptom formation and host defense. J Clin Invest 101: 643–9.
  9. 9. Ison MG, de Jong MD, Gilligan KJ, Higgs ES, Pavia AT, et al. (2010) End points for testing influenza antiviral treatments for patients at high risk of severe and life-threatening disease. J Infect Dis 201: 1654–62.
  10. 10. Tisoncik JR, Korth MJ, Simmons CP, Farrar J, Martin TR, et al. (2012) Into the eye of the cytokine storm. Microbiol Mol Biol Rev 76: 16–32.
  11. 11. Fukuyama S, Kawaoka Y (2011) The pathogenesis of influenza virus infections: the contributions of virus and host factors. Curr Opin Immunol 23: 481–6.
  12. 12. Dwyer DE (2011) The INSIGHT Influenza Study Group (2011) Surveillance of illness associated with pandemic (H1N1) 2009 virus infection among adults using a global clinical site network approach: the INSIGHT FLU 002 and FLU 003 studies. Vaccine 29 Suppl 2B56–62.
  13. 13. O’Brien PC (1984) Procedures for Comparing Samples with Multiple Endpoints. Biometrics 40: 1097–87.
  14. 14. Siye Wang, TrongQuang Le, Naoki Kurihara, Junji Chida, Youssouf Cisse, et al. (2010) Influenza Virus—Cytokine-Protease Cycle in the Pathogenesis of Vascular Hyperpermeability in Severe Influenza. J Infect Dis 202: 991–1001.
  15. 15. Kang YM, Song BM, Lee JS, Kim HS, Seo SH (2011) Pandemic H1N1 influenza virus causes a stronger inflammatory response than seasonal H1N1 influenza virus in ferrets. Arch Virol 156: 759–67.
  16. 16. Sumino KC, Walter MJ, Mikols CL, Thompson SA, Gaudreault-Keener M, et al. (2010) Detection of respiratory viruses and the associated chemokine responses in serious acute respiratory illness. Thorax 65: 639–44.
  17. 17. Hutchinson AF, Black J, Thompson MA, Bozinovski S, Brand CA, et al. (2010) Identifying viral infections in vaccinated Chronic Obstructive Pulmonary Disease (COPD) patients using clinical features and inflammatory markers. Influenza Other Respi Viruses 4: 33–9.
  18. 18. Sato M, Hosoya M, Wright PF (2009) Differences in serum cytokine levels between influenza virus A and B infections in children. Cytokine 47: 65–8.
  19. 19. Heltzer ML, Coffin SE, Maurer K, Bagashev A, Zhang Z, et al. (2009) Immune dysregulation in severe influenza. J Leukoc Biol 85: 1036–43.
  20. 20. Zúñiga J, Torres M, Romo J, Torres D, Jiménez L, et al. (2011) Inflammatory profiles in severe pneumonia associated with the pandemic influenza A/H1N1 virus isolated in Mexico City. Autoimmunity 44: 562–70.
  21. 21. Lee N, Wong CK, Chan PK, Chan MC, Wong RY, et al. (2011) Cytokine response patterns in severe pandemic 2009 H1N1 and seasonal influenza among hospitalized adults. PLoS One 6: e26050.
  22. 22. Lee N, Chan PK, Wong CK, Wong KT, Choi KW, et al. (2011) Viral clearance and inflammatory response patterns in adults hospitalized for pandemic 2009 influenza A(H1N1) virus pneumonia. Antivir Ther 16: 237–47.
  23. 23. Kim YH, Kim JE, Hyun MC (2011) Cytokine response in pediatric patients with pandemic influenza H1N1 2009 virus infection and pneumonia: comparison with pediatric pneumonia without H1N1 2009 infection. Pediatr Pulmonol 46: 1233–9.
  24. 24. Shen HH, Hou J, Chen WW, Bai BK, Wang HB, et al. (2011) Immunologic changes during Pandemic (H1N1) 2009, China. Emerg Infect Dis 17: 1053–5.
  25. 25. Liu Y, Chen H, Sun Y, Chen F (2012) Antiviral role of Toll-like receptors and cytokines against the new 2009 H1N1 virus infection. Mol Biol Rep 39: 1163–72.
  26. 26. Estella A (2011) Cytokine levels in bronchoalveolar lavage and serum in 3 patients with 2009 Influenza A(H1N1)v severe pneumonia. J Infect Dev Ctries 5: 540–3.
  27. 27. Agrati C, Gioia C, Lalle E, Cimini E, Castilletti C, et al. (2010) Association of profoundly impaired immune competence in H1N1v-infected patients with a severe or fatal clinical course. J Infect Dis 202: 681–9.
  28. 28. Hagau N, Slavcovici A, Gonganau DN, Oltean S, Dirzu DS, et al. (2010) Clinical aspects and cytokine response in severe H1N1 influenza A virus infection. Crit Care 14: R203.
  29. 29. Paquette SG, Banner D, Zhao Z, Fang Y, Huang SS, et al. (2012) Interleukin-6 Is a Potential Biomarker for Severe Pandemic H1N1 Influenza A Infection. PLoS One 7: e38214.
  30. 30. de Jong MD, Simmons CP, Thanh TT, Hien VM, Smith GJ, et al. (2006) Fatal outcome of human influenza A (H5N1) is associated with high viral load and hypercytokinemia. Nat Med 12: 1203–7.
  31. 31. Murphy EA, Davis JM, McClellan JL, Carmichael MD, Rooijen NV, et al. (2011) Susceptibility to infection and inflammatory response following influenza virus (H1N1, A/PR/8/34) challenge: role of macrophages. J Interferon Cytokine Res 31: 501–8.
  32. 32. Wang S, Le TQ, Kurihara N, Chida J, Cisse Y, et al. (2010) Influenza virus-cytokine-protease cycle in the pathogenesis of vascular hyperpermeability in severe influenza. J Infect Dis 202: 991–1001.