https://orcid.org/0000-0003-1402-2297
Accessible Data
Abstract
Purpose
Sepsis-3 is a life-threatening organ dysfunction caused by dysregulated host responses to infection; and defined using the Sepsis-3 criteria, introduced in 2016, however, the criteria need to be validated in specific clinical fields. We investigated mortality prediction and compared the diagnostic performance of quick Sequential Organ Failure Assessment (qSOFA), systemic inflammatory response syndrome (SIRS), and burn-specific SIRS (bSIRS) in burn patients.
Methods
This single-center retrospective cohort study examined burn patients in Seoul, Korea during January 2010–December 2020. Overall, 1,391 patients with suspected infection were divided into four sepsis groups using SOFA, qSOFA, SIRS, and burn-specific SIRS.
Results
Hazard ratios (HRs) of all unadjusted models were statistically significant; however, the HR (0.726, p = 0.0080.001) in the SIRS ≥2 group is below 1. In the adjusted model, HRs of the SOFA ≥2 (2.426, <0.001), qSOFA ≥2 (7.198, p<0.001), and SIRS ≥2 (0.575, p<0.001) groups were significant. The diagnostic performance of dichotomized qSOFA, SIRS, and bSIRS for sepsis was defined by the Sepsis-3 criteria. The mean onset day was 4.13±2.97 according to Sepsis-3. The sensitivity of SIRS (0.989, 95% confidence interval [CI]: 0.982–0.994) was higher than that of qSOFA (0.841, 95% CI: 0.819–0.861) and bSIRS (0.803, 95% CI: 0.779–0.825). Specificities of qSOFA (0.929, 95% CI: 0.876–0.964) and bSIRS (0.922, 95% CI: 0.868–0.959) were higher than those of SIRS (0.461, 95% CI: 0.381–0.543).
Figures
Citation: Yoon J, Kym D, Hur J, Cho YS, Chun W, Yoon D (2023) Validation of Sepsis-3 using survival analysis and clinical evaluation of quick SOFA, SIRS, and burn-specific SIRS for sepsis in burn patients with suspected infection. PLoS ONE 18(1): e0276597. https://doi.org/10.1371/journal.pone.0276597
Editor: David M. Burmeister, Uniformed Services University of the Health Sciences, UNITED STATES
Received: May 29, 2022; Accepted: October 10, 2022; Published: January 3, 2023
Copyright: © 2023 Yoon et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data are available from the following DOI: (10.5061/dryad.dbrv15f4d).
Funding: This article was supported by a National Research Foundation of Korea (NRF) grant funded by the Korea government (MIST) (Grant No. 2021R1A2C20060331222182102840102). I state 'The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Sepsis is a systemic response to infection and is one of the most common causes of organ dysfunction and mortality in burn patients and admission to intensive care units (ICUs) among critically ill patients [1, 2]. In particular, the susceptibility for infection or sepsis increases in proportion to the total body surface area (TBSA) burned [3], and the systemic response caused by the burn itself is difficult to distinguish from the systemic inflammatory response syndrome (SIRS) developed during sepsis. Therefore, the diagnosis of sepsis remains challenging, and SIRS and conventional sepsis are not appropriate in burn patients [4, 5]. Therefore, “the Consensus Conference to Define Sepsis and Infection in Burns” was held in Tucson on January 20, 2007, and it proposed alternative definitions of SIRS (bSIRS) and sepsis (i.e., burn sepsis [BS]) [6]. In 2016, Sepsis-3, which discards SIRS and severe sepsis, was redefined using the Sequential Organ Failure Assessment (SOFA) [7]. Many patients with SIRS have no infection, while some patients without SIRS have an infection; SIRS used to diagnose sepsis failed to detect organ dysfunction in one of eight infected patients with organ dysfunction [8, 9]. The quick SOFA (qSOFA) has also been suggested as a bedside screening tool to detect organ dysfunction; however, its use was criticized given the delayed diagnosis that relies on established organ dysfunction, potentially decreasing the sensitivity [10–12]. Since the introduction of Sepsis-3, many studies have validated the new Sepsis-3 and qSOFA in several clinical fields such as emergency departments [13] and ICUs [14] in different medical conditions [15, 16]. However, only few studies have assessed the validation of sepsis-3 in patients with burns. In this study, we investigated the clinical outcome by evaluating the ability of the tools in predicting morality for each sepsis category using survival analyses and assessing the diagnostic performance of SIRS, qSOFA, and bSIRS for sepsis defined by the Sepsis-3 criteria in burn patients with suspected infection.
Methods
Study design and population
In this single-center retrospective cohort study, we reviewed electronic medical records of patients with suspected infection from January 2010 to December 2020. The patients were aged over 18 years, were admitted to the Burn Intensive Care Unit (BICU) of our institution within 2 days after the burns, and had undergone acute fluid resuscitation during the first 2 days of admission. Overall, 2384 patients aged over 18 years were included, and we excluded 440 patients because they were burned 24 hours before admission, 185 owing to incomplete fluid resuscitation, and 368 owing to no-suspected infection. In total, 1,391 eligible patients were divided into four subgroups according to different criteria (Fig 1). All analyzed patients were primarily admitted as they had burns and a suspected infection during BICU admission. This study was approved by the Institutional Review Board of Hangang Sacred Heart Hospital (HG2017-063). The requirement for informed consent was waived owing to the retrospective nature of the study.
Data collection and missing values
The electrical medical records were anonymized and retrieved from the clinical database warehouse in Hangang Sacred Heart Hospital, Hallym University, Seoul, Korea. The database consists of structured data collected prospectively for all patients admitted to the BICU. Patient demographics such as age, sex, comorbidities, and variables for burn-specific scoring systems such as the Abbreviated Burn Severity Index (ABSI) [17], Hangang [18], and revised Baux (rBaux) [19] scores were calculated for each patient upon admission. For critically ill patients, additional scores, such as the Acute Physiology and Chronic Health Evaluation (APACHE) IV, were calculated [20]. The proportion of missing data at admission was <5% for all variables. These missing values reflected tests not being performed at the physicians’ discretion under the assumption that they would return normal results; therefore, these missing values were replaced by normal values in the analyses. The scores of SOFA, qSOFA, SIRS, and bSIRS were calculated daily. Missing data regarding longitudinal variables were replaced with the values at the last observation. Only four longitudinal biomarkers (prothrombin time, lactate, total bilirubin, and bicarbonate) had more than 30% missing data. If the measured data were collected several times per day, the worst value of the day was included. The proportions and patterns of missing baseline and daily data are shown in S1 and S2 Figs in S1 File.
Outcomes and definitions
The primary outcome was 60-day in-ICU mortality. The secondary outcome was ICU length of stay (LOS). The gold standard for the diagnosis of sepsis (i.e., S3) was defined as an acute change in total SOFA scores of ≥2 points according to the Sepsis-3 criteria. Sepsis-3q (S3q) was defined as a qSOFA score ≥2, consisting of an altered mental state, respiratory rate ≥22 breaths/min, and systolic blood pressure ≤100 mmHg [7]. Sepsis-1 (S1) was defined as a SIRS score ≥2, consisting of a temperature >38°C or <36°C, heart rate >90 beats/min, respiratory rate >20 breaths/min, and white blood cell count >12,000 cells/mm3, <4,000/mm3, or >10% immature (band) forms. BS was defined as bSIRS ≥3, consisting of (1) a temperature >39°C or <36.5°C, (2) progressive tachycardia >110 bpm, (3) progressive tachypnea >25 bpm (not ventilated) or with minute ventilation >12 L/min, (4) thrombocytopenia <100,000 cells/μl, (5) hyperglycemia (in the absence of pre-existing diabetes mellitus) as indicated by an untreated plasma glucose >200 mg/dl, or by the presence of insulin resistance, and (6) inability to continue enteral feedings over 24 hours [6]. A suspected infection was defined as a situation where cultures to determine the infection source (i.e., blood, wounds, sputum, urine) were performed and antibiotics were administered before the results were received. A documented infection was defined as the presence of positive culture infection, pathologic tissue source, or clinical response to antimicrobials. Four sepsis criteria were assessed on a daily basis throughout the study period.
Statistical analysis
All continuous variables are expressed as means ± SD for normal distributions or median and interquartile ranges (IQR) for non-normal distributions. The independent t-test or Wilcoxon signed-rank test was used to determine differences between the two groups. Categorical variables were analyzed using the chi-squared test, and the frequencies are presented as percentages. The differences in the onset day for each sepsis category were compared using paired t-tests, with the onset day of each category for Sepsis-3 as the reference. Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for the diagnosis of sepsis were calculated. The hazard ratios (HRs) of in-ICU mortality for four dichotomized groups (i.e., S3, S3q, S1, and BS) were estimated using an extended Cox proportional hazards model to obtain longitudinal insight into how the individual components of the model affected the overall mortality of patients [21]. HR was also calculated after adjusting for known predictors of burn severity such as age, TBSA burned, and inhalation injury. Harrell’s C-index was used to evaluate the prediction accuracy of the models. P <0.05 (two-sided) was considered to be statistically significant. Data management and statistical analysis were performed using R (A language and environment for statistical computing, Vienna, Austria, Version 4.1.0).
Results
Study population
In this study, 1,391 burn patients with a suspected infection between January 2010 and December 2021 were included. All patients were divided into four groups according to the sepsis criteria. In total, 1,237 patients were diagnosed as having Sepsis-3 (S3), 1,307 patients as having Sepsis-1 (S1), 1,051 patients as having Sepsis-3q (S3q), and 1,005 patients as having BS (Fig 1 and Table 1).
Among them, 329 patients (23.73%) died. The overall median age was 50 years with a male predominance (81%). The median TBSA was 32.0%, and 649 patients (47%) presented inhalation injury. The median LOS in the ICU was 19.0 days. The median (interquartile) ABSI, rBaux, Hangang, and APACHE IV scores were 8 (7–10), 93 (76–114), 133 (122–148), and 38 (25–55), respectively. The infection sources of the sepsis were mostly the sputum (40%), wounds (35%), urine (23%), and blood (17%) (Table 2).
Survival analysis according to the sepsis criteria
All survival curves showed statistically significant differences in the sepsis criteria between the groups using log-rank tests. All HRs in the unadjusted models were statistically significant; however, the HR (0.726, p = 0.008) in patients diagnosed using Sepsis-1 was lower than that in patients diagnosed using other sepsis criteria. In the adjusted model, the HRs of Sepsis-3 (2.426, <0.001), Sepsis-3q (7.198, p < 0.001), and Sepsis-1 (0.575, p < 0.001) were statistically significant, with the latter showing a negative prediction. The HR of BS (0.966, p = 0.775) was below 1 and was not statistically significant. Adjusting its HR for known predictors such as age, TBSA, and inhalation injury, lowered its value, indicating that BS may be affected by burn factors but not by sepsis itself (Fig 2; S1 Table in S1 File). The mortality in each sepsis subgroup was 30.0% in BS, 30.0% in S3q, 25.7% in S3, and 24.2% in S1 (S3 Fig in S1 File).
(A) Sepsis-3. (B) Sepsis-3q. (C) Sepsis-1. (D) Burn Sepsis.
Detection days of each sepsis criteria
We analyzed the first detection day for each sepsis criteria to evaluate whether Sepsis-3 could diagnose sepsis early in the 909 patients who met all four sepsis criteria. The intersection of the four sepsis categories is presented in S4 Fig in S1 File. The mean onset day was the lowest with 4.13 ± 2.97 in Sepsis-3 and was statistically different with the onset in Sepsis-1 (4.23 ± 2.83), Sepsis-3q (6.26 ± 4.76), and BS (5.36 ± 4.88), as shown in S2 Table in S1 File; Fig 3.
Diagnostic performance comparison of qSOFA, SIRS, and bSIRS for Sepsis-3
We evaluated the diagnostic performance of dichotomized qSOFA, SIRS, and bSIRS for sepsis defined by the Sepsis-3 criteria. The flow chart of the three divided groups is shown in S5 Fig in S1 File. The sensitivity of SIRS (0.989, 95% CI: 0.982–0.994) was higher than that of qSOFA (0.841, 95% CI: 0.819–0.861) and bSIRS (0.803, 95% CI: 0.779–0.825). The specificity of qSOFA (0.929, 95% CI: 0.876–0.964) and bSIRS (0.922, 95% CI: 0.868–0.959) was higher than that of SIRS (0.461, 95% CI: 0.381–0.543) (S3 Table in S1 File). The diagnostic performance of these criteria and the Venn diagram are presented in Fig 4.
(A) Diagnostic performance. (B) Venn diagrams.
Discussion
Key findings
This large cohort study of burned patients with suspected infection from South Korea showed the clinical utility of the Sepsis-3 criteria to diagnose sepsis and distinguish the systemic response induced by sepsis from that induced by burns themselves. The Sepsis-3 criteria can predict mortality even when adjusted for known burn predictors without delaying diagnosis. The Sepsis-1 criteria can diagnose sepsis similar to the Sepsis-3 criteria in the early days; however, it can predict the mortality reversely with an adjusted HR of 0.646. The qSOFA and bSIRS criteria showed high specificity and low sensitivity; therefore, limiting their utility as diagnostic tools. There is no need to use diagnostic criteria specific to burn patients in this cohort; thus, the Sepsis-3 criteria can be used to compare them with patients with other diseases associated with sepsis. Sepsis-3, defined by SOFA ≥2, can be useful as an alternative diagnostic tool.
Relationship to previous studies
We previously reported a somewhat different conclusion. In our previous study, the Sepsis-3 criteria were not superior to the other BS criteria in the present study [22]. This is because the performance of mortality prediction was evaluated cross-sectionally at the time of diagnosis. The evaluation of the cut-off value of SOFA as a Sepsis-3 criterion might be limited as a diagnostic tool. Burns and trauma are two well-known conditions in which SIRS develops without infections, implying that that SIRS may not immediately detect sepsis in this population. Therefore, the diagnosis of sepsis in burn patients is complicated by the inability to distinguish the systemic response induced by infection and the burn itself. Burn physicians ignore SIRS as it is considered to be a normal response. Due to this, there are cases in which patients with sepsis are missed, although the actual sepsis is progressing [23]. Therefore, the modified definition of SIRS is proposed by ABA and is determined by meeting at least three variables among the following: temperature, heart rate, respiration rate, platelet count, blood glucose, and intolerance of feeding [6]. Nevertheless, several studies have reported the best indicators of sepsis in thermally injured patients [24]. Yan et al. reported that Sepsis-3 showed higher sensitivity (85%) than BS (59%), defined by ABA, and that multiple organ dysfunction is more specific for sepsis than inflammation. However, their dataset did not collect all relevant biomarker data prospectively, and their study included a smaller cohort than our own [4]. In this study, we found that the SIRS is too sensitive with a low specificity to detect sepsis. SIRS could not stratify the severity of sepsis based on our results that the HR ratio is lower than 1. This showed a negative prediction for the group, in which sepsis was defined by SIRS ≥2. bSIRS has high specificity and lower sensitivity, which reflects the severity but is not an appropriate diagnostic tool. Additionally, it showed adjusted HR values of 0.966, indicating that it cannot stratify the risk of mortality and could be affected by burn factors but not sepsis itself. qSOFA showed similar sensitivity and specificity to bSIRS but not to bSIRS; its HR showed a better discriminative power for mortality prediction. The specificity of qSOFA was higher than that of SIRS in our study and was comparable to that reported in other studies [15, 25].
Strengths and limitations
This study represents the largest series ever reported in the literature in this population. Although this was a retrospective study, the data were prospectively collected and stored in an electronic data warehouse. Our study is the first evaluation of qSOFA in burn patients with suspected infection, and it showed that qSOFA had similar clinical sensitivity and specificity as bSIRS. It also used an extended Cox analysis to determine the predictive ability of daily-collected SIRS, SOFA, qSOFA, and bSIRS scores, as almost always, the infection and sepsis-like clinical symptoms occur in burn patients throughout ICU hospitalization. We also evaluated the onset days of each sepsis category in the same cohort to determine whether diagnosis was delayed. However, this study had several limitations. First, this was a single-center study that reflects the specific case-mix of our center, and we recognize that the results will depend on our population characteristics and their specific treatments. Second, qSOFA was introduced as a bedside screening tool, but it was evaluated as a diagnostic tool for sepsis in this study. Third, missing data were considered to be normal or previous values. Fourth, regarding the definition of a suspected infection, it is assumed that the culture was undertaken when the physician suspected an infection; however, it cannot be ascertained whether this was the actual time of suspicion or routine tests. Fifth, the primary outcome in this study is mortality, which cannot be used to definitively validate the ability to predict sepsis. However, given that it is very difficult to detect the systemic response associated with sepsis, numerous physicians simply refer to “septic” patients, rather than strictly defined “sepsis.” Therefore, prediction of mortality, which is a crucial indicator in critical care patients, is an indirect way of evaluating the efficacy of a diagnostic tool for sepsis [14]. Furthermore, several studies have reported the validation of sepsis diagnostic tools using mortality as the primary outcome [15].
Conclusion
Sepsis defined by SOFA ≥2 can be a good alternative diagnostic tool because it reflects the severity of sepsis well using the survival analysis. SIRS showed higher sensitivity and lower specificity, while qSOFA and bSIRS showed higher specificity and lower sensitivity for the diagnosis of sepsis; therefore, they are not adequate for the diagnosis of sepsis.
References
- 1. D’Abbondanza JA, Shahrokhi S. Burn infection and burn sepsis. Surg Infect. 2021;22: 58–64. pmid:32364824
- 2. Bhattacharjee P, Edelson DP, Churpek MM. Identifying patients with sepsis on the hospital wards. Chest. 2017;151: 898–907. pmid:27374948
- 3. Church D, Elsayed S, Reid O, Winston B, Lindsay R. Burn wound infections. Clin Microbiol Rev. 2006;19: 403–434. pmid:16614255
- 4. Yan J, Hill WF, Rehou S, Pinto R, Shahrokhi S, Jeschke MG. Sepsis criteria versus clinical diagnosis of sepsis in burn patients: A validation of current sepsis scores. Surgery. 2018;164: 1241–1245. pmid:30049483
- 5. Mann-Salinas EA, Baun MM, Meininger JC, Murray CK, Aden JK, Wolf SE, et al. Novel predictors of sepsis outperform the American Burn Association sepsis criteria in the burn intensive care unit patient. J Burn Care Res. 2013;34: 31–43. pmid:23135212
- 6. Greenhalgh DG, Saffle JR, Holmes JH, Gamelli RL, Palmieri TL, Horton JW, et al. American Burn Association consensus conference to define sepsis and infection in burns. J Burn Care Res. 2007;28: 776–790. pmid:17925660
- 7. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA. 2016;315: 801–810. pmid:26903338
- 8. Kaukonen KM, Bailey M, Pilcher D, Cooper DJ, Bellomo R. Systemic inflammatory response syndrome criteria in defining severe sepsis. N Engl J Med. 2015;372: 1629–1638. pmid:25776936
- 9. Churpek MM, Zadravecz FJ, Winslow C, Howell MD, Edelson DP. Incidence and prognostic value of the systemic inflammatory response syndrome and organ dysfunctions in ward patients. Am J Respir Crit Care Med. 2015;192: 958–964. pmid:26158402
- 10. Dorsett M, Kroll M, Smith CS, Asaro P, Liang SY, Moy HP. qSOFA has poor sensitivity for prehospital identification of severe sepsis and septic shock. Prehosp Emerg Care. 2017;21: 489–497. pmid:28121217
- 11.
Sprung CL, Schein RM, Balk RA. The new sepsis consensus definitions: the good, the bad and the ugly. Springer; 2016.
- 12. Cortés-Puch I, Hartog CS. Opening the debate on the new sepsis definition change is not necessarily progress: revision of the sepsis definition should be based on new scientific insights. Am J Respir Crit Care Med. 2016;194: 16–18. pmid:27166972
- 13. Usman OA, Usman AA, Ward MA. Comparison of SIRS, qSOFA, and NEWS for the early identification of sepsis in the emergency department. Am J Emerg Med. 2019;37: 1490–1497. pmid:30470600
- 14. Raith EP, Udy AA, Bailey M, McGloughlin S, MacIsaac C, Bellomo R, et al. Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA. 2017;317: 290–300. pmid:28114553
- 15. Mak MHW, Low JK, Junnarkar SP, Huey TCW, Shelat VG. A prospective validation of Sepsis-3 guidelines in acute hepatobiliary sepsis: qSOFA lacks sensitivity and SIRS criteria lacks specificity (cohort study). Int J Surg. 2019;72: 71–77. pmid:31678690
- 16. Piano S, Bartoletti M, Tonon M, Baldassarre M, Chies G, Romano A, et al. Assessment of Sepsis-3 criteria and quick SOFA in patients with cirrhosis and bacterial infections. Gut. 2018;67: 1892–1899. pmid:28860348
- 17. Tobiasen J, Hiebert JM, Edlich RF. The abbreviated burn severity index. Ann Emerg Med. 1982;11: 260–262. pmid:7073049
- 18. Zampieri FG, Romano TG, Salluh JIF, Taniguchi LU, Mendes PV, Nassar AP, et al. Trends in clinical profiles, organ support use and outcomes of patients with cancer requiring unplanned ICU admission: a multicenter cohort study. Intensive Care Med. 2021;47: 170–179. pmid:32770267
- 19. Osler T, Glance LG, Hosmer DW. Simplified estimates of the probability of death after burn injuries: extending and updating the Baux score. J Trauma. 2010;68: 690–697. pmid:20038856
- 20. Zimmerman JE, Kramer AA, McNair DS, Malila FM, Shaffer VL. Intensive care unit length of stay: benchmarking based on Acute Physiology and Chronic Health Evaluation (APACHE) IV. Crit Care Med. 2006;34: 2517–2529. pmid:16932234
- 21. Liaw PC, Fox-Robichaud AE, Liaw KL, McDonald E, Dwivedi DJ, Zamir NM, et al. Mortality risk profiles for sepsis: a novel longitudinal and multivariable approach. Crit Care Explor. 2019;1: e0032. pmid:32166273
- 22. Yoon J, Kym D, Hur J, Kim Y, Yang HT, Yim H, et al. Comparative usefulness of Sepsis-3, burn sepsis, and conventional sepsis criteria in patients with major burns. Crit Care Med. 2018;46: e656–e662. pmid:29620554
- 23. Zhang P, Zou B, Liou YC, Huang C. The pathogenesis and diagnosis of sepsis post burn injury. Burns Trauma. 2021;9: tkaa047. pmid:33654698
- 24. Hill DM, Percy MD, Velamuri SR, Lanfranco J, Romero Legro IR, Sinclair SE, et al. Predictors for identifying burn sepsis and performance vs existing criteria. J Burn Care Res. 2018;39: 982–988. pmid:29771353
- 25. Serafim R, Gomes JA, Salluh J, Póvoa P. A comparison of the quick-SOFA and systemic inflammatory response syndrome criteria for the diagnosis of sepsis and prediction of mortality: A systematic review and meta-analysis. Chest. 2018;153: 646–655. pmid:29289687
