Correction
5 Feb 2016: Rosa RG, Roehrig C, de Oliveira RP, Maccari JG, Antônio ACP, et al. (2016) Correction: Comparison of Unplanned Intensive Care Unit Readmission Scores: A Prospective Cohort Study. PLOS ONE 11(2): e0148834. https://doi.org/10.1371/journal.pone.0148834 View correction
Figures
Abstract
Purpose
Early discharge from the intensive care unit (ICU) may constitute a strategy of resource consumption optimization; however, unplanned readmission of hospitalized patients to an ICU is associated with a worse outcome. We aimed to compare the effectiveness of the Stability and Workload Index for Transfer score (SWIFT), Sequential Organ Failure Assessment score (SOFA) and simplified Therapeutic Intervention Scoring System (TISS-28) in predicting unplanned ICU readmission or unexpected death in the first 48 hours after discharge from the ICU.
Methods
We conducted a prospective cohort study in a single tertiary hospital in southern Brazil. All adult patients admitted to the ICU for more than 24 hours from January 2008 to December 2009 were evaluated. SWIFT, SOFA and TISS-28 scores were calculated on the day of discharge from the ICU. A stepwise logistic regression was conducted to evaluate the effectiveness of these scores in predicting unplanned ICU readmission or unexpected death in the first 48 hours after discharge from the ICU. Moreover, we conducted a direct accuracy comparison among SWIFT, SOFA and TISS-28 scores.
Results
A total of 1,277 patients were discharged from the ICU during the study period. The rate of unplanned ICU readmission or unexpected death in the first 48 hours after discharge from the ICU was 15% (192 patients). In the multivariate analysis, age (P = 0.001), length of ICU stay (P = 0.01), cirrhosis (P = 0.03), SWIFT (P = 0.001), SOFA (P = 0.01) and TISS-28 (P<0.001) constituted predictors of unplanned ICU readmission or unexpected death. The SWIFT, SOFA and TISS-28 scores showed similar predictive accuracy (AUC values were 0.66, 0.65 and 0.74, respectively; P = 0.58).
Citation: Rosa RG, Roehrig C, Oliveira RPd, Maccari JG, Antônio ACP, Castro PdS, et al. (2015) Comparison of Unplanned Intensive Care Unit Readmission Scores: A Prospective Cohort Study. PLoS ONE 10(11): e0143127. https://doi.org/10.1371/journal.pone.0143127
Editor: Jorge IF Salluh, D'or Institute of Research and Education, BRAZIL
Received: April 12, 2015; Accepted: October 31, 2015; Published: November 23, 2015
Copyright: © 2015 Rosa et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: Relevant data are within the paper and its Supporting Information files.
Funding: The authors have no support or funding to report.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Rates of intensive care unit (ICU) readmission have become a metric of hospital and provider performance as well as a means by which to incentivize efficient, high quality, and coordinated patient care [1]. The Quality Indicators Committee of the Society of Critical Care Medicine has stated that readmission within 48 hours is a major performance indicator of the quality of intensive care medicine [2,3]
Prolonged duration of stay in an ICU is costly, stressful for patients and families, reduces the number of beds available for other patients, and can increase risk for iatrogenic and nosocomial complications [4]. However, early discharge from the ICU is not without risk. If patients requiring high intensity care are discharged before they can be safely cared for in a lower-acuity care environment, they are at risk for both complications and delayed recognition of clinical deterioration. The former can result in the need for unplanned ICU readmission; the latter can result in patient death [5,6]. In addition, ICU readmission also places additional stress on patients, ICU staff and families.
Risk stratification of patients discharged from the ICU is a complex process with many potential challenges. Several risk stratification tools have been developed; however, at present it is unclear whether the existing tools provide value above clinical judgment or whether they can be used to improve healthcare delivery [7–14]. Previously identified predictors of death or ICU readmission include duration of ICU stay, Glasgow Coma Scale at the time of ICU discharge, mean arterial blood pressure, and ICU admission source [15]. Others have attempted to create decision support tools to assist in ICU discharge readiness assessment [16]. The Sequential (sepsis-related) Organ Failure Assessment score (SOFA) is used to track a patient’s status during admission to the ICU. SOFA is a scoring system primarily designed to determine the extent of a person’s organ function or rate of failure, not to predict ICU readmission [17,18]. The Stability and Workload Index for Transfer score (SWIFT), which was developed to predict readmission or death within 1 week of ICU discharge, has demonstrated only moderate discrimination power to predict these [14].
In addition to the severity of illness score, there is also an association between nursing workload and post-ICU mortality [19,20]. The simplified Therapeutic Intervention Scoring System (TISS-28) has been widely applied to assess workload and resource allocation in intensive care, thereby measuring treatment intensity [4]. Several authors [19–21] have shown an association between the TISS-28 value on the last ICU day and post-ICU mortality, and therefore, an indirect association with ICU readmission.
Unfortunately, there are few data available regarding a comparison among distinct ICU readmission scores. Accordingly, we performed the present study to compare the effectiveness of SWIFT, SOFA and TISS-28 scores in predicting early unplanned ICU readmission or death after discharge from the ICU.
Methods
Study design, patients and setting
A prospective cohort study was conducted at a single tertiary centre. The present study followed all intensive care patients >18 years of age who were consecutively discharged from the 31-bed mixed medical-surgical ICU of the Hospital Moinhos de Vento in Porto Alegre, Brazil, from January 2008 to December 2009. Subjects who had an ICU length of stay <24 hours were excluded. Patients were not allowed to reenter the study after a first episode of ICU admission.
Definitions
The main independent variables were SWIFT, SOFA and TISS-28 scores calculated on the day of discharge from the ICU by researchers who were not associated with the attending physician’s team. Elements of the SWIFT score include ICU admission source, ICU length of stay, day of discharge GCS, last PaO2/FiO2 ratio, and last arterial blood gas PaCO2 (Table 1) [14]. The SOFA score is based on extent of the patient’s organic function determined by physiological parameters of respiratory, cardiovascular, hepatic, coagulation, neurologic, and renal systems (Table 2) [17]. The TISS-28 score comprises interventions related to basic activities, cardiovascular support, specific interventions, ventilator support, renal support, neurologic support, and metabolic support (Table 3) [22].
Other covariates analysed included age, sex, type of comorbidity, need for and length of mechanical ventilation, need for tracheostomy, and length of ICU stay.
Outcomes and follow-up
The primary outcomes of the study were unplanned ICU readmission or unexpected death in the first 48 hours after discharge from the ICU. The 48 hours cut-off for readmission or death was chosen because this timeframe is often accepted for evaluating the quality of ICU discharge; given that earlier the readmission or death, the more likely the patient was unprepared to be discharge from ICU. [2,3].
Patients were followed up through interviews and medical record reviews using a standardised case report form, by researchers who were not associated with the attending physician’s team. Follow-up was maintained for 48 hours after discharge from the ICU.
Statistical analysis
A stepwise backward multivariate logistic regression was performed to determine whether SWIFT, SOFA and TISS-28 scores were predictors of unplanned ICU readmission or unexpected death in the first 48 hours after discharge from the ICU. All variables with P<0.15 in the univariate analysis were included. In the multivariate model, independent variables were eliminated from the highest to the lowest P-value, but retained in the model if P<0.05. Odds ratios (ORs) were estimated along with the 95% confidence intervals (CIs). The accuracy of different score systems for predicting unplanned ICU readmission or unexpected death in the first 48 hours after ICU discharge was evaluated through the area under the receiver operating characteristic (ROC) curve (AUC). Values of the AUC greater than 0.8 were considered good, between 0.6–0.8 moderate, and lower than 0.6 poor for prediction performance. Univariate AUCs modeling of the three scores were compared using the chi-squared test. Observed and predicted unplanned ICU readmission or unexpected death in the first 48 hours after discharge from the ICU were compared using the Hosmer-Lemeshow test. Stata Statistical Software Release 12 (StataCorp LP, College Station, TX, USA) was used for statistical analysis.
Results
During the study period, 1,277 patients were discharged from the ICU. The characteristics of study population are summarized in Table 4. The mean age of the cohort was 67 years. The mean Apache-II and SOFA scores on the day of ICU admission were 15.4 and 2.8, respectively. The predominant underlying comorbidities were ischemic heart disease (25.5%), diabete mellitus (22.0%), malignant neoplasia (21.0)%, peripheral vascular disease (13.0%), chronic obstructive pulmonary disease (11.9%) and heart failure (11.5%). Surgical patients comprised 39.3% of the study population. Mechanical ventilation was needed in 27.7% of patients and the mean length of mechanical ventilation was 5.3 days. The mean length of ICU stay was 7.3 days. On the day of discharge from ICU the mean SWIFT, SOFA and TISS-28 scores were 12.0, 1.0 and 11.5, respectively. All patients were directly discharged to general medical or surgical wards without hospitalization in intensive care step-down units.
The overall rate of unplanned ICU readmission or unexpected death in the first 48 hours after ICU discharge was 15.0% (192 patients). Of these 126 patients (65.6%) had unplanned ICU readmission and 66 patients died unexpectedly (34.4%). The main reasons for ICU readmission were acute respiratory failure (46.0%), sepsis (30.9%), cardiac conditions such as arrhythmia and congestive heart failure (15.8%) and neurologic impairment (7.1%). Among those patients who died unexpectedly, 31% had the “do not resuscitate” order at ICU discharge.
In the univariate analysis of risk factors for unplanned ICU readmission or unexpected death in the first 48 hours after ICU discharge (Table 5), age (P<0.001), previous heart failure (P = 0.004), previous chronic obstructive pulmonary disease (P = 0.02), previous cancer (P = 0.01), previous cirrhosis (P = 0.01), renal replacement therapy (P = 0.01), mechanical ventilation required during ICU stay (P<0.001), duration of mechanical ventilation (P<0.001), need for tracheostomy (P = 0.001), length of ICU stay (P<0.001), SWIFT (P<0.001), SOFA (P<0.001) and TISS-28 (P<0.001) scores were positively associated with the main outcomes. After multivariate analysis was conducted (Table 6), variables that constituted independent risk factors for unplanned ICU readmission or unexpected death in the first 48 hours after discharge from the ICU included age (OR, 1.01; 95% CI, 1.006–1.028), length of ICU stay (OR, 1.01; 95% CI, 1.003–1.030), previous cirrhosis diagnosis (OR, 2.70; 95% CI, 1.06–6.84), SWIFT (OR, 1.03; 95% CI, 1.01–1.06), SOFA (OR, 1.12; 95% CI, 1.02–1.24) and TISS-28 (OR, 1.12; 95% CI, 1.06–1.18) scores.
The accuracy analysis of SWIFT, SOFA and TISS-28 scores showed only moderate discrimination power to predict unplanned ICU readmission or unexpected death in the first 48 hours after discharge from the ICU, for all three scores (Table 7, Fig 1). The Hosmer-Lemeshow P-values for SWIFT, SOFA and TISS-28 scores were 0.66, 0.65 and 0.74, respectively, showing good calibration of the three predictive scores. A direct accuracy comparison among the three scores showed no statistical difference. Table 8 shows the relationship between sensitivity and specificity according to determined cutoffs for SWIFT, SOFA and TISS-28 scores.
Discussion
In the present study, SWIFT, SOFA and TISS-28 scores evaluated on the day of discharge from the ICU were independently associated with unplanned ICU readmission or unexpected death in the first 48 hours after ICU discharge; however, all three scores showed only moderate predictive accuracy. Additionally, this study failed to find any accuracy difference among SWIFT, SOFA and TISS-28 scores in the ability to predict unplanned ICU readmission or unexpected death in the first 48 hours after discharge from the ICU.
Previous publications have shown promising results of using the SWIFT score to predict unplanned ICU readmission [14,23,24]. For example, Gagic et al. [14] demonstrated the superiority of the SWIFT score, compared with APACHE III, in predicting unplanned ICU readmission (AUC 0.75 [95% CI, 0.70–0.80] versus AUC 0.62 [95% CI, 0.56–0.68]). Moreover, Oakes et al. [23] found good predictive accuracy for SWIFT (AUC 0.76 [95% CI, 0.61–0.91]) in a small sample of ICU patients in southern Brazil. However, in our cohort, the SWIFT, a tool designed specifically to predict unplanned ICU readmission, was not superior to conventional scores of clinical severity (SOFA) or loading interventions (TISS-28) on the day of discharge from the ICU. Congruent to our results, the retrospective study of Kastrup et al. [24] concluded that the SWIFT score was not advantageous when deciding whether a patient can be safely discharged from the ICU, owing to its poor accuracy (AUC 0.58 [95% CI, 0.55–0.60]).
Our findings reinforce that, at present, it is unclear whether existing ICU readmission scores provide value above clinical judgment or whether they can be used to improve outcomes in patient care transition scenarios. Similarly, a systematic review of tools for predicting severe adverse events following patient discharge from the ICU concluded that further evaluation of existing ICU readmission scores is required prior to clinical implementation, given that it is unclear whether a reliable and valid risk stratification tool for patient ICU discharge has been developed [25]. We hypothesise that unplanned ICU readmission or unexpected death is not fully explained by patient characteristics such as extent of organ dysfunction. Organizational aspects of care following ICU discharge should be incorporated into the predictive equation of unplanned readmission. For example, providing step-down units (e.g., respiratory or intermediate care units), for ICU-discharged patients who need more intensive monitoring and rehabilitation care than can be provided in a general medical ward, might be a good way to avoid unexpected clinical deterioration in some patients following ICU discharge [26]. Perhaps we are focusing too much on patient characteristics and ICU needs, and forgetting the type of hospital care required after discharge from the ICU [27,28].
It is interesting to note that the three tools (SWIFT, SOFA and TISS-28), which were developed for different purposes, demonstrated the same accuracy in assessing the outcomes studied. SOFA was designed to determine the extent of a person’s organ function or rate of failure, SWIFT was developed to predict readmission or death within 1 week of ICU discharge, and TISS-28 is applied to assess workload and resource allocation in intensive care, measuring treatment intensity [4,14,17,18]. In our opinion, these findings represent the current lack of understanding regarding the pathophysiological mechanisms of clinical deterioration in patients and need for correct identification of risk factors that accurately reflect the need for readmission of patients to an ICU.
The present study had some limitations. Our rates of ICU readmission in the first 48 hours after discharge from the ICU were higher than in previous reports from ICUs in Europe and North America, which have ICU readmission rates around 2–5% [6,9,10,14,15]. For example, the study of Badawi et al. found rates of readmission and death within 48hs of ICU discharge of 2.5% and 0.9%, respectively, in a robust cohort of more than 700,000 ICU patients [29]. These differences are possibly owing to intensive care practice in Brazil, which is characterized by difficulties in establishing exclusive palliative care during and after ICU discharge and a lack of step-down units for selected patients discharged from the ICU. In addition, assessment of patients only in the first 48 hours after discharge from the ICU may cause difficulties in generalizing the findings of the study, given that previous data suggest that fewer than approximately 50% of ICU readmissions occur less than 48 hours after discharge [6,9]. Another 25% of readmissions occur between 2 and 7 days after ICU discharge; these data were not evaluated in our study. Afternoon and evening discharge are important risk factors for ICU-readmission as well and were not evaluated here. Furthermore, approximately 60% of patients are readmitted for different diagnoses than their original diagnosis, an impossible risk to measure [6,9,16]. Nevertheless, the possibility of systematic errors was minimized by proper measurement of variables and outcomes using previously defined objective criteria, the use of standardized data collection, and prospective follow-up performed by a research team that was not involved in patient care.
Future research should explore how patients’ severity of illness at ICU discharge (rather than at ICU admission), as well as floor-based rather than ICU-based organizational structures, contribute to ICU readmission risk. Future research should also examine whether decision making by residents influences ICU readmission rates.
Conclusions
We conclude that SWIFT, SOFA and TISS-28 scores evaluated on the day of discharge from the ICU can be used to predict unplanned ICU readmission or unexpected death in the first 48 hours after discharge from the ICU, however, with only moderate predictive accuracy.
Acknowledgments
The authors thank the data collection team who created the database, as well as the Hospital Moinhos de Vento, particularly the Intensive Care Unit, for their support in conducting the study. This study received no financial support.
Author Contributions
Conceived and designed the experiments: CR JGM ACPA PSC PCB CT. Performed the experiments: RGR RPO JGM FLDN PCB CT. Analyzed the data: RGR RPO JGM CT. Wrote the paper: RGR CR JGM ACPA CT.
References
- 1. Center for Medicare and Medicaid Services. Medicare Program; Changes to the Hospital Inpatient Prospective Payment Systems and Fiscal Year 2009 Rates; Payments for Graduate Medical Education in Certain Emergency Situations; Changes to Disclosure of Physician Ownership in Hospitals and Physician Self-Referral Rules; Updates to the Long-Term Care Prospective Payment System; Updates to Certain IPPS-Excluded Hospitals; and Collection of Information Regarding Financial Relationships Between Hospitals. Final rules. Fed Regist. 2008;73(161): 48433–9084. Available: http://www.cms.gov/AcuteInpatientPPS/downloads/CMS-1390-F.pdf. pmid:18956499
- 2. Angus DC. Grappling with intensive care unit quality—does the readmission rate tell us anything? Crit Care Med. 1998;26: 1779–1780. pmid:9824061
- 3. Timmers TK, Hulstaert PF, Leenen LP. Patient outcomes can be associated with organizational changes: a quality improvement case study. Crit Care Nurs Q. 2014;37: 125–134. pmid:24309466
- 4. Capuzzo M, Moreno RP, Alvis R. Admission and discharge of critically ill patients. Curr Opin Crit Care. 2010;16: 499–504. pmid:20644469
- 5. Rosenberg AL, Watts C. Patients readmitted to ICUs: A systematic review of risk factors and outcomes. Chest. 2000;118: 492–502. pmid:10936146
- 6. Kramer AA, Higgins TL, Zimmerman JE. Intensive care unit readmissions in U.S. hospitals: Patient characteristics, risk factors, and outcomes. Crit Care Med. 2012;40: 3–10. pmid:21926603
- 7. Timmers TK, Verhofstad MHJ, Moons KGM, Leenen LPH. Intensive care performance: How should we monitor performance in the future? World J Crit Care Med. 2014;3(4): 74–79. pmid:25374803
- 8. Campbell AJ, Cook JA, Adey G, Cuthbertson BH. Predicting death and readmission after intensive care discharge. Br J Anaesth. 2008;100: 656–662. pmid:18385264
- 9. Brown SES, Ratcliffe SJ, Kahn JM, Halpern SD. The epidemiology of intensive care unit readmissions in the United States. Am J Respir Crit Care Med. 2012;185(9): 955–964. pmid:22281829
- 10. Chen LM, Martin CM, Keenan SP, Sibbald WJ. Patients readmitted to the intensive care unit during the same hospitalization: Clinical features and outcomes. Crit Care Med. 1998;26: 1834–1841. pmid:9824076
- 11. Rosenberg AL, Hofer TP, Hayward RA, Strachan C, Watts CM. Who bounces back? Physiologic and other predictors of intensive care unit readmission. Crit Care Med. 2001; 29: 511–518. pmid:11373413
- 12. Daly K, Beale R, Chang RW. Reduction in mortality after inappropriate early discharge from intensive care unit: Logistic regression triage model. BMJ. 2001; 322:1274–1276. pmid:11375229
- 13. Chandra S, Agarwal D, Hanson A, Farmer JC, Pickering BW, Gajic O, et al. The use of an electronic medical record based automatic calculation tool to quantify risk of unplanned readmission to the intensive care unit: A validation study. J Crit Care. 2011;26: 634.e9–634.e15.
- 14. Gajic O, Malinchoc M, Comfere TB, Harris MR, Achouiti A, Yilmaz M, et al. The Stability and Workload Index for Transfer score predicts unplanned intensive care unit patient readmission: Initial development and validation. Crit Care Med. 2008;36: 676–682. pmid:18431260
- 15. Chen LM, Martin CM, Keenan SP, Sibbald WJ. Patients readmitted to the intensive care unit during the same hospitalization: Clinical features and outcomes. Crit Care Med. 1998;26: 1834–1841. pmid:9824076
- 16. Zimmerman JE, Wagner DP, Draper EA, Knaus WA. Improving intensive care unit discharge decisions: supplementing physician judgment with predictions of next day risk for life support. Crit Care Med. 1994;22: 1373–1384. pmid:8062558
- 17. Vincent JL, Moreno R, Takala J, Willatts S, De Mendonça A, Bruining H, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure: On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22: 707–710. pmid:8844239
- 18. Vincent JL, de Mendonça A, Cantraine F, Moreno R, Takala J, Suter PM, et al. Use of the SOFA score to assess the incidence of organ dysfunction/failure in intensive care units: results of a multicenter, prospective study. Working group on “sepsis-related problems” of the European Society of Intensive Care Medicine. Crit Care Med. 1998;26: 1793–1800. pmid:9824069
- 19. Moreno R, Miranda DR, Matos R, Fevereiro T. Mortality after discharge from intensive care: the impact of organ system failure and nursing workload use at discharge. Intensive Care Med. 2001;27: 999–1004. pmid:11497159
- 20. Smith L, Orts CM, O’Neil I, Batchelor AM, Gascoigne AD, Baudouin SV. TISS and mortality after discharge from intensive care. Intensive Care Med. 1999;25: 1061–1065. pmid:10551959
- 21. Beck DH, McQuillan P, Smith GB. Waiting for the break of dawn? The effects of discharge time, discharge TISS scores and discharge facility on hospital mortality after intensive care. Intensive Care Med. 2002;28: 1287–1293. pmid:12209279
- 22. Moreno R, Morais P. Validation of the simplified therapeutic intervention scoring system on an independent database. Intensive Care Med. 1997;23: 640–644. pmid:9255643
- 23. Oakes DF, Borges IN, Forgiarini Junior LA, Rieder Mde M. Assessment of ICU readmission risk with the Stability and Workload Index for Transfer score. J Bras Pneumol. 2014;40(1): 73–76. pmid:24626273
- 24. Kastrup M, Powollik R, Balzer F, Röber S, Ahlborn , Von Dossow-Hanfstingl V, et al. Predictive ability of the stability and workload index for transfer score to predict unplanned readmissions after ICU discharge. Crit Care Med. 2013;41(7): 1608–1615. pmid:23660731
- 25. Hosein FS, Bobrovitz N, Berthelot S, Zygun D, Ghali WA, Stelfox HT. A systematic review of tools for predicting severe adverse events following patient discharge from intensive care units. Crit Care. 2013;17(3):R102. pmid:23718698
- 26. Vincent JL, Rubenfeld GD. Does intermediate care improve patient outcomes or reduce costs? Crit Care. 2015;19(1): 89.
- 27. Campos RA, Tena SA, Muncharaz AB, Font SM, Sellés AF, Campos LM, et al. Study of post-ICU mortality during 4 years (2006–2009): Analysis of the factors related to death in the ward after discharge from the ICU. Med Intensiva. 2011;35(3): 150–156. pmid:21356566
- 28. Giakoumidakis K, Eltheni R, Patelarou A, Patris V, Kuduvalli M, Brokalaki H. Incidence and predictors of readmission to the cardiac surgery intensive care unit: A retrospective cohort study in Greece. Ann Thorac Med. 2014;9(1): 8–13. pmid:24551011
- 29. Badawi O, Breslow M. Readmissions and death after ICU discharge: development and validation of two predictive models. PLoS One. 2012;7(11): e48758. pmid:23144958