Introduction

Traumatic head injury is a major cause of death [1]. Practitioners are requested to identify patients with such injuries who will survive after arrival at the emergency department, and start appropriate medical practice immediately. For this reason, many attempts to establish appropriate prediction models have been conducted worldwide [2–14]. Available models include the Abbreviated Injury Scale developed by the Association for the Advancement of Automobile Medicine [2] and the Injury Severity Score (ISS) for evaluating emergency care for multiple injuries [3]. The Traumatic Coma Data Bank, which was a multicenter epidemiological study of traumatic head injury conducted in the USA [4], led to the evaluation and classification of computed tomography findings for prediction of prognosis [5, 6].

The Trauma and Injury Severity Score (TRISS) model [7] was developed in 1987, and has been used as a tool for predicting survival [8]. The TRISS includes the ISS for anatomical severity [3] and Revised Trauma Score for physiologic reserve. The TRISS also covers the Glasgow Coma Scale (GCS), systolic blood pressure (SBP), respiratory rate (RR), and age [7]. The TRISS method was later modified for intubated patients, which improved the prediction accuracy [9]. The modified Kampala [10], Trauma Mortality Prediction Model [11], and modified McPeek [12] were further suggested as models that improved prediction accuracy.

Recently, two new prediction models have been proposed and discussed worldwide: the Corticosteroid Randomization After Significant Head Injury (CRASH) and the International Mission for Prognosis and Analysis of Clinical Trials in Traumatic Brain Injury (IMPACT) [13]. The National Neuroscience Institute (NNI) in Singapore also reported new prediction models (NNI clinical and NNI+) following a cohort study [14].

The Japan Neurotrauma Data Bank (JNTDB) was founded in 1998, and is an authorized nationwide database for epidemiological studies of traumatic brain injury (TBI) [15]. The JNTDB has conducted three separate series of studies, with Project 2009 (conducted from 2009–2011) being the most recent open database available. Although the JNTDB is well established, there has been no comprehensive report on the availability of effective prediction models for death and prognosis following TBI in Japan, including the TRISS, CRASH, and IMPACT models.

The present study aimed to apply these international prediction models to JNTDB data and externally evaluate these models, thereby establishing the appropriateness of the models in the Japanese population.

Methods

The dataset used in this study was drawn from the JNTDB [15]. The JNTDB started collecting data in 1998 To date, three studies have drawn on these data, the most recent being Project 2009, in which information for patients from 22 registered institutions was collected for 2 years (from 1 July 2009 to 30 June 2011), as in S1 Table. The JNTDB included patients of any age, but excluded patients with cardiopulmonary arrest on arrival (CPAOA) not suspected of being attributable to TBI. All patients with a GCS score ≤8 on admission or during follow-up were included. Patients with a GCS score >8 who had undergone craniotomy (chronic subdural hematoma excluded; burr holes included) were also included.

The inclusion and exclusion criteria for patients in the present study are presented in Table 1. Project 2009 included 1,091 patients. Of those patients, 325 patients with a GCS score >8 were excluded from the present study. After exclusion of 36 patients younger than 16 years, one patient with unknown outcome at discharge, and 94 patients with CPAOA, data for 635 patients remained for analysis in this study. Of these patients, 265 (41.7%) had been injured in motor vehicle accidents and 309 (48.7%) by falls. Among the 635 cases, 293 cases died in hospital. For unfavorable outcome at 6 months, an additional 11 cases died after discharge, meaning that 304 of the 635 cases died.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Inclusion and exclusion criteria for patients in the present study.

https://doi.org/10.1371/journal.pone.0221791.t001

Permission to analyze JNTDB data was obtained from the Japan Society of Neurotraumatology before starting this study. This study was approved by the Institutional Ethics Review Board of Nihon University School of Medicine (notice number 26–9). The requirement for patient consent was waived because only anonymized data were used.

Results

Demographic and examined factors for patients in univariate analyses are summarized in Table 2. The mean age of patients with recorded mortality in hospital (M-IH) was 60.0 ± 21.1 years (range 16–98 years), and 69.7% were men. Logistic analysis of the three SBP groups showed statistically significant differences among the groups (p<0.0001, N = 627), indicating that both higher and lower SBP were related to higher mortality. Low heart rate also appeared to be related to higher mortality. No special tendency in terms of mortality was observed in relation to RR, but high SG and the presence of mydriasis were risk factors for mortality. In addition, the presence of acute subdural hematoma, intracerebral hemorrhage, and traumatic subarachnoid hemorrhage were risk factors for mortality, whereas the presence of acute epidural hematoma (AEDH) tended to reduce mortality. There appeared to be no significant relationship between cerebral contusion and mortality.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2.

https://doi.org/10.1371/journal.pone.0221791.t002

In total, 512 patients were included in the analysis for prediction of unfavorable outcome 6 months after TBI (UO-6M), 70.1% of which were men. The GCS was lower in patients with unfavorable outcomes, whereas the ISS was higher. Both low and high SBP tended to be related to poorer outcomes, but there was no obvious relationship between heart rate or RR and outcome. Higher SG may also be a risk factor for an unfavorable outcome. The presence of mydriasis and acute subdural hematoma appeared to be related to poorer outcomes, whereas there was no clear risk associated with the presence of intracerebral hemorrhage, cerebral contusion and traumatic subarachnoid hemorrhage. In contrast, the presence of AEDH indicated better outcomes for patients.

AUC analyses were performed to determine the efficacy of the TRISS, CRASH, and IMPACT models (Table 3). Cases with unavailable data for some factors were included in the analyses of 635 patients for M-IH and 512 patients UO-6M. As a result of eliminating those patients for the validation of each prediction model, the number of patients varied among the prediction models (Table 3). To compare the accuracy of the prediction models for M-IH, data for 600 patients were examined using the TRISS model, and showed an AUC of 0.75. For prediction of UO-6M, data were examined using CRASH basic (511 patients), CRASH computed tomography (504 patients), IMPACT core (511 patients), and IMPACT extended (450 patients). The IMPACT lab model could not be applied in this study because of the lack of blood hemoglobin data in the JNTDB. For UO-6M, the AUCs for the CRASH basic, CRASH computed tomography, IMPACT core, and IMPACT extended models were 0.86, 0.86, 0.81, and 0.85, respectively. The AUCs for the CRASH models tended to be higher than those for the IMPACT model.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Area under the receiver operating characteristic curves for the TRISS, CRASH, and IMPACT models for external validation using Japan Neurotrauma Data Bank data.

https://doi.org/10.1371/journal.pone.0221791.t003

Discussion

The major focus of the present study was the applicability of currently used international prediction models for patients with severe TBI in Japan. Although the TRISS model has often been applied for the Japanese population, there have been no evaluations of the applicability of other international models such as the CRASH and IMPACT models. This highlights a lack of information about sophisticated standards for predicting UO-6M in Japan. We therefore attempted to externally evaluate the three international prediction models for UO-6M based on JNTDB data.

A major concern was if the prediction models were applicable to patients severe TBI. Sophisticated predictive models should require minimal routinely-measured factors, which may save clinical time and resources. In the JNTDB, RR and SG were the prediction factors most commonly missing. Of the 635 patients included in the present study, RR and SG were not recorded for 4.4% and 1.7% of patients, respectively. Because RR is needed for calculating TRISS scores, TRISS scores could only be calculated for 600 patients (94.5%). Despite the TRISS model having limited applicability to JNTDB data, the AUC for the TRISS was 0.75, indicating this model can be used for the Japanese population. However, no authorized Japanese prediction models are available to date.

The applicability of the CRASH and IMPACT models for predicting UO-6M for 512 patients is summarized in Table 3. The prediction scores for the CRASH basic and computed tomography models could be calculated for 511 (99.8%) and 450 (87.9%) patients, respectively. Core and extended IMPACT scores could be calculated for 511 (99.8%) and 450 (87.9%) patients, respectively.

Internationally, a number of studies have examined the usefulness of the TRISS, CRASH, and IMPACT models. In 2016, Sun et al. summarized the performance of the IMPACT models as determined by external validation [17]. They compared six studies that had investigated prediction of mortality or unfavorable outcomes [14, 18–22]. The AUC ranges for mortality prediction using the IMPACT models were 0.65–0.85 (core), 0.69–0.88 (extended), and (0.69–0.90) lab. For unfavorable outcome prediction, the ranges were 0.66–0.84 (core), 0.71–0.88 (extended), and 0.70–0.87 (lab). In the JNTDB, the AUCs for the core and extended IMPACT models were 0.81 and 0.85, respectively. These AUCs suggest the IMPACT models have a comparable fit with patients in Japan, although these scores can be calculated for fewer patients.

Han et al. reported that the new NNI models performed better than CRASH and IMPACT models, as indicated by a lower Akaike information criterion and greater AUC [14]. We were unable to analyze their models using JNTDB data because of insufficient information on the model’s structure. Future studies are needed to determine the applicability and limitations of those models for JNTDB data.

Brennan et al. recently proposed a simplified prediction model for TBI prognosis called the GCS-Pupils score (GCS-P), which was derived from the CRASH and IMPACT databases [23]. The GCS-P combines the GCS score and state of the pupils with ranges of 3–15 and 0–2, respectively (the overall GCS-P has a range of 1–15). The simplicity of the model makes it convenient for clinical staff; however, the loss of accuracy in prediction is a potential disadvantage of the GCS-P. To clarify the value of the GCS-P, further study is necessary using JNTDB data.

We could not determine why the presence of AEDH was a better prognostic factor for UO-6M in our multivariate analysis. In patients with AEDH, the complication of acute subdural hematoma was significantly less frequent (p<0.0001). Such a difference in these patients’ pathological state might have influenced the prognosis of patients with AEDH. Further studies are necessary to clarify this issue.

The present study had several major limitations. First, the obtained data were limited to patients who had been admitted to the 22 participating institutions. Of note, all institutions that participated in JNTDB Project 2009 were either university hospitals or central medical centers. It is possible that the characteristics of patients with TBI presenting to small hospitals differ from those of patients presenting to major institutions. Second, we were unable to compare the accuracy of prediction for the IMPACT lab, NNI, and NNI+ models because we were unable to obtain the prediction equations for these models and therefore could not assess them using JNTDB Project 2009 data. It is possible that these models would have better AUCs in our dataset. Third, the JNTDB dataset used for the present analysis was not contemporary because of limitations in access to raw data for our research purposes. The overall mortality and outcomes after TBI in Japan are likely to have been improved following changes in resuscitation techniques (e.g., volume resuscitation) since 2009 [24]. We intend to reevaluate these prediction models using contemporary data from the next available database in the near future. Fourth, we could externally evaluate three international prediction models only for patients with severe TBI because of the limitation of bias in the inclusion criteria for the JNTDB dataset. The JNTDB Project 2009 included three categories of patients aged 16 years or older: patients with a GCS score of ≤8, patients with a GCS score of ≤8 during follow up, and patients with a GCS score >8 who had undergone craniotomy. The major bias for inclusion of patients for the present study lies with the latter two groups because these groups were not defined in the three prediction models. Inclusion of these groups might have led to partial inclusion of patients with a GCS score >8 on admission, leading to incorrect validation of the models. For perfect validation of three models using JNTDB data, it would be necessary to include all patients with a GCS >8. Nevertheless, our results clearly indicate that all three models were appropriate for prediction of prognosis in those with severe TBI. It should also be noted that we did not perform a power analysis to evaluate the reliability of the JNTDB. A power analysis would have allowed estimation of the quality of the JNTDB data. However, the JNTDB is the only database for TBI in Japan, and we had no other options against which to compare the appropriateness of our analyses in the present study. Power analysis will be of particular value when we analyze the next JNTDB project in comparison with the results of the present study.

In addition to our previous approach to establish the prediction model for severe head injury in Japanese children [25], the present external validation of the TRISS, CRASH, and IMPACT models has clarified their value for use in Japan.

Conclusion

A systematic external validation of the TRISS, CRASH, and IMPACT prediction models revealed these models have convincing values for prediction of outcomes for Japanese patients with severe TBI. These models should be widely applied in institutions admitting patients with severe TBI in Japan.

Supporting information

Acknowledgments

The authors are grateful to all staff and participants of the JNTDB Project 2009 for their contributions. The 22 participating institutions were: Department of Neurosurgery, Kurume University School of Medicine, Fukuoka; Department of Neurosurgery, Japanese Red Cross Kumamoto Hospital, Kumamoto; Department of Neurosurgery, National Disaster Medical Center, Tokyo; Department of Neurosurgery, Saiseikai Shiga Hospital, Shiga; Department of Emergency and Critical Care Medicine, Showa University Hospital, Tokyo; Department of Neurosurgery, St. Marianna University School of Medicine, Kanagawa; Department of Neurosurgery and Department of Emergency and Critical Care Medicine, Sendai City Hospital, Miyagi; Department of Neurosurgery, Chiba Emergency Medical Center, Chiba; Department of Neurosurgery, Nihon University Itabashi Hospital, Tokyo; Department of Neurosurgery, Kyoto Kujo Hospital, Kyoto; Department of Neurosurgery and Emergency Medical Center, Kagawa University Hospital, Kagawa; Department of Neurosurgery, Yamaguchi University Hospital, Yamaguchi; Department of Emergency Medicine, Teikyo University Hospital, Tokyo; Department of Neurosurgery, Tokai University Hospital, Kanagawa; Department of Neurosurgery, Tokyo Medical University Ibaraki Medical Center, Ibaraki; Department of Emergency Medicine, Jikei University School of Medicine, Kashiwa Hospital, Chiba; Department of Emergency Medicine, Nippon Medical School Hospital, Tokyo; Department of Neurosurgery, Japanese Red Cross Musashino Hospital, Tokyo; and Department of Neurosurgery, Chiba Cerebral and Cardiovascular Center, Chiba.; Department of Emergency Medicine, Sapporo Medical University, Hokkaido; Department of Neurosurgery, Aomori Prefectural Central Hospital, Aomori; and Department of Acute Critical Care Medicine, Juntendo University Shizuoka Hospital, Shizuoka. We also thank Audrey Holmes, MA, from Edanz Group (www.edanzediting.com/ac) for editing a draft of this manuscript.