http://orcid.org/0000-0002-4279-6532
Figures
Abstract
The Posttraumatic stress disorder checklist (PCL) is the most widely used questionnaire to screen for symptoms of posttraumatic stress disorder (PTSD), based on the Diagnostic and Statistical manual of Mental disorders (DSM-IV) criteria. In the latest edition of the DSM (DSM-5), the criteria for PTSD were revised leading to the development of the PCL-5. So far, there is no validated Dutch version of the PCL-5. The aim of this study is to determine psychometric characteristics of the Dutch translation and linguistic validation of the PCL-5 and to evaluate internal consistency, criterion and structural validity. In a population of 495 civilian, traumatic brain injury patients, the PCL-5, the Generalized anxiety disorder questionnaire and the Depression scale of the Patient health questionnaire were administered. The PCL-5 was translated in Dutch following a strict procedure of linguistic validation and cognitive debriefing. Results show an excellent internal consistency and high criterion validity. Confirmatory factor analysis demonstrated a good fit for the four-factor DSM-5 model, but a superior fit for the six-factor Anhedonia model and the seven-factor Hybrid model, similar to the English version of the PCL-5. Preliminary validation of the Dutch translation of the PCL-5 was proven to be psychometrically sound and can be used for clinical and academic purposes, specifically for TBI patients. Future research should examine concurrent and discriminant validity for the Dutch translation in broader populations at risk for PTSD, and include a structured interview to evaluate diagnostic utility.
Citation: Van Praag DLG, Fardzadeh HE, Covic A, Maas AIR, von Steinbüchel N (2020) Preliminary validation of the Dutch version of the Posttraumatic stress disorder checklist for DSM-5 (PCL-5) after traumatic brain injury in a civilian population. PLoS ONE 15(4): e0231857. https://doi.org/10.1371/journal.pone.0231857
Editor: Thach Duc Tran, Monash University, AUSTRALIA
Received: June 6, 2019; Accepted: April 2, 2020; Published: April 20, 2020
Copyright: © 2020 Van Praag et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: CENTER-TBI is committed to data sharing, and in particular to responsible further use of the data. Hereto, we have a data sharing statement in place: https://www.center-tbi.eu/data/sharing The CENTER-TBI dataset is hugely complex, and the CENTER researchers wish to encourage correct and appropriate use of the data; this means that we encourage researchers to contact the CENTER-TBI team for any research plans and the Data Curation Team for any help in appropriate use of the data, including sharing of scripts. Requests for data access, can be submitted online: https://www.center-tbi.eu/data The complete Manual for data access is also available online: https://www.center-tbi.eu/files/SOP-Manual-DAPR-20181101.pdf.
Funding: This study was supported by the Framework 7 program of the European Union in the context of CENTER-TBI (grant number 602150). Additional funding was obtained from the Hannelore Kohl Stiftung (Germany), from OneMind (USA) and from Integra LifeSciences Corporation (USA). The funders did not play a role in the study design, data collection and analysis, decision to publish or preparation of the manuscript.
Competing interests: This study was supported by the Framework 7 program of the European Union in the context of CENTER-TBI (grant number 602150). Additional funding was obtained from the Hannelore Kohl Stiftung (Germany), from OneMind (USA) and from Integra LifeSciences Corporation (USA). Acknowledged (non-author) contributors LD, RAF, SD, VVT, VHW, VVE, and VJ are affiliated with icoMetrix NV, Leuven, Belgium. This does not alter our adherence to PLOS ONE's policies on materials or data sharing.
Introduction
Posttraumatic Stress Disorder (PTSD) is a mental disorder following a traumatic event, expressed by different symptoms such as persistent remembering or reliving the event, trying to avoid specific stressors, difficulty in falling asleep, irritability, hyper-vigilance and others [1]. The topic of PTSD has been widely investigated in patients following a traumatic brain injury (TBI) [2]. TBI forms a large public health and societal problem and over two million people are admitted to hospital each year in Europe [3]. Reported prevalence rates of PTSD following civilian TBI range from 0 to 50% [4–11]. There is no clear evidence for effects of the severity of TBI on the development of PTSD. For mild TBI, prevalence rates range from 4 to 34% [12–28], and for moderate to severe TBI from 0 to 25% [13,25,26,29–39]. In military settings, the incidence rate is higher and ranges from 33 to 65% [40,41]. Despite broad ranges of reported incidence rates of PTSD after TBI, PTSD is recognized as an important cause of disability following TBI [42].
The Posttraumatic stress disorder checklist (PCL) is the most widely used questionnaire to screen for symptoms of PTSD in research and clinical contexts [43]. The original 17-item version of the PCL questionnaire was based on the Diagnostic and Statistical manual of Mental disorders–fourth edition (DSM-IV) criteria [44]. The symptoms can be categorized into three DSM-clusters: Intrusive recollections, Avoidance/Numbing and Arousal. The majority of structural validity studies however, support a four factor model [45,46].
In the fifth edition of the DSM, diagnostic criteria for PTSD were revised, and a new category of Trauma- and Stressor-Related disorders was integrated [1]. PTSD can be diagnosed following exposure to a traumatic event, e.g. (threatened) death, or (threatened) serious injury, and now includes four factors (Intrusion, Avoidance, Negative alterations in cognitions and mood, and Arousal) [45,46]. The most prominent change concerns anhedonic/dysphoric symptoms which are reflected by negative cognitions and mood states. Examples of negative appraisals are ‘nobody can be trusted’ or ‘nothing good can happen to me’. Negative emotions can occur and include anger, shame, guilt, or the inability to experience positive emotions. Symptoms need to be present for more than one month, need to cause significant symptom-related distress or functional impairment, and cannot be initiated by medication, substance or illness. In accordance to the DSM-5, the PCL underwent several changes and is now composed of 20 items (PCL-5) [47].
The PCL-5 was developed and published in English and has been translated in French [48], in German [49], in Swedish [50], in Chinese [51,52], in Brazilian [53], in Turkish [54], in Arab and Kurdish [55] and in Filipino [56]. However, these translations have not been validated in TBI. Up till now, there was no validated Dutch version of the PCL-5. This may be considered highly relevant as it confounds comparability of Dutch studies to those from other countries, and poses challenges to clinicians wishing to screen Dutch speaking patients after TBI. The objective of the present study is to determine psychometric characteristics of the Dutch translation of the PCL-5 and to evaluate the quality of the translation, the linguistic validation and the psychometrics. Internal consistency, criterion validity and structural validity will be investigated.
Materials and methods
Participants
Data for the current study were collected as sub-study within the European CENTER-TBI Core study (Collaborative European NeuroTrauma Effectiveness Research: www.center-tbi.eu), an observational prospective study that aims to better characterize TBI and identify the most effective clinical interventions for managing TBI (clinicaltrials.gov NCT02210221) [57]. The CENTER-TBI Core study collected data on 4509 patients from 65 hospitals across 19 countries over a 3 year period from dec 2014 to dec 2017. All patients with a clinical diagnosis of TBI, an indication for a CT-scan and presentation to the study centre within 24 hours of the injury were eligible for enrollment. Pre-existing severe neurological disorders were an exclusion criterion. Informed consent was obtained in all subjects according to local and national requirements.
On enrollment, patients were stratified by care pathway into three strata. The ER stratum included patients that visited the emergency department and were discharged the same day. The admission stratum included patients that were admitted to the ward and the ICU stratum included patients primarily admitted to the intensive care unit.
For the current study, we selected adult patients (≥ 16 years of age) from the Dutch speaking centers. The analyzed data came from seven clinics in the Netherlands and two clinics in Dutch speaking cities in Belgium (Antwerp University Hospital and University Hospital Leuven).
Procedure
The CENTER-TBI study aimed to follow patients up to two years post injury. At different time points, self-report questionnaires and neuropsychological assessments were performed depending on the stratum of inclusion. All included patients were asked to complete the PCL-5, the Patient health questionnaire-9 (PHQ-9) [58] and the Generalized anxiety disorder questionnaire-7 (GAD-7) [59] at six months post injury.
The medical assessments of the study were performed by physicians and other assessment by research personnel, who were trained to collect and accurately enter data in the electronic case report form.
Participants were invited to revisit the hospital for an interview at the scheduled follow up time points and were examined by a research nurse or a neuropsychologist. Research nurses or neuropsychologists collecting outcome data were extensively trained in collecting psychological data and administering cognitive tests. If a visit was not possible or the patient refused to perform the neuropsychological assessments, participants received the PCL-5 instrument among other questionnaires by mail with a pre-stamped return envelope. In case the patient did not return the questionnaire(s) or did not respond to all questions, they were contacted to obtain the missing data. All data entered in the electronic case report form were de-identified and stored in a secure database.
Ethical approval
The CENTER-TBI study (EC grant 602150) has been conducted in accordance with all relevant laws of the EU if directly applicable or of direct effect and all relevant laws of the country where the recruiting sites were located, including but not limited to, the relevant privacy and data protection laws and regulations (the “Privacy Law”), the relevant laws and regulations on the use of human materials, and all relevant guidance relating to clinical studies from time to time in force including, but not limited to, the ICH Harmonized Tripartite Guideline for Good Clinical Practice (CPMP/ICH/135/95) (“ICH GCP”) and the World Medical Association Declaration of Helsinki entitles “Ethical Principles for Medical Research Involving Human Subjects”. Informed Consent was obtained for all patients recruited in the Core Dataset of CENTER-TBI and documented in the electronic case report form. Ethical approval was obtained for each recruiting sites. The list of sites, Ethical Committees, approval numbers and approval dates can be found on the website: https://www.center-tbi.eu/project/ethical-approval.
Measures
Self-report questionnaires administered included the PCL-5 instrument, the PHQ-9, GAD-7, and a socio-demographic questionnaire.
PTSD checklist for DSM-5 (PCL-5).
The PTSD Checklist for DSM-5 (PCL-5) is a self-report measure to screen for PTSD, to determine symptom severity of PTSD, to monitor symptom change after treatment or to make a provisional PTSD diagnosis [47]. The PCL-5 includes 20 items that reflect the DSM-5 diagnostic criteria of PTSD. Patients are asked how much they have been bothered by each item over the past month. Items are scored on a Likert scale ranging from 0 to 4, where higher scores indicate more pronounced PTSD symptoms. The sum of scores can range from 0 to 80. A cut-off score of 31 was suggested to best screen for PTSD with a specificity of .95, a sensitivity of .85 and an efficiency of .95 [48]. Related to the DSM-5, the PCL-5 can also be interpreted using the Symptom Cluster Method to screen for PTSD and provide a provisional diagnosis. An item with a score of 2 or higher is considered clinically relevant. A diagnosis of PTSD requires at least one intrusion item, at least one avoidance item, two or more negative alterations in cognitions and mood, and two or more arousal symptoms. A formal diagnosis however, needs a more thorough evaluation for example by using the Clinician-Administered PTSD Scale (CAPS) [49].
Patient health questionnaire (PHQ-9).
The Patient health questionnaire (PHQ-9) is a self-report questionnaire to screen for depression [58]. The PHQ-9 consists of nine items, related to the DSM-IV criteria, asking how often the patient was bothered by the symptom. Items can be scored from 0 ‘Not at all’ to 3 ‘Almost each day’, resulting in a total score of 0 to 27. A high score suggests a depressive disorder. The PHQ-9 shows a good agreement with assessments by mental health professionals (K = .65) [60]. Internal consistency was high and produced Cronbach alphas from .86 to .89 [58]. The Dutch translation of the PHQ-9 has previously been validated, showing similar internal consistency (C’s α = 0.83) and interrater reliability (K = 0.81), as the English version [61,62].
Generalized anxiety disorder questionnaire (GAD-7).
The Generalized anxiety disorder questionnaire (GAD-7) is a self-administered patient questionnaire, used as a screening tool and severity measure for anxiety [59]. Patients are asked whether they are bothered by one of the seven items, related to DSM-IV Generalized anxiety disorder. The items are scored on a scale from 0 meaning ‘Not at all’ to 3 indicating ‘Almost each day’. A high total score refers to a possible anxiety disorder with a maximum score of 21. Internal consistency is excellent (α = .89) and confirmatory factor analysis produced a one-dimensional structure of the GAD-7 [63]. A cut-off score of 7 or higher yielded a sensitivity of .73 and a specificity of .67 [64]. The Dutch version of the GAD-7 has shown good reliability (C’s α = 0.86) and good convergent validity (r = 0.82) [65,66].
Sociodemographic questionnaire.
Patients’ socio-demographic data including gender, age, race, marital and family status as well as patients’ level of education and parents’ level of education was gathered. Furthermore, patients were asked about pre-injury employment status, previous history of psychiatric disorders, and also the cause of injury.
Translation and linguistic validation
To ensure the linguistic validity of the translated instrument, a strict procedure was specified prior to the translation of PCL-5 from English to Dutch, including linguistic validation and cognitive debriefing. First, two independent forward translations of PCL-5 were created by native Dutch speakers in the medical academic field. Reviewing the translations, the consensus version was developed by combining the translations. Furthermore, the consensus version was reviewed and edited by a psychologist collaborating face-to-face with a translator to ensure the conceptual equivalence of the translated version. Next, the consensus version was back-translated to English by a native English speaker and after comparing it to the source instrument, it was approved by the linguistic validation team at UMG.
To carry out the cognitive debriefing, three healthy volunteers and three patients with PTSD were examined using the translated questionnaire and clinical interview. Further adjustments were made by two professional translators after reviewing the results of the cognitive debriefing, which were approved by an expert panel. Final harmonization was applied by five language coordinators of the UMG linguistic validation team at the UMG and finally approved by the study board.
Statistical analysis
Sample characteristics were described for gender, age, level of education, injury type, racial background and prior mental health status. In order to evaluate the validity and reliability of the PCL-5 translation, its properties were examined and reported both at item-level and scale level.
Reliability.
For the item-level analysis, items distribution and skewness are reported. For scale-level analyses, internal consistency of the scale was examined using Cronbach’s alpha and split-half reliability. Furthermore, item-total correlations and Cronbach’s alpha—if item omitted–is reported.
Criterion validity.
We examined the criterion validity of the PCL-5 instrument by reporting its correlations with two related instruments, GAD-7 and PHQ-9. Previous research has reported high correlations ranging from 0.70 to 0.77 between PTSD and depression symptoms using PCL-5 and PHQ-9 instruments [56,67,68]. Similar correlations ranging from 0.61 to 0.67 have been reported between PTSD and anxiety symptoms by using PCL-5 and GAD-7 instruments [56,67,68].
Structural validity.
The structural validity of the PCL-5 instrument was analyzed using confirmatory factor analysis (CFA). We applied the DSM-5 four-factor model, the six-factor Anhedonia model [54], and the seven-factor Hybrid model [69]. The Anhedonia model includes Intrusion, Avoidance, Negative affect, Anhedonia, Dysphoric arousal, and Anxious arousal factors. The seven-factor hybrid model is very similar to the anhedonia model and differs in only two items. Namely, it suggests an additional factor of Externalizing behavior by extracting two items from the Dysphoric arousal in the six-factor model.
In all of the CFA models, items were specified to a single factor only and the latent variables were specified to correlate with one another. A number of criteria were considered for evaluating the fitness of the factor analyses. In addition to Chi-Square goodness of fit, the Root Mean Square Error of Approximation (RMSEA) and its confidence interval as well as Standardized Root Mean Square Residual (SRMR), Comparative Fit Index (CFI) and Tucker Lewis Index (TLI) were investigated. The CFA analyses were carried out using R version 3.4.1 and the lavaan package [70] version 0.5.23 with WLSMV estimator, because the data was ordinal. As a result, the Akaike information criterion (AIC) and Bayesian information criterion (BIC) that are only available for maximum likelihood estimator were not computed. Based on the recommended cut-off suggested in the literature, the CFA model fit was considered adequate for CFI and TLI values above 0.95, RMSEA value less than 0.06, and SRMR value below 0.08 [71].
Results
Sample characteristics
Of the 815 TBI patients admitted to the Dutch-speaking hospitals, 320 subjects (99 females and 204 males) did not complete the questionnaires at the 6-months follow up after the injury and thus were dropped from the analysis. The analysis included 495 subjects, where 419 subjects were included in the Netherlands and 76 subjects in Antwerp and Leuven, two Dutch speaking cities in Belgium. From the participants, 186 were female (median age = 58, IQR = 31) and 309 were male (median age = 50, IQR = 37). The sample characteristics are summarized in Table 1.
As shown in Table 1, the majority of the subjects were caucasian and over 45 years of age, and 97.4% of the subjects had a closed head injury.
While the PCL-5 score can range from 0 to 80, the score ranged from 0 to 72 in our study population. More importantly, the majority of the subjects scored low on the PCL-5 scale (median = 7, mean = 11.25, SD = 13.06). Therefore, the distribution of the PCL-5 score was right-skewed (skew = 1.82). Only 45 patients (9%) had a total score of 31 or higher, suggested as best cut-off score to screen for PTSD [48]. As expected, the PCL-5 score also varied based on the demographics of the study population. For example, the PCL-5 score was found to be negatively correlated with age (r = -0.20) and level of education (r = -0.12) and positively correlated with prior history of mental health problems (r = 0.21). However, the correlation of the PCL-5 score with gender was near 0.
Reliability
The translated scale shows an excellent internal consistency of 0.93. Table 2 summarizes the Cronbach's Alpha for the total score and each of the subscales of the PCL-5 instrument. As shown below, all of the subscales have a good internal consistency. In addition, using split-half method on all of the items reveals a high reliability of 0.96 for the total scale.
Further details about the scale reliability are provided in Table 3. The table shows the mean, SD, item-total correlation, Cronbach's alpha of the scale if each item is dropped, as well as Skewness and Kurtosis of each translated PCL-5 item. The low mean and the positive skewness values of each item are in-line with the total score of the scale, indicating that the TBI patients scored low on PCL-5 instrument. Nevertheless, a high positive item-total correlation and a constant Cronbach's alpha of 0.93 provide further information about the scale's reliability.
Criterion validity
For the GAD-7 and PHQ-9 instruments, the total score ranged from 0 to 21 (maximum possible score of 24) and 27 (maximum possible score of 27) respectively. However, the majority of the subjects scored low on these scales and the median score was 2 for GAD-7 and 3 for PHQ-9.
Both male (median = 6, IQR = 15) and female (median = 8, IQR = 13) participants had similar PTSD symptom scores, the median scores for the total PCL-5 score (Mann-Whitney U-test: χ2 = 1.473, p = 0.225) and subscales intrusion, avoidance, negative alterations in cognition/mood, arousal (resp. χ2 = 2.027, p = 0.154, χ2 = 0.720, p = 0.396, χ2 = 0.186, p = 0.666, χ2 = 0.781, p = 0.377) did not differ significantly for males and females. However, females mean score was significantly higher for GAD-7 (χ2 = 8.637, p = 0.003) and for the PHQ-9 (χ2 = 5.115, p = 0.024).
Based on the previous findings, high correlations were expected between scores of PTSD and depression [48,50] and PTSD and anxiety symptoms [54]. Table 4 shows the correlation matrix between these variables, which were all statistically significant.
Structural validity
Multiple confirmatory factor analyses were applied to examine the factor structure of the PCL-5 questionnaire. The factor analysis for the four-factor DSM-5 model, reveals acceptable CFI, SRMR and RMSEA, showing an adequate fit for the model. However, the Chi-Square statistics indicate that the model fit is not ideal (χ2 = 246.49, p<0.000) [72]. The six-factor Anhedonia model (χ2 = 152.97, p< 0.531) and the seven-factor Hybrid model (χ2 = 138.631, p<0.718), both show a superior goodness-of-fit for the model compared to the four-factor DSM-5 model (Table 5). The Scaled Chi-Square Difference Test revealed that with p<0.001, the six-factor Ahnhedonia and the seven-factor Hybrid models fit significantly better than the four-factor DSM-5 and six-factor Anhedonia models respectively.
Table 6 shows the standardized estimates of the models. The correlations between the latent variables within each model were very high and range from 0.77 to 0.94 for the four-factor DSM-5 model (S1 Table), from 0.70 to 0.93 for the six-factor Anhedonia model (S2 Table), and from 0.70 to 0.98 for the seven-factor Hybrid model (S3 Table).
Discussion
Properties of the PCL-5
In this study, we performed a preliminary validation of the Dutch translation of the PCL-5 instrument for a civilian sample of 495 patients with TBI from the Netherlands and Belgium.The Dutch translation was proven to be psychometrically sound as it demonstrated excellent internal consistency and reliability and high criterion validity. This has important implications for future, international research on PTSD in a TBI population, as this version is now available in Dutch. Confirmatory Factor Analysis showed a good fit for the most frequently tested models; four-factor DSM-5 model, the six-factor Anhedonia model and the seven-factor Hybrid model. The latter are considered to be a superior fit compared to the DSM-5 model. The results are similar to those of the original, English version [48,73].
Overall, the patients reported low anxiety, low depression, and low PTSD symptoms causing the score distribution to be positively skewed. However, these scores were very close to the predicted correlations, providing support for the criterion validity of the instrument. The correlation of PCL-5 with anxiety and depression was found to be 0.71 and 0.72, respectively. These correlations are similar to those reported by Ashbaugh and colleagues (2016) using the English and French version of the PCL-5 and the Impact of Event Scale–Revised and the Center for Epidemiological Studies—Depression Scale, supporting convergent and divergent validity (S4 Table) [48,74,75]. Hall and collegeagues translated the PCL-5 to Filipino and used the same questionnaires to determine criterion validity; PHQ-9 and GAD-7 and found similar correlations (resp. 0.71 and 0.61) [56]. The correlations between the Turkish version of the PCL-5 and the Beck Depression Inventory [76] and Beck Anxiety Inventory [54,77], and the Arab/Kurdish version of the PCL-5 and the Depression Hopkins Symptom checklist [55,78], and the Swedish version of the PCL-5 and the Montgomery-Asberg Depression Rating Scale [50,79] range from 0.60 to 0.81. Nine percent of the patients showed a total score of 31 or higher which suggests a possible diagnosis of PSTD. This finding is in line with reported prevalence rates of studies measuring PTSD using the PCL in civilian TBI populations (0–22.7%) [17,22,25,26,29,32,80–82].
The PCL-5 was not associated with female gender as was found in previous studies [83]. Previous findings suggesting older age is a risk factor to develop PTSD is not supported, older patients show a slightly lower total score for PTSD symptoms [84]. Patients with a lower education level are more likely to develop PTSD symptoms. A history of mental health problems was associated with higher PTSD symptoms, and confirms previous findings [30,84].
Moreover, the internal consistency of the Dutch version of the PCL-5 is within the range of the internal consistency of the English instrument (α = 0.93–0.95) and other versions of the PCL-5 [48,73]. Psychometric evaluations of the French, German, Swedish, Filipino and Turkish translations show excellent internal consistency for the total score (α = 0.90–0.95). The Cronbach’s alpha of the subscales of the PCL-5 were very similar to those reported for the French (α = 0.79 to 0.87), the English (α = 0.81 to 0.90), the German (α = 0.79 to 0.89) and the Turkish version (α = 0.78 to 0.87) ranging from 0.79 to 0.90 [48–50,54].
The current study also confirms the structural validity of the PCL-5 in a Dutch, civilian TBI sample. The results of the CFA analysis show that the four-factor DSM-5 PTSD model as well as the six-factor Anhedonia and seven-factor Hybrid models, all provide an adequate fit. Apart from Chi-Square, which was significant for the DSM-5 model, all the indexes were within the defined cut-offs for the defined adequate fit. Moreover, the analysis also confirms previous findings of the French and English version [48], and the Chinese version [51,52] that the six-factor Anhedonia model and the seven-factor Hybrid model both are superior to the four-factor DSM-5 model [48].
Limitations
The current manuscript made use of data of a larger study and the data was not specifically collected for validating the new instrument. As a result, some of the routine validation and reliability procedures such as convergent and divergent validity were not performed, since they require additional resources, i.e. including more questionnaires. Test-retest reliability test was not performed, since only four patients completed the retest PCL-5 ten days after the first PCL-5. The Dutch version of the PCL-5 was not validated against a structured interview for PTSD to evaluate diagnostic utility. The PCL-5 is a checklist and therefore should not be used as diagnostic tool. However, the reported reliability measures, the magnitudes of the correlations and the factor loadings, as well as the results of the confirmatory factor analysis, all provide evidence for the reliability and the validity of the instrument for the Dutch translation of the PCL-5 instrument.
There was a relatively high number of patients who did not return the questionnaires or attend the follow-up visit. However, compared to other observational studies, this is a reasonable response rate in health research [85].
Research in patients with TBI implies a certain risk for spurious results because of cognitive difficulties. Concentration problems, impulsivity or the tendency to underestimate their functional problems may influence the PCL-5 total score [6].
Conclusions and outlook
The psychometric properties of the Dutch translation of the PCL-5 show solid reliability and criterion validity in a population of civilian TBI patients. The PCL-5 now can be used for clinical purposes, particularly in Dutch speaking TBI patients, to quantify PTSD symptom severity and to screen for PTSD.
Future research should complete psychometric evaluation by examining concurrent and discriminant validity for the Dutch translation in broader populations at risk for PTSD, and should include a structured interview as a measure for PTSD to evaluate diagnostic utility. Validating against a diagnostic tool for PTSD will allow determination of the best cut-off score for the Dutch version of the PCL-5 in a civilian TBI population. In addition, longitudinal research should evaluate the sensitivity to change over time. Validating the PCL-5 in broader populations is relevant to increase generalizability. A fully validated PTSD instrument will allow researchers to reliably estimate the prevalence of PTSD, compare Dutch samples to those from other regions, and facilitate international collaborative studies.
Supporting information
S1 Table. Latent variable correlations in four-factor DSM-5 model.
https://doi.org/10.1371/journal.pone.0231857.s001
(PDF)
S2 Table. Latent variable correlations in six-factor Anhedonia model.
https://doi.org/10.1371/journal.pone.0231857.s002
(PDF)
S3 Table. Latent variable correlations in seven-factor Hybrid model.
https://doi.org/10.1371/journal.pone.0231857.s003
(PDF)
S4 Table. Correlations of depression and anxiety instruments with PCL-5.
https://doi.org/10.1371/journal.pone.0231857.s004
(PDF)
Acknowledgments
Data used in preparation of this manuscript were obtained in the context of CENTER-TBI, a large collaborative project. The manuscript is submitted on behalf of the following CENTER-TBI investigators;
Principal Investigators and contact information: Professor A.I. Maas: Andrew.Maas@azu.be1; Professor D. Menon: dkm31@wbic.cam.ac.uk3.
Ardon Hilko4, Bartels Ronald5, Carpenter K6, Covic Amra2, Cnossen Maryse7, De Keyser Véronique1, De Ruiter Godard C.W23, Depreitere Bart8, Dippel Diederik9, Engemann Heiko2, Foks Kelly9, Geleijns Karin6, Haagsma Juanita A.7, Haitsma Iain10, Hoedemaekers Astrid11, Jacobs Bram12, Janssens Koen1, Kalala Jean-Pierre13, Ketharanathan Naomi6, Kompanje Erwin14, Lecky Fiona15, Lingsma Hester7, Loeckx Dirk16, Luijten-Arts Chantal11, Maas Andrew I.R.1, Menon David3, Menovsky Tomas1, Schoonman Guus20, Oldenbeuving Annemarie17, Parizel Paul M.18, Peul Wilco19, Polinder Suzanne7, Pullens Pim18, Roks Gerwin20, Ruiz de Arcaute Felix16, Schipper Inger21, Sir Özcan22, Smakman Lidwien23, Smeets Dirk16, Steyerberg Ewout W.7, Tibboel Dick6, Vande Vyvere Thijs16, Van Der Jagt Mathieu25, Van Der Naalt Joukje12, Van Dijck Jeroen24, Van Hecke Wim16, Van Vlierberghe Eline16, Verheyden Jan16, Wildschut Eno6, Van Essen Thomas A.19, Van Praag Dominique1, Van Roost Dirk13, Vleggeert-Lankamp Carmen23, Volovici Victor10,Von Steinbüchel Nicole2.
1 Department of Neurosurgery, Antwerp University Hospital and University of Antwerp, Edegem, Belgium
2 Institute of Medical Psycholology and Medical Sociology, Universitätsmedizin Göttingen, Göttingen, Germany
3 Division of Anaesthesia, University of Cambridge, Addenbrooke’s Hospital, Cambridge, UK
4 Department of Neurosurgery, Elisabeth-Tweesteden Ziekenhuis, Tilburg, the Netherlands
5 Department of Neurosurgery, Radboud University Medical Center, The Netherlands
6 Intensive Care and Department of Pediatric Surgery, Erasmus Medical Center, Sophia Children’s Hospital, Rotterdam, The Netherlands
7 Department of Public Health, Erasmus Medical Center-University Medical Center, Rotterdam, The Netherlands
8 Department of Neurosurgery, University Hospitals Leuven, Leuven, Belgium
9 Department of Neurology, Erasmus MC, Rotterdam, the Netherlands
10 Department of Neurosurgery, Erasmus MC, Rotterdam, the Netherlands
11 Department of Intensive Care Medicine, Radboud University Medical Center, The Netherlands
12 Department of Neurology, University Medical Center Groningen, Groningen, Netherlands
13 Department of Neurosurgery, UZ Gent, Gent, Belgium
14 Department of Intensive Care and Department of Ethics and Philosophy of Medicine, Erasmus Medical Center, Rotterdam, The Netherlands
15 Emergency Medicine Research in Sheffield, Health Services Research Section, School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK
16 icoMetrix NV, Leuven, Belgium
17 Department of Intensive Care, Elisabeth-Tweesteden Ziekenhuis, Tilburg, the Netherlands
18 Department of Radiology, Antwerp University Hospital and University of Antwerp, Edegem, Belgium
19 Dept. of Neurosurgery, Leiden University Medical Center, Leiden, The Netherlands and Dept. of Neurosurgery, Medical Center Haaglanden, The Hague, The Netherlands
20 Department of Neurology, Elisabeth-TweeSteden Ziekenhuis, Tilburg, the Netherlands
21 Department of Traumasurgery, Leiden University Medical Center, Leiden, The Netherlands
22 Department of Emergency Care Medicine, Radboud University Medical Center, The Netherlands
23 Neurosurgical Cooperative Holland, Department of Neurosurgery, Leiden University Medical Center and Medical Center Haaglanden, Leiden and The Hague, The Netherlands
24 Department of Neurosurgery, The HAGA Hospital, The Hague, The Netherlands
25 Department of Intensive Care, Erasmus MC, Rotterdam, the Netherlands
References
- 1.
American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Fifth Edition. Washington, DC: American Psychiatric Association; 2013.
- 2. Van Praag D.L.G., Cnossen M.C., Polinder S., Wilson L., & Maas A.I.R. Post-traumatic stress disorder after civilian traumatic brain injury: A systematic review and meta-analysis of prevalence rates. J Neurotrauma. 2019 Dec 1; 36(23):3220–3232. pmid:31238819
- 3. Majdan M, Plancikova D, Brazinova A, Rusnak M, Nieboer D, Feigin V, et al. Epidemiology of traumatic brain injuries in Europe: a cross-sectional analysis. Lancet Public Health. 2016 Dec;1(2):e76–e83. pmid:29253420
- 4. Riggio S. Traumatic brain injury and its neurobehavioral sequelae. Neurol Clin. 2011 Febr;29(1):35–47. pmid:21172569
- 5. Carlson KF, Kehle SM, Meis LA, Greer N, Macdonald R, Rutks I, et al. Prevalence, assessment, and treatment of mild traumatic brain injury and posttraumatic stress disorder: a systematic review of the evidence. J Head Trauma Rehabil. 2011 Mar-Apr;26(2):103–15. pmid:20631631
- 6. Moore EL, Terryberry-Spohr L, Hope DA. Mild traumatic brain injury and anxiety sequelae: a review of the literature. Brain inj. 2006 Febr;20(2):117–32. pmid:16421060
- 7. Bahraini NH, Breshears RE, Hernandez TD, Schneider AL, Forster JE, Brenner LA. Traumatic brain injury and posttraumatic stress disorder. Psychiatr Clin North Am. 2014 Mar;37(1):55–75. pmid:24529423
- 8. Bryant R. Post-traumatic stress disorder vs traumatic brain injury. Dialogues Clin Neurosci. 2011;13(3):251–62. pmid:22034252
- 9. Scholten AC, Haagsma JA, Cnossen MC, Olff M, van Beeck EF, Polinder S. Prevalence of and risk factors for anxiety and depressive disorders after traumatic brain injury: a systematic review. J Neurotrauma. 2016 Nov 15;33(22):1969–94. pmid:26729611
- 10. Bryant RA, Harvey AG. The influence of traumatic brain injury on acute stress disorder and post-traumatic stress disorder following motor vehicle accidents. Brain inj. 1999 Jan 13;13(1):15–22. pmid:9972438
- 11. Hickling EJ, Blanchard EB. Post-traumatic stress disorder and motor vehicle accidents. J Anxiety Disord. 1992 July;6(3):285–91.
- 12. Ahman S, Saveman B, Styrke J, Björnstig U, Stalnacke B. Long-term follow-up of patients with mild traumatic brain injury: a mixed-methods study. J Rehabil Med. 2013 Sep;45:758–64. pmid:24002311
- 13. Alexander MP. Neuropsychiatric correlates of persistent postconcussive syndrome. J Head Trauma Rehabil. 1992 June;7(2):60–9.
- 14. Bryant RA, Harvey A. Postconcussive symptoms and posttraumatic stress disorder after mild traumatic brain injury. J Nerv Ment Dis. 1999 May;187(5):302–5. pmid:10348085
- 15. Bryant RA, Creamer M, O’Donnell M, Silove D, Clark CR, McFarlane AC. Post-traumatic amnesia and the nature of post-traumatic stress disorder after mild traumatic brain injury. J Int Neuropsychol Soc. 2009 Nov;15(6):862–7. pmid:19703323
- 16. Bryant RA, O’Donnell ML, Creamer M, McFarlane AC, Clark CR, Silove D. The psychiatric sequelae of traumatic injury. Am J Psychiatry 2010 Mar;167(3):312–20. pmid:20048022
- 17. Choi MS, Seo SJ, Oh CH, Kim S, Cho JM. Incidence of post-traumatic stress disorder after a mild traumatic brain injury: preliminary investigation using the Brief Neuropsychological Screening Test. J Korean Neurosurg Soc. 2014 Apr;55(4):190–4. pmid:25024821
- 18. Creamer M, O’Donnell ML, Pattison P. Amnesia, traumatic brain injury, and posttraumatic stress disorder: a methodological inquiry. Behav Res Ther. 2005 Oct;43(10):1383–9. pmid:16086988
- 19. Dischinger P, Read K, Kerns T, Ho S, Kufera J, Burch C, et al. Causes and outcomes of mild traumatic brain injury: an analysis of CIREN data. Annu Proc Assoc Adv Automot Med. 2003;47:577–89. pmid:12941252
- 20. Haagsma JA, Scholten AC, Andriessen TM, Vos PE, Van Beeck EF, Polinder S. Impact of depression and post-traumatic stress disorder on functional outcome and health-related quality of life of patients with mild traumatic brain injury. J Neurotrauma. 2015 Jun 1;32(11):853–62. pmid:25320845
- 21. Harvey AG, Bryant RA. Two-year prospective evaluation of the relationship between acute stress disorder and posttraumatic stress disorder following mild traumatic brain injury. Am J Psychiatry. 2000 Apr;157(4):626–8. pmid:10739425
- 22. Hoffman JM, Dikmen S, Temkin N, Bell KR. Development of posttraumatic stress disorder after mild traumatic brain injury. Arch Phys Med Rehabil. 2012 Feb;93(2):287–92. pmid:22289239
- 23. Kjeldgaard D, Forchhammer H, Teasdale T, Jensen RH. Chronic post-traumatic headache after mild head injury: a descriptive study. Cephalalgia. 2014 Mar;34(3):191–200. pmid:24045573
- 24. Lagarde E, Salmi L-R, Holm LW, Contrand B, Masson F, Ribéreau-Gayon R, et al. Association of Symptoms following Mild Traumatic Brain Injury with Posttraumatic Stress Disorder vs Postconcussion syndrome. JAMA Psychiatry. 2014 Sep;71(9):1032–40. pmid:25029015
- 25. Levin HS, Brown SA, Song JX, McCauley SR, Boake C, Contant CF, et al. Depression and posttraumatic stress disorder at three months after mild to moderate traumatic brain injury. J Clin Exp Neuropsychol. 2001 Dec;23(6):754–69. pmid:11910542
- 26. McCauley SR, Boake C, Levin HS, Contant CF, Song JX. Postconcussional disorder following mild to moderate traumatic brain injury: anxiety, depression, and social support as risk factors and comorbidities. J Clin Exp Neuropsychol. 2001 Dec;23(6):792–808. pmid:11910545
- 27. Meares S, Shores A, Taylor AJ, Batchelor J, Bryant RA. The prospective course of postconcussion syndrome: the role of mild traumatic brain injury. Neuropsychol. 2011 Jul;25(4):454–65.
- 28. Powell TJ, Collin C, Sutton K. A follow-up study of patients hospitalized after minor head injury. Disabill Rehabil. 1996 May;18(5):231–7.
- 29. Zatzick DF, Rivara FP, Jurkovich GJ, Hoge CW, Wang J, Fan MY, et al. Multisite investigation of traumatic brain injuries, posttraumatic stress disorder, and self-reported health and cognitive impairments. Arch Gen Psychiatry. 2010 Dec;67(12):1291–1300. pmid:21135329
- 30. Alway Y, Gould KR, McKay A, Johnston L, Ponsford J. The evolution of Post-Traumatic Stress Disorder following Moderate-to-Severe Traumatic Brain Injury. J Neurotrauma. 2016 May 1;33(9):825–31. pmid:26176500
- 31. Alway Y, McKay A, Gould KR, Johnston L, Ponsford J. Factors associated with posttraumatic stress disorder following moderate to severe traumatic brain injury: a prospective study. Depress Anxiety. 2016 Jan;33(1):19–26. pmid:26219232
- 32. Bombardier CH, Fann JR, Temkin N, Esselman PC, Pelzer E, Keough M, et al. Posttraumatic stress disorder symptoms during the first six months after traumatic brain injury. J Neuropsychiatry Clin Neurosci. 2006 Fall;18(4):501–8. pmid:17135375
- 33. Gould KR, Ponsford JL, Johnston L, Schönberger M. The nature, frequency and course of psychiatric disorders in the first year after traumatic brain injury: a prospective study. Psychol Med. 2011 Oct;41(10):2099–109. pmid:21477420
- 34. Baranyi A, Leithgob O, Kreiner B, Tanzer K, Ehrlich G, Hofer HP, et al. Relationship between posttraumatic stress disorder, quality of life, social support, and affective and dissociative status in severely injured accident victims 12 months after trauma. Psychosomatics 2010 May;51(3):237–47. pmid:20484722
- 35. Bryant RA, Marosszeky JE, Crooks J, Gurka JA. Elevated resting heart rate as a predictor of Posttraumatic Stress Disorder after severe traumatic brain injury. Psychosom Med. 2004 Sep;66(5):760–1. pmid:15385703
- 36. Hoofien D, Gilboa A, Vakil A, Donovick PJ. Traumatic Brain Injury (TBI) 10–20 years later: a comprehensive outcome study of psychiatric symptomatology, cognitive abilities and psychosocial functioning. Brain inj. 2001 Mar;15(3):189–209. pmid:11260769
- 37. Reid LM, McMillan TM, Harrison AG. PTSD, attention bias, and heart rate after severe brain injury. J Neuropsychiatry Clin Neurosci. 2011 Fall;23(4):454–6. pmid:22231318
- 38. Sumpter RE, McMillan TM. Misdiagnosis of post-traumatic stress disorder following severe traumatic brain injury. Br J Psychiatry. 2005 May;186:423–6. pmid:15863748
- 39. Williams WH, Evans JJ, Wilson BA, Needham P. Prevalence of post-traumatic stress disorder symptoms after severe traumatic brain injury in a representative community sample. Brain inj. 2002 Aug;16(8):673–9. pmid:12167192
- 40. Ommaya AK, Salazar AM, Dannenberg AL, Ommaya AK, Chervinsky AB, Schwab K. Outcome after traumatic brain injury in the U.S. military medical system. J Trauma. 1996 Dec;41(6):972–5. pmid:8970548
- 41. Pietrzak RH, Johnson DC, Goldstein MB, Malley JC, Southwick SM. Posttraumatic stress disorder mediates the relationship between mild traumatic brain injury and health and psychosocial functioning in veterans of Operations Enduring Freedom and Iraqi Freedom. J Nerv Ment Dis. 2009 Oct;197(10):748–53. pmid:19829203
- 42. Haarbauer-Krupa J, Taylor CA, Yue JK, Winkler EA, Pirracchio R,Cooper SR, et al. Screening for Post-Traumatic Stress Disorder in a Civilian Emergency Department Population with Traumatic Brain Injury. J Neurotrauma. 2017 Jan 1;34(1):50–8. pmid:26936513
- 43.
Weathers FL, Litz BT, Herman DS, Huska JA, Keane TM. The PTSD Checklist: reliability, validity and diagnostic utility. Annual meeting of the international society for traumatic stress studies. 1993. Paper presented at the Annual Convention of the International Society for Traumatic Stress Studies, San Antonio, TX.
- 44.
American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Fourth Edition. American Psychiatric Association; 1994.
- 45. Asmundson GF. Dimensionality of posttraumatic stress symptoms: a confirmatory factor analysis of DSM-IV symptom clusters and other symptom models. Behav Res Ther. 2000 Feb;38(2):203–14. pmid:10661004
- 46. Schinka JB. Confirmatory factor analysis of the PTSD checklist in elderly. J Trauma Stress. 2007 Jun;20(3):281–9. pmid:17597125
- 47. Weathers FL, Litz BT, Keane TM, Palmieri PA, Marx BP, Schnurr PP. The PTSD Checklist for DSM-5 (PCL-5). 2013. Scale available from the National Center for PTSD at www.ptsd.va.gov.
- 48. Ashbaugh A, Houle-Johnson S, Herbert C, El-Hage W, Brunet A. Psychometric validation of the English and French versions of the Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5). Plos One. 2016 Oct 10;11(10):e0161645. pmid:27723815
- 49. Krüger-Gottschalk A, Knaevelsrud C, Rau H, Dyer A, Schäfer I, Ehring T. The German version of the posttraumatic stress disorder checklist for DSM-5 (PCL-5): psychometric properties and diagnostic utility. BMC Psychiatry. 2017 Nov 28;17(1):379. pmid:29183285
- 50. Sveen J, Bondjers K, Willebrand M. Psychometric properties of the PTSD checklist for DSM-5: a pilot study. Eur J Psychotraumatol. 2016 Apr 19;7:30165. pmid:27098450
- 51. Wang L, Zhang L, Armour C, Cao C, Qing Y, Zhang J, et al. Assessing the underlying dimensionality of DSM-5 PTSD symptoms in Chinese adolescents surviving the 2008 Wenchuan earthquake. J Anxiety Disord. 2015 Apr;31,90–7. pmid:25768398
- 52. Liu P, Wang L, Cao C, Wang R, Zhang J, Zhang B, et al. The underlying dimensions of DSM-5 posttraumatic stress disorder symptoms in an epidemiological sample of Chinese eartquake survivors. J Anxiety Disord. 2014 May;28(4):345–51. pmid:24792723
- 53. Lima EP, Vasconcelos AG, Berger W, Kristensen CH, Nascimento ED, Mendlowicz MV. Cross-cultural adaptation of the Posttraumatic stress disorder checklist 5 (PCL-5) and Life events checklist-5 (LEC-5) for the Brazilian context. Trends Psychitry Psychother. 2016 Oct;38(4):207–15.
- 54. Boysan M, Ozdemir PG, Ozdemir O, Seliv Y, Ekrem Y, Nuray K. Psychometric properties of the Turkish version of the PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders, Fifth edition (PCL-5). Psychiatry Clin psychopharmacology. 2017 Jul 8;27(3):306–16.
- 55. Ibrahim H, Ertl V, Catani C, Ismail AA, Neuner F. The validity of posttraumatic stress disorder checklist for DSM-5 (PCL-5) as screening instrument with Kurdish and Arab displaced populations living in the Kurdistan region of Iraq. BMC Psychiatry. 2018 Aug 16;18(1):259. pmid:30115040
- 56. Hall BJ, Yip PSY, Garabiles MR, Lao CK, Chan EWW, Marx BP. Psychometric validation of the PTSD Checklist-5 among female Filipino migrant workers. Eur J Osychotraumatol. 2019 Feb 4;10(1):1571378.
- 57. Maas AI, Menon DK, Steyerberg EW, Citerio G, Lecky F, Manley GT, et al. Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI): a prospective longitudinal observational study. Neurosurgery. 2015 Jan;76(1):67–80. pmid:25525693
- 58. Kroenke K, Spitzer RL, Williams JBW. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001 Sep;16(9):606–13. pmid:11556941
- 59. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006 May 22;166(10):1092–7. pmid:16717171
- 60. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary care evaluation of mental disorders. Patient Health Questionnaire. JAMA. 1999 Nov 10;282(18):1737–44. pmid:10568646
- 61. Lamers F, Jonkers CCM, Bosma H, Penninx BWJH, Knottnerus JA, van Eijk JTM. Summed score of the Patient Health Questionnaire-9 was a reliable and valid method for depression screening in chronically ill elderly patients. J Clin Epidemiol. 2008 Jul;61(7):679–87. pmid:18538262
- 62. Persoons P, Luyckx K, Desloovere C, Vandenberghe J, Fischler B. Anxiety and mood disorders in otorhinolaryngology outpatients presenting with dizziness: validation of the self-administered PRIME-MD Patient Health Questionnaire and epidemiology. Gen Hosp Psychiatry. 2003 Sep-Oct;25(5):316–23. pmid:12972222
- 63. Löwe B, Decker O, Müller S, Brähler E, Schellberg D, Herzog W. et al. Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Med Care. 2008 Mar;46(3):266–74. pmid:18388841
- 64. Zhong QY, Gelaye B, Zaslavsky AM, Fann JR, Rondon MB, Sanchez SE, et al. Diagnostic validity of the generalized anxiety disorder-7 (GAD-7) among pregnant women. Plos one. 2015 Apr 27;10(4):e0125096. pmid:25915929
- 65. Donker T, van Straten A, Marks I, Cuijpers P. Quick and easy self-rating of Generalized Anxiety Disorder: Validity of the Dutch web-based GAD-7, GAD-2 and GAD-SI. Psychiatry Res. 2011 Jun 30;188(1):58–64. pmid:21339006
- 66. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983 Jun;67(6):361–70. pmid:6880820
- 67. Bovin M.J., Marx B.P., Weathers F.W., Gallagher M.W., Rodriguez P., Schnurr P.P., et al. Psychometric properties of the PTSD Checklist for Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (PCL-5) in veterans. Psychol Assess. 2016 Nov;28(11):1379–91. pmid:26653052
- 68. Phelps LF, Williams RM, Raichle KA, Turner AP, Ehde DM. The importance of cognitive processing to adjustmment in the 1st year following amputation. Rehab Psychol. 2008 Feb;53(1):28–38.
- 69. Armour C, Tsai J, Durham TA, Charak R, Biehn TL, Elhai JD, et al. Dimensional structure of DSM-5 posttraumatic stress symptoms: support for a hybrid Anhedonia and Externalizing Behaviors model. J Psychiatr Res. 2015 Feb;61:106–13. pmid:25479765
- 70. Rosseel Y. Lavaan: an R package for structural equation modeling. J statistical software. 2012;48(2):1–36.
- 71. Hu L, Bentler P. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling. 1999 Jan 1;6(1):1–55.
- 72. Cochran WG. The Chi-square test of goodness of fit. The Annals of mathematical statistics. 1952;23(3):315–45.
- 73. Blevins CA, Weathers FW, Davis MT, Witte TK, Domino JL. The posttraumatic stress disorder checklist for DSM-5 (PCL-5): development and initial psychometric evaluation. J Trauma Stress. 2015 Dec;28(6):489–98. pmid:26606250
- 74. Horowitz M, Wilner N, Alvarez W. Impact of event scale: a measure of subjective stress. Psychosom Med. 1979 May;41(3):209–18. pmid:472086
- 75. Radloff LS. The CES-D scale: A self-report depression scale for research in the general population. Appl Psychol Med. 1977 Jun 1;1(3):385–401.
- 76.
Beck AT, Rush J, Shaw BF, Emery G. Cognitive therapy of depression. New York (NY): Guildford Press; 1979.
- 77. Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psych. 1988 Dec;56(6):893–7.
- 78. Hesbacher PT, Rickels K, Morris RJ, Newman H, Rosenfeld H. Psychiatric illness in family practice. J Clin Psychiatry. 1980 Jan;41(1):6–10. pmid:7351399
- 79. Montgomery SA, Asberg M. A new depression scale designed to be sensitive to change. Br J Psychiatry. 1979 Apr;134:382–9. pmid:444788
- 80. Dahm J, Ponsford J. Comparison of long-term outcomes following traumatic injury: what is the unique experience for those with brain injury compared with orthopaedic injury? Injury. 2015 Jan;46(1):142–9. pmid:25123975
- 81. Dams-O’Connor K, Spielman L, Singh A, Gordon WA, Lingsma HF, Maas AIR, et al. The impact of previous traumatic brain injury on health and functioning: A TRACK-TBI study. J Neurotrauma. 2013 Dec 15;30(24):2014–20. pmid:23924069
- 82. Gfeller JD, Roskos T. A comparison of insufficient effort rates, neuropsychological functioning, and neuropsychiatric symptom reporting in military veterans and civilians with chronic traumatic brain injury. Behav Sci Law. 2013 Nov;31(6):833–49. pmid:24123226
- 83. Ashman TA, Spielman LA, Hibbard MR, Silver JM, Chandna T, Gordon WA. Psychiatric challenges in the first 6 years after traumatic brain injury: cross-sequential analyses of Axis I disorders. Arch Phys Med Rehabil. 2004 Apr;85(4):S36–42.
- 84. Gil S, Caspi Y, Ben-Ari IZ, Koren D, Klein E. Does memory of a traumatic event increase the risk for posttraumatic stress disorder in patients with a traumatic brain injury? A prospective study. Am J Psychiatry. 2005 May;162(5):963–9. pmid:15863799
- 85. Polinder S, Haagsma JA, Belt E, Lyons RA, Erasmus V, Lund J, et al. A Systematic review of studies measuring health-related quality of life of general injury populations. BMC Public Health. 2010 Dec 23;10:783. pmid:21182775