Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Differences among Research Domain Criteria score trajectories by Diagnostic and Statistical Manual categorical diagnosis during inpatient hospitalization

  • Thomas H. McCoy Jr. ,

    Roles Writing – original draft, Writing – review & editing

    thmccoy@partners.org

    Affiliations Center for Quantitative Health, Massachusetts General Hospital, Boston, MA, United States of America, Department of Psychiatry, Harvard Medical School, Boston, MA, United States of America, Department of Medicine, Harvard Medical School, Boston, MA, United States of America

  • Amelia M. Pellegrini,

    Roles Writing – original draft, Writing – review & editing

    Affiliation Center for Quantitative Health, Massachusetts General Hospital, Boston, MA, United States of America

  • Roy H. Perlis

    Roles Conceptualization, Funding acquisition, Project administration, Supervision, Writing – original draft, Writing – review & editing

    Affiliations Center for Quantitative Health, Massachusetts General Hospital, Boston, MA, United States of America, Department of Psychiatry, Harvard Medical School, Boston, MA, United States of America

Abstract

With brief psychiatric hospitalizations, the extent to which symptoms change is rarely characterized. We sought to understand symptomatic changes across Research Domain Criteria (RDoC) dimensions, and the extent to which such improvement might be associated with risk for readmission. We identified 3,634 individuals with 4,713 hospital admissions to the psychiatric inpatient unit of a large academic medical center between 2010 and 2015. We applied a natural language processing tool to extract estimates of the five RDoC domains to the admission note and discharge summary and calculated the change in each domain. We examined the extent to which symptom domains changed during admission, and their relationship to baseline clinical and sociodemographic features, using linear regression. Symptomatic worsening was rare in the negative valence (0.4%) and positive valence (5.1%) domains, but more common in cognition (25.8%). Most diagnoses exhibited improvement in negative valence, which was associated with significant reduction in readmission risk. Despite generally brief hospital stays, we detected reduction across multiple symptom domains, with greatest improvement in negative symptoms, and greatest probability of worsening in cognitive symptoms. This approach should facilitate investigations of other features or interventions which may influence pace of clinical improvement.

Introduction

In an era of brief psychiatric hospitalizations, the extent to which individuals achieve symptomatic improvement is not well characterized. On one Italian inpatient unit with an average length of stay of 5.7 days, for example, only 13.6% of individuals achieved 'clinically meaningful change' [1]. The modest improvement is not wholly attributable to shorter lengths of stays, as older studies indicated substantial variability in degree of improvement [2]. In general, the aim of hospitalization is not recovery, but sufficient stabilization for safe management at a lower level of care. Such stabilization often entails decrease in a single particularly acute symptom that precipitated hospitalization, such as suicidality or aggressive behavior, rather than modest improvements in a wide range of symptom domains. As such, while many studies focus on changes in syndrome-specific symptomatology, like depression [3] or psychosis [4], few have examined psychiatric symptomatology from a broader transdiagnostic perspective, nor sought to investigate whether some symptoms actually worsen during hospitalization.

Another rarely studied question is whether such inpatient improvement actually matters in longer-term outcomes; that is, does more rapid improvement aid in identifying a population at lower readmission risk? Or, conversely, might such improvement instead represent a fluctuating illness course, such that readmission is actually more likely compared to individuals with more chronic symptomatology? In one study of inpatients receiving psychotherapy, rapid improvement in a general measure of psychopathology indicated better outcomes at 6 months[5].

Recognizing the difficulty in extrapolating from this sparse literature to a modern inpatient psychiatric unit, we applied a recently-validated method for characterizing symptom burden across five dimensions approximating the National Institute of Mental Health (NIMH) Research Domain Criteria (RDoC) [69]. The NIMH RDoC system is a framework for organizing neuropsychiatric symptoms into five domains which are rooted in neuroscience and independent of conventional diagnostic labels [1012]. Estimates of the RDoC domains based on clinical documentation have shown associations with healthcare utilization and death by suicide and have been applied successfully in a genome-wide association study [1315]. Given the emerging importance of RDoC and the uncertainty of current clinical trajectories of hospitalized patients, we sought to understand both the extent to which symptom domains change for better or worse, and then whether these changes are associated with differential risk for readmission.

Materials and methods

Study cohort and variables

We investigated a retrospective cohort drawn from an inpatient general psychiatry unit in a large academic medical center. Electronic health records were used to identify the admission notes and discharge summaries (which exclude admission notes) for all inpatient psychiatric admissions between 2010 and 2015, as well as sociodemographic features, including age at admission, sex, race, and insurance type (public versus private payer). As a measure of overall comorbidity, Charlson comorbidity index was calculated based on billing codes prior to admission and the primary diagnosis for each admission was categorized based on ICD9 codes. Registration data was used to calculate length of hospital admission in days. A data mart with all data was generated using Informatics for Integrating Biology and the Bedside, or i2b2, server software [16,17].

The institutional review board of Partners HealthCare reviewed and approved the study protocol; the requirement for informed consent was waived as a retrospective health care utilization study with no patient contact.

Calculation of estimated RDoC (eRDoC) scores by natural language processing

We have previously reported a method for deriving estimates of the five RDoC domains from narrative clinical notes [8] and made this tool available as open-source software [18]. This freely available tool draws on expert-curated token lists, expanded via unsupervised machine learning to identify synonyms of those terms which occur in clinical documentation. These final lists are then converted into scores as the proportion of terms appearing in any given note. For further details of the scoring method and validation see McCoy, Yu et al., 2018 [8,15]. In the present study, estimated RDoC difference scores were calculated as the difference between the discharge summary domain estimate and the admission note domain estimate. For example, a patient with an eRDoC negative valence estimate of 0.5 on admission and 0.1 on discharge would have a difference score of -0.4.

Outcomes

Time to hospital readmission was determined by identifying the date of rehospitalization documented in the electronic health record, where available. Individuals lost to follow-up within 90 days—i.e., those for whom there were no documented facts (ICD9 diagnosis or CPT codes) at or beyond 90 days—were excluded from these analyses.

Analysis

As we sought to understand the relationship between clinical features and admissions, the primary unit of analysis was admission, recognizing that such admissions could be clustered within individuals (i.e., an individual might have multiple admissions). We first characterized individual (subject) level difference from admission to discharge in estimated symptoms within domain to describe the change during hospitalization. To characterize changes over the course of a hospital encounter within a given domain by diagnostic groups, we used one sample t-tests of each domain difference score within diagnostic group. We then identified the subset of encounters associated with worsening in each domain, defined a priori as at least a 1 standard deviation increase in domain score between admission and discharge. Finally, we used analysis of variance to test for diagnostic category group differences in change in estimated domain symptom burden over the course of hospitalization. Where omnibus test was nominally significant at p<0.05, Tukey’s HSD test was used to examine pairwise comparisons while controlling type 1 error. We also sought to understand the association between change score and 90-day readmission, using mixed effects logistic regression (to address admissions nested within patients, where available), among those patients who had adequate follow up to observe the outcome. This analysis was conducted as both a univariate (crude) analysis and as a multivariate analysis adjusting for age, race (white vs nonwhite), sex, insurance type (presence/absence of public insurance), length of hospital stay, and age-adjusted Charlson comorbidity index, consistent with our prior work [13]. These covariates were included to adjust for potential confounding effects, rather than to examine independent effects.

Results

The full cohort (Table 1, top) included 4,713 admissions for 3,634 individuals; the mean age was 45.0 (SD 16.6) years, and admissions were 49% male, 61% publicly insured, and 72% white. The most common primary diagnosis was major depressive disorder (n = 1,021, or 42.0%) followed by psychotic disorder (n = 775, or 31.9%), bipolar depression (n = 265, or 10.9%), and bipolar mania/mixed state (n = 168, or 6.9%). All but 63 (1.3%) had adequate follow-up for subsequent inclusion in the 90-day readmission analysis.

While the majority of individuals exhibited improvement, a subset worsened in each domain (Table 1, bottom), ranging from 21/4,713 (0.4%) for negative valence and 241/4,713 (5.1%) for positive valence to 1,216/4,713 (25.8%) for cognition.

The within diagnosis average of individual eRDoC difference scores for each domain are illustrated in Fig 1 (for numeric values, see S1 Table). Fig 1 illustrates the overall pattern of change, with negative symptoms significantly declining between admission and discharge (i.e., the confidence interval does not include zero) regardless of diagnostic category and cognitive symptoms failing to change significantly across all categories. Other domains were more variable across diagnostic categories with positive valence the second most frequently-improved (i.e., reduction between admission and discharge was observed in all diagnostic categories except PTSD [S1 Table]).

thumbnail
Fig 1. Average changes in estimated Research Domain Criteria scores between admission and discharge by ICD9 categorical diagnosis.

Bars indicate 95% confidence intervals. MDD = major depressive disorder; BPAD-D = bipolar affective disorder, depression; BPAD-M = bipolar affective disorder, mania/mixed; PTSD = post-traumatic stress disorder.

https://doi.org/10.1371/journal.pone.0237698.g001

Turning from comparisons within diagnostic category (i.e., for a given diagnosis, what domains change the most) to those across diagnostic category (i.e., which domains change differentially by diagnosis), the average change in negative [F(6) = 18.582, p < .00001], positive [F(6) = 6.643, p < .00001], and arousal [F(6) = 14.679, p < .00001] eRDoC score over the course of admission differed significantly by ICD9 diagnostic category, whereas neither social [F(6) = 1.781, p = 0.10] nor cognitive [F(6) = 1.293, p = 0.26] eRDoC score did so. Diagnostic groups which differed significantly in post hoc pairwise testing of negative, positive, and arousal domain score main effects are shown in S2 Table.

Readmission within 90 days of hospital discharge occurred following 1,639 (34.8%) of the discharges. Of the five RDoC domains, change in positive valence, negative valence, as well as in arousal and regulatory process were all associated with early readmission in both crude and adjusted mixed effects logistic regression models (Table 2). Social process and cognitive symptom change were not significantly associated with 90-day regardless of covariates. To illustrate comparison across domains Fig 2 plots odds ratios and 95% confidence intervals from the logistic regression models predicting odds of 90-day readmission for changes in individual eRDoC domains, unadjusted and then after adjustment for sociodemographic and clinical features as shown in Table 2. Negative valence exhibited the strongest association with 90-day readmission [crude odds ratio 1.05–1.22] such that a patient who had a 1-point increase in eRDoC negative valence score between admission and discharge had ~13% greater odds of readmission.

thumbnail
Fig 2. Crude and adjusted odds ratios (OR) for 90-day readmission risk, by change in estimated Research Domain Criteria score between discharge and admission.

https://doi.org/10.1371/journal.pone.0237698.g002

thumbnail
Table 2. Crude and adjusted odds ratios (OR) for 90-day readmission risk, by change in estimated Research Domain Criteria score between admission and discharge.

https://doi.org/10.1371/journal.pone.0237698.t002

Discussion

In this study of transdiagnostic symptom change during inpatient admission, we identified distinct patterns of association with individual estimated RDoC domains. Nearly all hospitalized patients across disorders experienced improvement in the negative valence domain, and such improvement was associated with reduction in readmission risk. A prior study of brief hospital stays found that only a minority—less than 15%—were associated with clinically meaningful improvement [1]. Our results suggest that most patients achieve some improvement in negative valence, which is meaningful insofar as it associates with reduced readmission risk. A prior study using a brief depression screen during hospitalization (while not a change score per se) likewise suggested that less improvement is associated with readmission [19].

We further found that worsening in cognitive domain scores was detectable in about one-fourth of hospitalized patients, but that short-term cognitive change did not associate with likelihood of readmission. Our findings raise the important question of why a subset of hospitalized psychiatric patients might experience short-term worsening in cognition, identifying a rarely-studied but potentially informative subgroup which may give insight into both important connections among mood and memory disorders and into the utility of RDoC based cognitive screening.[20,21] Further study of those who worsen, in some regards, is an important further direction for transdiagnostic research building on prior work within diagnostic groups. [3,2224]

We note multiple limitations in interpreting our results. First, absent structured scales, we cannot relate these findings to standard measures beyond our prior validation efforts [1315]. In particular, it is possible that our observation of modest change in cognitive symptoms reflects a lack of sensitivity for such change, compared to (for example) neurocognitive measures. On the other hand, we would underscore that rating scales are not a standard aspect of care in most clinical settings, so non-research measures may be more feasible for large-scale application. Second, some of the change observed in a given domain may reflect changes in words used in admission compared to discharge notes–in the former case, a note may seek to justify admission, while in the latter a note may seek to justify discharge. In general, this expectation would likely bias us toward greater improvement (if symptoms are minimized for discharge) but does not preclude examining associations with outcomes or comparisons across diagnoses. Third, these findings pertain to a single hospital unit; while we have previously shown them to be valid across populations, the ability to detect symptom change merits study in other populations. We also note that the tool used in this analysis does not include the newly added sixth RDoC domain, which focuses on sensorimotor symptoms.[25] The study is strengthened by the use of a large general psychiatric cohort cared for in a comprehensive medical system and use of a transdiagnostic approach—estimating RDoC scores from clinical text–which has been studied in both clinical and translational contexts and is freely available. In combination this study combines the strength of high throughput RDoC phenotyping and a diverse clinical cohort to directly address the RDoC aim of look across, and sub-dividing within current diagnostic categories.[26,27]

Taken together, our results showing variability in RDoC domain trajectory within and across diagnosis and subsequent association with clinical utilization illustrate the application of a transdiagnostic approach to understand not only dimensions of psychopathology, but patterns of change, that is trajectories, in these symptoms over time. They suggest that even as some domains improve, others may remain the same or worsen during a hospital stay, and that such improvement patterns associate with posthospital outcomes. This evidence of clinical outcome variability based on the specific composition of transdiagnostic neuropsychiatric constructs, and changes among those symptom domains, is a critical step toward realizing the goal of psychiatric precision medicine.[11,24,2830]

The study results also show the ability of a natural language processing tool to detect magnitude of clinical improvement, even in the absence of formal rating scales. This approach does not replace such scales but acknowledges that, particularly in the inpatient setting, it may be challenging to collect comprehensive and transdiagnostic phenotypic data using traditional means.

References

  1. 1. Barbato A, Parabiaghi A, Panicali F, Battino N, D’Avanzo B, de Girolamo G, et al. Do patients improve after short psychiatric admission? A cohort study in Italy. Nordic Journal of Psychiatry. 2011 Sep;65(4):251–8. pmid:21062122
  2. 2. Neimeyer RA, Baker KD, Haykal RF, Akiskal HS. Patterns of symptomatic change in depressed patients in a private inpatient mood disorders program. Bull Menninger Clin. 1995;59(4):460–71. pmid:8535385
  3. 3. Clapp JD, Grubaugh AL, Allen JG, Mahoney J, Oldham JM, Fowler JC, et al. Modeling Trajectory of Depressive Symptoms Among Psychiatric Inpatients: A Latent Growth Curve Approach. The Journal of Clinical Psychiatry. 2013 May 15;74(5):492–9. pmid:23759452
  4. 4. Anda L, Brønnick KS, Johnsen E, Kroken RA, Jørgensen H, Løberg E-M. The Course of Neurocognitive Changes in Acute Psychosis: Relation to Symptomatic Improvement. PLoS One. 2016 Dec 15;11(12):e0167390. pmid:27977720
  5. 5. Melchior H, Schulz H, Kriston L, Hergert A, Hofreuter-Gätgens K, Bergelt C, et al. Symptom change trajectories during inpatient psychotherapy in routine care and their associations with long-term outcomes. Psychiatry Research. 2016 Apr 30;238:228–35. pmid:27086238
  6. 6. Casey BJ, Craddock N, Cuthbert BN, Hyman SE, Lee FS, Ressler KJ. DSM-5 and RDoC: progress in psychiatry research? Nature reviews Neuroscience. 2013 Nov;14(11):810–4. pmid:24135697
  7. 7. Clark LA, Cuthbert B, Lewis-Fernández R, Narrow WE, Reed GM. Three Approaches to Understanding and Classifying Mental Disorder: ICD-11, DSM-5, and the National Institute of Mental Health’s Research Domain Criteria (RDoC). Psychol Sci Public Interest. 2017 Nov;18(2):72–145. pmid:29211974
  8. 8. McCoy TH Jr, Yu S, Hart KL, Castro VM Brown HE, Rosenquist JN, et al. High Throughput Phenotyping for Dimensional Psychopathology in Electronic Health Records. Biol Psychiatry. 2018 Jun 15;83(12):997–1004. pmid:29496195
  9. 9. Patrick CJ, Hajcak G. Reshaping clinical science: Introduction to the Special Issue on Psychophysiology and the NIMH Research Domain Criteria (RDoC) initiative: Introduction to RDoC Special Issue. Psychophysiology. 2016 Mar;53(3):281–5. pmid:26877114
  10. 10. Insel TR, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. The American Journal of Psychiatry. 2010 Jul;167(7):748–51. pmid:20595427
  11. 11. Insel TR. The NIMH Research Domain Criteria (RDoC) Project: Precision Medicine for Psychiatry. American Journal of Psychiatry. 2014 Apr;171(4):395–7. pmid:24687194
  12. 12. Cuthbert BN, Insel TR. Toward the future of psychiatric diagnosis: the seven pillars of RDoC. BMC Med. 2013 May 14;11:126. pmid:23672542
  13. 13. McCoy TH, Castro VM, Rosenfield HR, Cagan A, Kohane IS, Perlis RH. A clinical perspective on the relevance of research domain criteria in electronic health records. Am J Psychiatry. 2015 Apr;172(4):316–20. pmid:25827030
  14. 14. McCoy TH, Pellegrini AM, Perlis RH. Research Domain Criteria scores estimated through natural language processing are associated with risk for suicide and accidental death. Depress Anxiety. 2019 May;36(5):392–9. pmid:30710497
  15. 15. McCoy TH Jr, Castro VM Hart KL, Pellegrini AM, Yu S, Cai T, et al. Genome-wide Association Study of Dimensional Psychopathology Using Electronic Health Records. Biological Psychiatry. 2018 Jun 15;83(12):1005–11. pmid:29496196
  16. 16. Murphy SN, Mendis M, Hackett K, Kuttan R, Pan W, Phillips LC, et al. Architecture of the open-source clinical research chart from Informatics for Integrating Biology and the Bedside. AMIA Annual Symposium proceedings. 2007;548–52. pmid:18693896
  17. 17. Murphy SN, Weber G, Mendis M, Gainer V, Chueh HC, Churchill S, et al. Serving the enterprise and beyond with informatics for integrating biology and the bedside (i2b2). J Am Med Inform Assoc. 2010 Mar;17(2):124–30. pmid:20190053
  18. 18. McCoy, Jr TH, Yu S, Cai T, Perlis RH. CQH Dimensional Phenotyper [Internet]. [cited 2018 Mar 1]. Available from: https://github.com/thmccoy/CQH-Dimensional-Phenotyper
  19. 19. Cancino RS, Culpepper L, Sadikova E, Martin J, Jack BW, Mitchell SE. Dose-response relationship between depressive symptoms and hospital readmission. Journal of Hospital Medicine. 2014 Jun;9(6):358–64. pmid:24604881
  20. 20. Mirza SS, Wolters FJ, Swanson SA, Koudstaal PJ, Hofman A, Tiemeier H, et al. 10-year trajectories of depressive symptoms and risk of dementia: a population-based study. The Lancet Psychiatry. 2016 Jul;3(7):628–35. pmid:27138970
  21. 21. McCoy TH, Han L, Pellegrini AM, Tanzi RE, Berretta S, Perlis RH. Stratifying risk for dementia onset using large‐scale electronic health record data: A retrospective cohort study. Alzheimer’s & Dementia. 2020 Jan 16;j.jalz.2019.09.084.
  22. 22. van Mierlo M, van Heugten C, Post MWM, Hoekstra T, Visser-Meily A. Trajectories of health-related quality of life after stroke: results from a one-year prospective cohort study. Disability and Rehabilitation. 2018 Apr 24;40(9):997–1006. pmid:28288530
  23. 23. Franke B, Michelini G, Asherson P, Banaschewski T, Bilbow A, Buitelaar JK, et al. Live fast, die young? A review on the developmental trajectories of ADHD across the lifespan. European Neuropsychopharmacology. 2018 Oct;28(10):1059–88. pmid:30195575
  24. 24. Fojo AT, Musliner KL, Zandi PP, Zeger SL. A precision medicine approach for psychiatric disease based on repeated symptom scores. Journal of Psychiatric Research. 2017 Dec;95:147–55. pmid:28863391
  25. 25. Garvey MA, Cuthbert BN. Developing a Motor Systems Domain for the NIMH RDoC Program. Schizophrenia Bulletin. 2017 Sep;43(5):935–6. pmid:28911051
  26. 26. Insel TR, Cuthbert BN. Brain disorders? Precisely. Science. 2015 May 1;348(6234):499–500. pmid:25931539
  27. 27. Insel T. Transforming Diagnosis [Internet]. 2013. Available from: https://www.nimh.nih.gov/about/directors/thomas-insel/blog/2013/transforming-diagnosis.I
  28. 28. Insel TR, Cuthbert BN. Medicine. Brain disorders? Precisely. Science. 2015 May 1;348(6234):499–500. pmid:25931539
  29. 29. McMahon FJ, Insel TR. Pharmacogenomics and Personalized Medicine in Neuropsychiatry. Neuron. 2012 Jun;74(5):773–6. pmid:22681682
  30. 30. Fernandes BS, Williams LM, Steiner J, Leboyer M, Carvalho AF, Berk M. The new field of ‘precision psychiatry.’ BMC Med. 2017 Dec;15(1):80. pmid:28403846