Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

The Effects of Cognitive Therapy versus ‘No Intervention’ for Major Depressive Disorder

  • Janus Christian Jakobsen ,

    janusjakobsen@mac.com

    Affiliations Psychiatric Research Unit, Copenhagen University Hospital and Region Zealand, Roskilde, Denmark, Copenhagen Trial Unit, Centre for Clinical Intervention Research, Department 3344, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark

  • Jane Lindschou Hansen,

    Affiliation Copenhagen Trial Unit, Centre for Clinical Intervention Research, Department 3344, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark

  • Ole Jakob Storebø,

    Affiliation Psychiatric Research Unit, Copenhagen University Hospital and Region Zealand, Roskilde, Denmark

  • Erik Simonsen,

    Affiliation Psychiatric Research Unit, Copenhagen University Hospital and Region Zealand, Roskilde, Denmark

  • Christian Gluud

    Affiliation Copenhagen Trial Unit, Centre for Clinical Intervention Research, Department 3344, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark

Abstract

Background

Major depressive disorder afflicts an estimated 17% of individuals during their lifetimes at tremendous suffering and costs. Cognitive therapy may be an effective treatment option for major depressive disorder, but the effects have only had limited assessment in systematic reviews.

Methods/Principal Findings

We used The Cochrane systematic review methodology with meta-analyses and trial sequential analyses of randomized trials comparing the effects of cognitive therapy versus ‘no intervention’ for major depressive disorder. Participants had to be older than 17 years with a primary diagnosis of major depressive disorder to be eligible. Altogether, we included 12 trials randomizing a total of 669 participants. All 12 trials had high risk of bias. Meta-analysis on the Hamilton Rating Scale for Depression showed that cognitive therapy significantly reduced depressive symptoms (four trials; mean difference −3.05 (95% confidence interval (Cl), −5.23 to −0.87; P<0.006)) compared with ‘no intervention’. Trial sequential analysis could not confirm this result. Meta-analysis on the Beck Depression Inventory showed that cognitive therapy significantly reduced depressive symptoms (eight trials; mean difference on −4.86 (95% CI −6.44 to −3.28; P = 0.00001)). Trial sequential analysis on these data confirmed the result. Only a few trials reported on ‘no remission’, suicide inclination, suicide attempts, suicides, and adverse events without significant differences between the compared intervention groups.

Discussion

Cognitive therapy might be an effective treatment for depression measured on Hamilton Rating Scale for Depression and Beck Depression Inventory, but these outcomes may be overestimated due to risks of systematic errors (bias) and random errors (play of chance). Furthermore, the effects of cognitive therapy on no remission, suicidality, adverse events, and quality of life are unclear. There is a need for randomized trials with low risk of bias, low risk of random errors, and longer follow-up assessing both benefits and harms with clinically relevant outcome measures.

Introduction

According to the WHO, major depressive disorder is the second largest healthcare problem worldwide in terms of illness induced disability [1]. Major depressive disorder afflicts an estimated 17% of individuals during their lifetimes at tremendous cost to the individual and society [2], [3]. Roughly a third of all depressive disorders take a chronic course and, compared to other medical disorders, depressive illnesses causes the most significant deterioration in individual quality of life [4][6]. Approximately 15% of all depressive patients will commit suicide over a 10–20 year period [7].

Antidepressant medication remains the mainstay in the treatment of depression [8]. However, meta-analyses have shown that the new antidepressants only obtained beneficial effect in severely depressed patients, and that this effect seems to be clinically small [9], [10]. Antidepressants are, however, known to decrease the risk of relapse [11]. The benefits of antidepressant medication seem to be limited and this raises the question if there are other effective treatments for this serious illness?

Aaron T. Beck originally developed cognitive therapy for depression [12]. Beck believed that critical life events could accentuate hidden negative beliefs, which could generate negative automatic thoughts. These negative thoughts could lead to symptoms of depression, which then could reinforce more negative automatic thoughts. The main goal of the ‘cognitive model of depression’ is to correct these negative beliefs and thoughts, in order to treat the depressive symptoms [12]. A recently published systematic review showed that cognitive therapy might not be an effective treatment for major depressive disorder compared with ‘treatment as usual’ (different forms of non-specific supportive interventions) [13]. Another systematic review shows that cognitive therapy has a preventive effect against recurrent depression, and that this effect clearly surpasses the preventive effects of antidepressant medication [14]. Cognitive therapy versus ‘no intervention’ appears to be an effective treatment for major depressive disorder [15]. We have been unable to find any systematic reviews with meta-analyses, using Cochrane methodology, examining the effect of cognitive therapy versus ‘no intervention’ for major depressive disorder, and the effect size of cognitive therapy is therefore unclear. We embarked on a systematic review using Cochrane methodology to assess the effect of cognitive therapy versus ‘no intervention’ in the treatment of major depressive disorder [16]. We used assessment of bias risk to reduce systematic errors, and trial sequential analysis to reduce the risk of random errors [16][19].

Methods

We conducted our systematic review of randomized clinical trials involving meta-analysis and trial sequential analysis [16][19] to answer the question: what are the beneficial and harmful effects of cognitive therapy versus ‘no intervention’ in the treatment of major depressive disorder?

For details regarding the methodology please consult our protocol published on our website (www.ctu.dk) in February 2010, before we began the systematic literature searches in all relevant databases, data-extraction, and analyses [20].

In short, we included all randomized clinical trials comparing the effects of cognitive therapy alone versus ‘no intervention’ alone or cognitive therapy in combination with any co-intervention versus ‘no intervention’ in combination with a similar co-intervention. These co-interventions had to be administered equally in both intervention groups. We did this because we wanted to quantify the effect of cognitive therapy versus ‘no intervention’. No intervention encompassed all potential interventions, including medical treatments; talk therapy; psychology; etc.- except ‘waiting list’. The trials were included irrespective of language, publication status, publication year, and publication type - based on searches in The Cochrane Library's CENTRAL, MEDLINE via PubMed, EMBASE, Psychlit, PsycInfo, and Science Citation Index Expanded (Figure S1. The timeframe for the search was all trials published before February 2010.

To be included, participants had to be older than 17 years with a primary diagnosis of major depressive disorder. Trials were only included if the diagnosis of depression was based on one of the standardized criteria, such as ICD 10 [21], DSM III [22], DSM III-R [23], or DSM IV [24]. Comorbidity with other psychiatric diagnoses was not an exclusion criterion. The following types of trials were excluded:

  • Trials focusing on depressed participants with comorbid serious somatic illness, e.g., myocardial infarction, multiple sclerosis, cerebral stroke, cancer, etc.
  • Trials focusing on ‘late life’ depression or depression in the elderly, most often participants over 65 years.
  • Trials focusing on pregnancy related depression, e.g., postpartum depression, postnatal depression, etc.
  • Drug or alcohol dependence related depression.

These exclusions were conducted because we expect participants in such trials to respond differently to standardized psychotherapy than other depressed patients, and these types of depressed patients are traditionally examined in separate trials [25][28].

Interventions

Cognitive therapy.

Cognitive therapy and cognitive-behavioral therapy are collective terms for a range of different forms of interventions, and it is difficult to find a simple definition which adequately describes this psychotherapeutic method. However, we selected the following criteria from Beck as being necessary for the intervention to be classified as ‘cognitive therapy’ [12]:

  1. That the intervention sought to link thoughts, feelings, and behavior, and related these to the depressive symptoms.
  2. That the intervention sought to record and correct irrational thoughts or behavioral patterns, and related these to the depressive symptoms.
  3. That the intervention sought to teach the patient alternative methods of thinking or behaving, and related these to the depressive symptoms.
  4. That the intervention was undertaken face-to-face either individually or in a group.

We accepted any co-intervention to cognitive therapy as long as this co-intervention was similar and administered equally to the experimental group (cognitive therapy) and the control group (‘no intervention’). As mentioned in the introduction, this was done because we wanted to quantify the effect of cognitive therapy.

Furthermore, the trials had to present a treatment manual and had to document adherence to the treatment manual in order for the intervention to be classified as ‘cognitive therapy, adequately defined’. All other trials that classified their intervention as ‘cognitive’ or ‘cognitive-behavioral’ were included, but the intervention was classified under ‘cognitive therapy, not adequately defined’.

Trial selection

Three of the review authors (JJ, OJS, and JLH) independently selected relevant trials. If a trial was selected by three or two of the three, it was included. If a trial only was identified only by one of the three, it was discussed whether the trial should be included. Excluded trials were entered on a list, stating the reason for exclusion.

Data extraction

Data were extracted for trial design, bias risk, and outcomes independently by two authors (JJ and JLH). Disagreements were resolved by discussion or through arbitration (CG). We used the instructions in The Cochrane Handbook for Systematic Reviews of Interventions in our evaluation of the methodology and hence bias risk of the trials [16]. We assessed the bias risk in respect to generation of the allocation sequence; allocation concealment; blinding; intention-to-treat analysis; drop-outs; reporting of outcome measures; economic bias; and academic bias. Economic bias may be present if a trial is financed by an individual or organisation that might have an interest in a given result from the trial [16]. Aqcademic bias may be present if one or more of the trialists have an academic or personal interest in a given result from the trial [16]. These components enable classification of the included trials into trials with ‘low risk of bias’ or with ‘high risk of bias’. The trials were overall classified as ‘high risk of bias’ if one or more of the above components was ‘uncertain’ or ‘high risk of bias’ [16], [29][32]. This classification is important because trials with ‘high risk of bias’ may overestimate positive intervention effects and underestimate negative intervention effects, and we wanted to relate the validity of our results to the risk of bias in the included trials [16], [29][32].

Primary outcomes

Depressive symptoms.

Our primary outcomes were the mean value of Hamilton Rating Scale for Depression (HDRS) [33], Beck Depression Inventory (BDI) [34], or Montgomery-Asberg Depression Rating Scale (MADRS) [35] at follow-up. We included data based on the total number of randomized patients (intention-to-treat analysis) if these data were reported. We planned to estimate the therapeutic follow-up responses at two time points:

  • At cessation of treatment: The trials' original primary choice of completion date was used. This was the most important outcome measure time point in this review.
  • At maximum follow-up.

Adverse events.

We classified adverse events as serious or non-serious. Serious adverse events were defined as medical events that are life threatening; result in death; disability or significant loss of function; that cause hospital admission or prolonged hospitalization; a hereditary anomaly; or fetal injury [36]. All other adverse events (that is, events that have not necessarily had a causal relationship with the treatment, but that resulted in a change in- or cessation of the treatment) were considered as non-serious events.

Quality of life.

We included any measure of quality of life noting each assessment measure.

Secondary outcomes

The proportion of patients not having achieved remission was our first secondary outcome. We included data based on the total number of randomized participants (intention-to-treat analysis) - if possible. If the results were not based on the total number of participants, we preformed an intention-to-treat analysis assuming that the participants not included in the results did not achieve remission [16]. We pragmatically defined remission as a HDRS of less than 8, BDI less than 10, or MADRS less than 10 [34][36]. These definitions are also the most commonly used.

Records of suicide inclination, suicide attempts, or suicides were other secondary outcomes.

Statistical methods

This meta-analysis was undertaken according to the recommendations stated in The Cochrane Handbook for Systematic Reviews of Interventions [16]. In analyzing continuous outcomes with both fixed-effect and with random-effects models, we used the mean difference (MD) with a 95% confidence interval. For statistical calculations we used RevMan version 5.0 [37]. We did not use ‘standardized mean difference’ so each outcome measure was analyzed separately. We did not adjust the outcome variables at follow-up according to the baseline values [16].

We used the odds ratio with a 95% confidence interval to estimate intervention effects on dichotomous outcomes with both fixed-effect and with random-effects models [37].

We performed ‘test of interaction’ [38] for all subgroup analyses [20].

For the primary outcome measure and significant secondary outcome measure, we also conducted trial sequential analysis [39]. In order to calculate the required information size and the cumulative Z-curve's eventual breach of relevant trial sequential monitoring boundaries [16][19], the required information size for continuous outcomes was based on a type I error of 5%, a beta of 10% (power of 90%), the variance of all the trials (as no trial had low risk of bias), and a minimal relevant difference of 2 points on the HDRS. For dichotomous outcomes, we based the control proportion on our meta-analytic findings, an assumed relative risk reduction of 30%, a type I error of 5%, a beta of 10% (power of 90%), and the heterogeneity in the meta-analysis.

Results

Search results

Our primary literature search identified 4536 publications. According to our protocol [20] we excluded 4137 publications on the basis of the title or abstract, and further 339 citable units were excluded on the basis of the full publication. These exclusions were done either because the publications did not relate to cognitive therapy and depression, or because they were not randomized trials comparing cognitive therapy versus ‘no intervention’. Further 41 publications were excluded because the trial participants or the interventions did not meet our inclusion criteria.

Included trials

We identified and included 19 publications [40][58] on 12 trials [40][43], [46][51], [53], [55] randomizing a total of 669 participants (Figure S2).

Only six of the trials [40][43], [47], [50] used an intervention that we classified as ‘adequately defined’ (see above). We classified the therapists’ level of experience and/or education in two trials as ‘high’ [49], [55], in two trials as ‘intermediate’ [46], [50], in one trials as ‘low’ [40], and in the last seven as ‘unclear’ [41][43], [47], [48], [51], [53]. Three trials used cognitive group therapy [40], [42], [51], one trial used a combination of group and individual therapy [53], the remaining eight trials used only individual therapy [41], [43], [46][50], [55].

The duration and the extent of the therapy varied in the different trials from six weekly 30 minute sessions of treatment [47] to 24 weeks of treatment (five times a week during the inpatient stay and weekly during the outpatient phase) [55].

Eight trials used the experimental intervention cognitive therapy as add on therapy to antidepressant medicine [46][51], [53], [55]. All of the eight trials used different antidepressants (Figure 1). The antidepressant medicine was delivered similarly in the experimental and control groups in all of the trials.

thumbnail
Figure 1. The effect of cognitive therapy versus ‘no intervention’ at cessation of treatment on Hamilton Rating Scale for Depression (HDRS).

Below figure: All four trials used only individual cognitive therapy. The therapists' level of experience and/or education was classified as ‘high’ in Dozios (2009), as ‘intermediate’ in Murphy (1984) and Hollon (1992), and as ‘unclear’ in Scott (1997).

https://doi.org/10.1371/journal.pone.0028299.g001

Blackburn et al. (1981) examined the effect of cognitive therapy and antidepressants versus antidepressants [51]. The participants were assessed with HDRS and BDI. The results at the end of treatment show a significant effect of cognitive therapy compared with the control. However, the trial did not report SD for the mean values.

Teasdale et al. (1984) examined the effect of cognitive therapy and ‘treatment as usual’ versus ‘treatment as usual’ [41]. The participants were assessed with HDRS, BDI and MADRS. However, the results were only reported as median values. Participants receiving cognitive therapy had improved significantly greater than the control group on all three scales at cessation of treatment. There was no significant difference at three months follow-up.

Usaf et al. (1990) examined the effect of cognitive therapy versus ‘waiting list’ [42]. The participants were assessed with BDI. Participants receiving cognitive therapy had a non-significant greater improvement on outcome measures compared with control. However, the trial did not report SD for the mean values

Wright et al. (2005) examined the effect of cognitive therapy versus ‘waiting list’ [43]. The participants were assessed with HDRS and BDI. The results at end of treatment showed a significant effect of cognitive therapy compared with control. However, the trial did not report mean values and SD. The authors chose to report change in scores instead of mean on continuous outcome, because the baseline-means were significantly different between the two intervention groups.

We have written to the authors of the four trials in the above [41][43], [51] requesting the necessary data. We have received no answer so we have not been able to include the data from these four trials in the following analysis.

Table 1 summarizes the characteristics of the 12 included trials.

Bias risk

We assessed all of the 12 included trials [40][43], [46][51], [53], [55] as having ‘high risk of bias’ due to unclear or inadequate components as described in Table 2.

Primary outcome measures

Depressive symptoms.

Four trials assessed and reported HDRS as a continuous outcome measure at the end of treatment [46], [47], [49], [50]. Eight trials also assessed and reported BDI [40], [46][50], [53], [55].

Meta-analysis with the fixed-effect model on the HDRS data from the four trials [46], [47], [49], [50], shows that cognitive therapy at the end of therapy significantly reduced depressive symptoms compared with ‘no intervention’. We found a mean difference on −3.05 HDRS (95% CI −5.23 to −0.87; P<0.006, I2 = 0) (Figure 1). The I2 statistic describes the percentage of variation across trials that are due to heterogeneity rather than chance. Meta-analysis with the random-effects model gave identical results.

Meta-analysis with the fixed-effect model on the BDI data from the eight trials [40], [46][50], [53], [55] was in agreement with the results from HDRS (mean difference on −4.86 BDI (95% CI −6.44 to −3.28; P = 0.00001, I2 = 0)) (Figure 2). Meta-analysis with the random-effects gave identical results.

thumbnail
Figure 2. The effect of cognitive therapy versus ‘no intervention’ at cessation of treatment on Becks Depression Inventory (BDI).

https://doi.org/10.1371/journal.pone.0028299.g002

Trial sequential analysis on the HDRS data showed that ‘insufficient data’ have been obtained to decide if cognitive therapy is superior compared with ‘no intervention’ (Figure 3). Trial sequential analysis on the BDI data showed a significant beneficial effect of cognitive therapy compared with ‘no intervention’ (Figure 4).

thumbnail
Figure 3. Trial sequential analysis of the cumulative meta-analysis of the effect of cognitive therapy versus ‘no intervention’ for major depressive disorder on the Hamilton Rating Scale for Depression (HDRS).

Below figure: The required information size of 994 is calculated based on an intervention effect compared with ‘no intervention’, of 2 points on the HDRS, a variance of 126.5.04 on the mean difference, a risk of type I error of 5%, and a power of 80%. With these presumptions, the cumulated Z-curve (blue curve) do not cross the trial sequential monitoring boundaries (red inner sloping lines) implying that there is no firm evidence for a beneficial effect of cognitive therapy compared with no intervention.

https://doi.org/10.1371/journal.pone.0028299.g003

thumbnail
Figure 4. Trial sequential analysis of the cumulative meta-analysis of the effect of cognitive therapy versus no ‘intervention’ for major depressive disorder on the Beck Depression Inventory (BDI).

Below figure: The required information size of 570 is calculated based on an intervention effect compared with ‘no intervention’, of 4 points on the BDI, a variance of 153.1 on the mean difference, a risk of type I error of 1% and a power of 90%. With these presumptions, the cumulated Z-curve (blue curve) crosses the trial sequential monitoring boundaries (red inner sloping lines) implying that there is no risk of random error in the estimate of a beneficial effect of cognitive therapy compared with no intervention. However, all trials were considered as high risk of bias, which could explain the positive findings.

https://doi.org/10.1371/journal.pone.0028299.g004

Only two of the trials included assessment data after the cessation of treatment on the HRDS [46], [47]. Murphy et al. (1984) assessed the participants at one month after cessation of treatment and Scott et al. (1997) at one year after cessation of treatment [46], [47]. Meta-analysis with fixed-effect model on these data showed a mean difference on −0.32 HDRS points (95% CI −0.85 to −0.22; P = 0.25, I2 = 57%) and −3.68 BDI points (95% CI −8.11 to −0.75; P = 0.10, I2 = 0) in favor of cognitive therapy. Meta-analysis with random-effects gave an identical result.

Adverse events.

Two trials reported adverse events [40], [50]. Hollon et al. (1992) reported five serious adverse events in the control group (two participants hospitalized due to symptomatic worsening and three experiencing severe adverse reactions to concomitant medications) [50]. Wong et al. (2008) reported one hospitalization in the control group [40]. None of the remaining trials reported on adverse events.

Quality of life.

None of the included trials assessed the quality of life of the participants.

Secondary outcome measures

Participants without remission.

Three trials reported the proportion of participants without remission as a dichotomous outcome measure [46], [50], [55]. We had planned to define remission as a Hamilton score of less than 8, BDI less than 10, or MADRS less than 10. However, this was not possible, so we adopted the slightly different definitions of the individual trials. All three trials defined remission as HRDS less than 7 [46], [50], [55], while one trial also defined remission as a Hamilton score of less than 8 [46]. All three trials also defined remission as BDI less than 10 [46], [50], [55].

Meta-analysis on the HDRS data from the three trials [46], [50], [55] showed that cognitive therapy compared with ‘no intervention’ significantly decreases the risk of ‘no remission’ with an odds ratio of 0.42 (95% CI, 0.21 to 0.85; P = 0.02, I2 = 0) (Figure 5). The number needed to treat to obtain one extra patient with remission is about four patients (95% CI, 3 to 13). Trial sequential analysis on these data shows that we cannot exclude risk of random errors due to sparse data and repetitive testing as the cause for the meta-analysis result (Figure 6).

thumbnail
Figure 5. Effect of cognitive therapy versus ‘no intervention’ on ‘no remission’ (HDRS>7) at cessation of treatment.

https://doi.org/10.1371/journal.pone.0028299.g005

thumbnail
Figure 6. Trial sequential analysis of the cumulative meta-analysis of the effect of cognitive therapy versus no ‘intervention’ for no remission according to the Hamilton Rating Scale for Depression.

Below figure: The required information size of 303 is calculated based on a control event proportion of 62%, an assumed relative risk reduction of 30%, a type I error of 5%, a beta of 10% (power of 90%), and the heterogeneity in the meta-analysis. With these presumptions, the cumulated Z-curve (blue curve) do not cross the trial sequential monitoring boundaries (red inner sloping lines) implying that there is a risk of random error in the estimate of a beneficial effect of cognitive therapy compared with no intervention, either due to sparse data or repetitive testing in the cumulative meta-analysis. Furthermore, all trials were considered as high risk of bias, which could explain the positive findings in the conventional meta-analysis.

https://doi.org/10.1371/journal.pone.0028299.g006

The meta-analysis on the BDI-data from the three trials [46], [50], [55] showed that cognitive therapy compared with ‘no intervention’ did not significantly decrease the risk of ‘no remission’ with an odds ratio of 0.54 (95% CI, 0.27 to 1.09; P = 0.08, I2 = 0).

Participants with suicidal inclination.

Teasdale et al. (1984) reported numbers of patients that deliberate self-poisoned [41]. No patient in the cognitive therapy group self-poisoned. Two of the patients in the control group were treated for deliberate self-poisoning.

Miller et al. (1989) trial used the Modified Scale for Suicidal Ideation [55]. They found no significant difference in suicidal ideation between the different intervention groups, and recorded no suicide attempts or suicides during the trial period.

Hollon et al. (1992) reported three suicide attempts [50], one participant randomized to cognitive therapy and two participants randomized to the control intervention. One from each group died from their attempt.

Wong et al. (2008) reported no suicide attempts in the cognitive therapy group and two suicide attempts in the control group during the intervention period [40]. Neither of these participants died from their attempt.

None of the remaining trials included records of suicide inclination, suicide attempts, or suicides.

Subgroup analyses

According to our protocol [20] we had planned a number of subgroup analyzes, but we found no heterogeneity in our results. We therefore did not conduct subgroup analyses of therapists' level of education and experience (high versus intermediate versus low versus unclear), type of therapy (group versus individual), and use of antidepressants as co-intervention (antidepressant co-intervention versus no antidepressant co-intervention). Our findings indicate that these factors do not seem to influence the effect of cognitive therapy.

We had also planned a subgroup-analysis according to risk of bias [20]. However, as all trials were classified as ‘high risk of bias’ it was not possible to conduct this analysis.

Discussion

The results of our systematic review with meta-analysis and trial sequential analysis (on the BDI-data) suggest that cognitive therapy may significantly reduce depressive symptoms on the HDRS and BDI, and may increase the probability of remission compared with ‘no intervention’. The number needed to treat to obtain one extra patient with remission is about four patients (95% CI, 3 to 13), but trial sequential analysis suggests that this result could be due to random error due to sparse data or repetitive testing (NY ref). When evaluating these data, one should notice that all trials were considered to have high risks of bias which could lead to overestimation of beneficial intervention effects.

The present review has a number of strengths. Our protocol [20] was published before we began systematic literature searches in all relevant databases, data extraction, and data analysis. Data was extracted by two independent authors minimizing the risk of inaccurate data-extraction, and we assessed the risk of bias in all trials according to the Cochrane Handbook for Systematic Reviews of Interventions [16]. We meta-analyzed data both with fixed-effect and random-effects models and both analyses were in agreement in all our results. Furthermore, we performed trial sequential analysis to assess the risk of random errors [16][19]. The results of the trial sequential analysis on the BDI data confirmed the cumulative meta-analysis result. Trial sequential analysis on the HDRS data showed that insufficient data have been obtained, but this analysis was based on the results from only four trials. The trial sequential analysis result also indicates that in order to detect or reject an intervention effect with a minimal relevant difference of two points on HDRS, a required information size of 994 participants may be needed. With a relatively limited number of trials and trial participants and with an increasing number of repetitive tests, the risk of falsely rejecting the null hypothesis (type I error) is substantial Trial sequential analysis is a statistical analysis that enables one to assess the risks of random errors that may occur due to sparse data and multiple testing on accumulating data. Trial sequential analysis is therefore a more robust analysis than the traditional cumulative meta-analysis [16][19].

Our systematic review has a number of limitations. The characteristics of the participants in the different trials, as well as the severity of the depressive symptoms differed. E.g., the participants in Miller et al. (1989) were inpatients and Scott et al. (1997) examined the effect of cognitive therapy for depression in primary care patients [47], [55]. Nine [41], [46][51], [53], [55] of the 12 included trials used some form of antidepressants as co-intervention to cognitive therapy. We did not, however, find any heterogeneity in our analyses and although head-to-head comparisons are needed in order to thoroughly examine differences between intervention groups, this indicates that there is a comparable treatment effect regardless of the use of antidepressants as co-intervention and among the different populations treated. This may make our results more generally applicable. On the other hand, only few trials with few participants were included, which may decrease the external validity of our results.

The fact that we were only able to include 12 trials with a limited number of participants also raises other concerns. Only four of the 12 trials reported mean and SD for HDRS, and only 8 of the 12 trials reported means and SD for BDI. None of the included trials were assessed as being free of ‘selective outcome measure reporting bias’ [16]. There is therefore a risk of within-study selective outcome reporting in the 12 trials. Furthermore, all 12 trials had an overall assessment as ‘high risk of bias’ - so our results may be questionable. Moreover, trial sequential analysis on the effect on HDRS showed that we could not exclude the risk of random errors [16][19]. Due to the limited number of included trials we did not perform a funnel plot or other analysis to explore the risk of publication bias [16]. Other meta-analyses have shown that publication bias significantly has influenced the results from former publications [9]. It is a further limitation that we are not able to assess the risk of publication bias.

Only six of the trials used an intervention that we classified as ‘adequately defined’, i.e., using and documenting the use of a therapeutic manual. And although we did not find any heterogeneity in our results it is imperative in clinical trials that the interventions are adequately defined and described [59]. Factors like personal style, communication skills, and personality of the therapist evidently will influence the way psychotherapy is delivered [60], and it is difficult to describe and control for these subjective factors. It is therefore important to relate psychotherapeutic interventions to a treatment manual. Otherwise it is unclear what kind of intervention the participants were receiving, and it is difficult to apply any result in clinical practice.

A number of subgroups of depressed patients were not included in the trials of this review. These subgroups may react differently to psychotherapy and of course our review cannot be generalized to other than the included patient groups.

Because we wanted to quantify the effect of cognitive therapy, any co-intervention had to be delivered similarly in the experimental intervention group and the control group. So forth, our results show that the benefit from this relatively extensive treatment compared with ‘no intervention’ was only a few points on HDRS and BDI. We believe that these mean differences are relatively small from a clinical viewpoint - especially if you relate these mean differences to the extent and length of the intervention. On the other hand, our analyses demonstrate that the number needed to treat to obtain one extra patient in remission was only about four patients. This estimate was based on only three trials, which primarily defined remission as a HDRS or a BDI score under a given value. Again, we are not able to exclude the risks of systematic errors (bias) and random errors (play of chance) on this estimate.

Depression is generally a difficult condition to treat effectively and other reviews have found similar effects for other forms of psychotherapy [61]. However, the HDRS might not be a useful instrument to quantify the effect of cognitive therapy. Other assessment methods could demonstrate a more substantial effect of any given intervention for depression. Furthermore, severity of depression as measured by the total HDRS score has failed to predict suicide attempts [62], and some publications have questioned the usefulness of the HDRS and concluded that the scale is psychometrically and conceptually flawed [63]. The two other outcome measures often used to assess depressive symptoms, MADRS and BDI, probably correspond to HDRS [64], [65]. The HDRS has during 40 years been the gold standard to quantify depressive symptoms in clinical trials [63]. There may be a need for other assessment methods.

Only two of the trials included assessments after the cessation of treatment. Therefore it is not clear whether cognitive therapy has any effect on depressive symptoms in the longer term.

None of the trials reported measures of quality of life. Outcome measures of quality of life are generally not standardized and thoroughly individually validated [69]. The use of standardized outcome measures for quality of life in research has been limited by difficulties in administering and scoring quality of life, but quality of life can be used as a valid outcome measure [31], [66].

Only two of the included trials reported on some adverse events and only four of the included trials included some records of suicide inclination, suicide attempts, or suicides. Typically adverse events are not reported as thoroughly as beneficial outcome measures [67]. Some psychological interventions might have harmful effects. E.g., psychological debriefing for preventing post-traumatic stress disorder has in some clinical trials showed to have a harmful effect [68]. Possible harmful effects of cognitive therapy are not thoroughly examined.

Future research should focus on comparing different forms of manualized psychotherapy and comparing cognitive therapy with other treatments for depression. First and foremost such trials should be conducted with lower risk of bias (‘systematic errors’) and lower risk of random errors (‘play of chance’) as well as longer follow up [69]. Such trials should also report on adverse events, suicide inclination, suicide attempts, and numbers of suicides. There may also be a need for a new gold standard assessment method other than HRDS to assess depressive symptoms, and if possible more effective interventions for depression must be developed. But first and foremost trials with lower risk of bias and lower risk of random errors are needed.

Conclusions

Cognitive therapy might be an effective treatment for depression measured on HDRS and BDI, but the effects on suicidality, adverse events, and quality of life are unclear. There is a need for randomized trials with low risk of bias, low risk of random errors, and longer follow-up assessing both benefits and harms with clinically relevant outcome measures.

Supporting Information

Author Contributions

Performed the experiments: JJ JLH OJS. Analyzed the data: JCJ CG. Wrote the paper: JCJ CG. Resolved disagreements by discussion or through arbitration: CG. Contributed psychiatric expertise: ES. Contributed to and approved the manuscript: JCJ JLH OJS ES CG.

References

  1. 1. Levav I, Rutz W (2002) The WHO world health report 2001. New understanding- new hope. Israel Journal of Psychiatry & Related Sciences 39: 50–56.
  2. 2. Greenberg P, Stiglin LE, Finkelstein SN, Berndt ER (1993) The economic burden of depression in 1990. Journal of Clinical Psychiatry 405–418.
  3. 3. Kessler RC, McGnagle KA, Zhao S, Nelson CB, Hughes M, et al. (1994) Lifetime and 12- month prevalence of DSM - III- R psychiatric disorders in the united states: Results from the Natinal Comorbidity Survey. Arch Gen Psychiatry 51: 8–19.
  4. 4. Bech P (1999) Stress & livskvalitet (Stress & quality of life). Psykiatrifondens Forlag.
  5. 5. Spijker J, de GR, Bijl RV, Beekman AT, Ormel J, et al. (2002) Duration of major depressive episodes in the general population: results from The Netherlands Mental Health Survey and Incidence Study (NEMESIS). Br J Psychiatry 181: 208–213.
  6. 6. Arnow BA, Constantino MJ (2003) Effectiveness of psychotherapy and combination treatment for chronic depression.893–905.
  7. 7. Fawcett J (1993) The morbitity and mortality of clinical depression. International Clinical Psychopharmacology 217–220.
  8. 8. Cipriani A, Santilli C, Furukawa TA, Signoretti A, Nakagawa A, et al. (2009) Escitalopram versus other antidepressive agents for depression. Cochrane Database Syst Rev CD006532.
  9. 9. Kirsch I, Deacon BJ, Huedo-Medina TB, Scoboria A, Moore TJ, et al. (2008) Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Medicine 5(2): e45.
  10. 10. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358: 252–260.
  11. 11. Geddes JR, Carney SM, Davies C, Furukawa TA, Kupfer DJ, et al. (2003) Relapse prevention with antidepressant drug treatment in depressive disorders: a systematic review. Lancet 361: 653–661.
  12. 12. Beck AT, Rush AJ, Shaw BF, Emery G (1979) Cognitive therapy of depression. Aust NZ J Psychiatry 36: 275–278.
  13. 13. Jakobsen JC, Lindschou Hansen J, Storebø OJ, Simonsen E, Gluud C (2011) The Effects of Cognitive Therapy Versus ‘Treatment as Usual’ in Patients with Major Depressive Disorder. PLoS ONE 6: e22890.
  14. 14. Vittengl JR, Clark LA, Dunn TW, Jarrett RB (2007) Reducing Relapse and Recurrence in Unipolar Depression: A Comparative Meta-Analysis of Cognitive-Behavioral Therapys Effects Reducing Relapse and Recurrence in Unipolar Depression: A Comparative Meta-Analysis of Cognitive-Behavioral Therapys Effects. J CONSULT CLIN PSYCHOL 75: 475.
  15. 15. Elkin I, Shea MT, Watkins JT, Imber SD, Sotsky SM, et al. (1989) National Institute of Mental Health Treatment of Depression Collaborative Research Program. General effectiveness of treatments. Arch Gen Psychiatry 46: 971–982.
  16. 16. Higgins JPT, Green S (2008) “The Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0” (updated March 2011). The Cochrane Collaboration Available: http://www.cochrane-handbook.org. Accessed 2011 Oct 27.
  17. 17. Wetterslev J, Thorlund K, Brok J, Gluud C (2008) Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. Journal of Clinical Epidemiology 61: 64–75.
  18. 18. Brok J, Thorlund K, Gluud C, Wetterslev J (2008) Trial sequential analysis reveals insufficient information size and potentially false positive results in many meta-analysis. J Clin Epidemiol 763–769.
  19. 19. Thorlund K, Devereaux PJ, Wetterslev J, Guyatt G, Ioannidis JP, et al. (2009) Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses? Int J Epidemiol 38: 276–286.
  20. 20. Jakobsen J, Lindschou Hansen J, Simonsen E, Gluud C (2010) “The effect of cognitive therapy versus no intervention in patients with major depressive disorder. A systematic review of randomised clinical trials with meta-analysis and trial sequential analysis (protocol)” Available: http://www.ctu.dk Published 2010 Feb.
  21. 21. World Health Organization (1992) International Statistical Classification of Diseases an Related Health Problems (10th Revision) ICD 10.
  22. 22. American Psychiatric Association (1980) Diagnostic and Statistical Manual of Mental Disorders (DSM III). Washington, D.C..
  23. 23. American Psychiatric Association (1987) Diagnostic and Statistical Manual of Mental Disorders (DSM III-R). Washington, D.C..
  24. 24. American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorders (DSM IV). Washington, D.C..
  25. 25. Sofuoglu M, Sugarman DE, Carroll KM (2010) Cognitive function as an emerging treatment target for marijuana addiction. Exp Clin Psychopharmacol 18: 109–119.
  26. 26. Davidson KW, Rieckmann N, Clemow L, Schwartz JE, Shimbo D, et al. (2010) Enhanced depression care for patients with acute coronary syndrome and persistent depressive symptoms: coronary psychosocial evaluation studies randomized controlled trial. Arch Intern Med 170: 600–608.
  27. 27. Wilkins CH, Mathews J, Sheline YI (2009) Late life depression with cognitive impairment: Evaluation and treatment. Clinical Interventions in Aging 4: 51–57.
  28. 28. Howard I, Turner R, Olkin R, Mohr DC (2006) Therapeutic alliance mediates the relationship between interpersonal problems and depression outcome in a cohort of multiple sclerosis patients. 62: 1197–1204.
  29. 29. Gluud LL (2006) Bias in clinical intervention research. Am J Epidemiol 493–501.
  30. 30. Wood L, Egger M, Gluud LL, Schulz KF, Juni P, et al. (2008) Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336: 601–605.
  31. 31. Gluud C (2006) The culture of designing hepato-biliary randomised trials. J Hepatol 44: 607–615.
  32. 32. Kjaergaard L, Villumsen J, Gluud C (2001) Reported methodologic quality and discrepancies between large and small randomized trials in meta-analysis. Annals of Internal Medicine 135: 982–989.
  33. 33. Hamilton M (1960) A rating scale for depression. Journal of Neurology, Neurosurgery & Psychiatry 23: 56–61.
  34. 34. Bech AT (1961) An inventory for measuring depression. Arch Gen Psychiatry 561–571.
  35. 35. Montgomery SA, Asberg M (1979) A new depression scale designed to be sensitive to change. Br J Psychiatry 134: 382–389.
  36. 36. ICH GCP (1997) “Code of Federal Regulations & Guidelines Vol. 1” International Committee on Harmonization Philadelphia, US: Barnett International/PAREXEL, 1997.
  37. 37. The Nordic Cochrane Centre TCC (2008) Review Manager (RevMan), version 5.0. (Copenhagen): The Nordic Cochrane Centre.
  38. 38. Altman DG, Bland JM (2003) Interaction revisited: the difference between two estimates. BMJ 326: 219.
  39. 39. Thorlund K, Engstrøm J, Wetterslev J, Brok J, Imberger G, et al. (2011) “User manual for Trial Sequential Analysis (TSA)” Copenhagen Trial Unit Available: http://wwwctudk/tsa/files/tsa_manualpdf. Accessed 2011 Oct.
  40. 40. Wong DF (2008) Cognitive behavioral treatment groups for people with chronic depression in Hong Kong: a randomized wait-list control design. Depression and Anxiety 25: 142–148.
  41. 41. Teasdale JD, Fennell MJ, Hibbert GA, Amies PL (1984) pp. 400–406. Cognitive therapy for major depressive disorder in primary care.
  42. 42. Usaf SO, Kavanagh DJ (1990) pp. 51–70. Mechanisms of improvement in treatment for depression: Test of a self-efficacy and performance model.
  43. 43. Wright JH, Wright AS, Albano AM, Basco MR, Goldsmith LJ, et al. (2005) Computer-Assisted Cognitive Therapy for Depression: Maintaining Efficacy While Reducing Therapist Time. American Journal of Psychiatry 162: 1158–1164.
  44. 44. Wong FKD (2009) A six-month follow-up study of cognitive-behavioural treatment groups for Chinese people with depression in Hong Kong. 26: 130–140.
  45. 45. Simons AD, Levine JL, Lustman PJ, Murphy GE (1984) pp. 163–173. Patient attrition in a comparative outcome study of depression: A follow-up report.
  46. 46. Murphy GE, Simons AD, Wetzel RD, Lustman PJ (1984) Cognitive therapy and pharmacotherapy. Singly and together in the treatment of depression. Archives of General Psychiatry 33–41.
  47. 47. Scott C, Tacchi MJ, Jones R, Scott J (1997) pp. 131–134. Acute and one-year outcome of a randomised controlled trial of brief cognitive therapy for major depressive disorder in primary care.
  48. 48. Shamsaei F, Rahimi A, Zarabian MK, Sedehi M (2008) Efficacy of pharmacotherapy and cognitive therapy, alone and in combination in major depressive disorder. Hong Kong Journal of Psychiatry 18: 76–80.
  49. 49. Dozois DJ, Bieling PJ, Patelis-Siotis I, Hoar L, Chudzik S, et al. (2009) Changes in self-schema structure in cognitive therapy for major depressive disorder: a randomized clinical trial. Journal of consulting and clinical psychology 77: 1078–1088.
  50. 50. Hollon SD, DeRubeis RJ, Evans MD, Wiemer MJ (1992) pp. 774–781. Cognitive therapy and pharmacotherapy for depression: Singly and in combination.
  51. 51. Blackburn IM, Bishop S, Glen AIM (1981) The efficacy of cognitive therapy in depression: A treatment trial using cognitive therapy and pharmacotherapy, each alone and in combination. British Journal of Psychiatry 139(3): 181–189.
  52. 52. Scott MJ, Stradling SG (1990) pp. 1–19. Group cognitive therapy for depression produces clinically significant reliable change in community-based settings.
  53. 53. Ross M, Scott M (1985) An evaluation of the effectiveness of individual and group cognitive therapy in the treatment of depressed patients in an inner city health centre. Journal of the Royal College of General Practitioners 35(274): 239–42.
  54. 54. Blackburn IM, Bishop S (1983) pp. 609–617. Changes in cognition with pharmacotherapy and cognitive therapy.
  55. 55. Miller IW, Norman WH, Keitner GI, Bishop SB (1989) pp. 25–47. Cognitive-behavioral treatment of depressed inpatients.
  56. 56. Whisman MA, Miller IW, Norman WH, Keitner GI (1991) pp. 282–288. Cognitive therapy with depressed inpatients: Specific effects on dysfunctional cognitions. [References].
  57. 57. Miller IW, Norman WH, Keitner GI (1990) pp. 62–71. Treatment response of high cognitive dysfunction depressed inpatients.
  58. 58. Miller IW, Norman WH, Keitner GI (1989) pp. 1274–1279. Cognitive-behavioral treatment of depressed inpatients: Six- and twelve-month follow-up.
  59. 59. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P (2008) Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med 148: 295–309.
  60. 60. Walwyn R, Roberts C (2010) Therapist variation within randomised trials of psychotherapy: implications for precision, internal and external validity. Stat Methods Med Res 19: 291–315.
  61. 61. Jakobsen JC, Lindschou Hansen J, Simonsen E, Gluud C (2011) The effect of adding psychodynamic therapy to antidepressants in patients with major depressive disorder. A systematic review with meta-analyses and trial sequential analyses. Journal of Affective Disorders. Apr 16 (Epub ahead of print).
  62. 62. Chakraborty R, Chatterjee A (2007) Predictors of Suicide Attempt Among those with Depression in an Indian Sample: A Brief Report. The Internet Journal of Mental Health 4:
  63. 63. Bagby RM, Ryder AG, Schuller DR, Marshall MB (2004) The Hamilton Depression Rating Scale: has the gold standard become a lead weight? Am J Psychiatry 161: 2163–2177.
  64. 64. Fitzgibbon ML, Cella DF, Sweeney JA (1988) Redundancy in measures of depression. J Clin Psychol 44: 372–374.
  65. 65. Heo M, Murphy CF, Meyers BS (2007) Relationship between the Hamilton Depression Rating Scale and the Montgomery-Asberg Depression Rating Scale in depressed elderly: a meta-analysis. Am J Geriatr Psychiatry 15: 899–905.
  66. 66. Higginson IJ, Carr AJ (2001) Measuring quality of life: Using quality of life measures in the clinical setting. BMJ 322: 1297–1300.
  67. 67. Hopewell S, Wolfenden L, Clarke M (2008) Reporting of adverse events in systematic reviews can be improved: survey results. J Clin Epidemiol 61: 597–602.
  68. 68. Rose S, Bisson J, Churchill R, Wessely S (2002) Psychological debriefing for preventing post traumatic stress disorder (PTSD). Cochrane Database Syst Rev 2002(2): CD000560.
  69. 69. Keus F, Wetterslev J, Gluud C, van Laarhoven CJ (2010) Evidence at a glance: error matrix approach for overviewing available evidence. BMC Med Res Methodol 10: 90.