Figures
Abstract
Background and aims
Psychodynamic psychotherapy (PDT) is an effective treatment for major depressive disorder (MDD), but not all clients with MDD can receive psychotherapy. Using the Internet to provide psychodynamic treatments is one way of improving access to psychological treatments for MDD. The aim of this randomised controlled trial was to investigate the efficacy of an Internet-based psychodynamic guided self-help treatment for MDD.
Methods
Ninety-two participants who were diagnosed with MDD according to the Mini-International Neuropsychiatric Interview were randomised to treatment or an active control. The treatment consisted of nine treatment modules based on psychodynamic principles with online therapist contact. The active control condition was a structured support intervention and contained psychoeducation and scheduled weekly contacts online. Both interventions lasted for 10 weeks. The primary outcome measure was the Beck Depression Inventory-II (BDI-II).
Results
Mixed-effects model analyses of all randomised participants showed that participants receiving Internet-based PDT made large and superior improvements compared with the active control group on the BDI-II (between-group Cohen's d = 1.11). Treatment effects were maintained at a 10-month follow-up.
Citation: Johansson R, Ekbladh S, Hebert A, Lindström M, Möller S, Petitt E, et al. (2012) Psychodynamic Guided Self-Help for Adult Depression through the Internet: A Randomised Controlled Trial. PLoS ONE 7(5): e38021. https://doi.org/10.1371/journal.pone.0038021
Editor: Amanda Bruce, University of Missouri-Kansas City, United States of America
Received: February 29, 2012; Accepted: April 27, 2012; Published: May 29, 2012
Copyright: © 2012 Johansson et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The study was sponsored in part by the Swedish council for working and life research, the Swedish research council, and Linköping University (Professor contract). The authors report no financial relationships with commercial interests. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Major depressive disorder (MDD) is a major health problem, which lowers the quality of life for the individual and generates huge costs for society [1], [2]. Only about half of the 12-month cases in the USA were receiving treatment for MDD and only 18–25% were adequately treated [3]. Several forms of psychotherapy have been found to be effective in the treatment of MDD [4]. Among these, cognitive–behavioural therapy (CBT) has a strong empirical base [4].
Several studies have found that it is possible to deliver CBT as Internet-based guided self-help [5], and an increasing number of studies show that this treatment format can be as effective as face-to-face CBT for mild to moderate MDD and anxiety disorders [6]. Guided Internet treatments have provided a way to reach out to more patients in a manner that in most cases requires less therapist time [7].
Psychodynamic psychotherapy (PDT) is another psychological treatment that is effective for depression [8]. However, it is not known if it is possible to deliver PDT for MDD as guided self-help via the Internet. To our knowledge, no trial on Internet-delivered PDT has been published. It is important to examine if Internet-delivered PDT is effective, both from a theoretical and a practical point of view, since patients may prefer it above CBT. The aim of this study was to investigate the efficacy of a 10-week psychodynamic treatment for MDD, delivered in the form of guided self-help via the Internet. We compared the treatment to an active control condition that consisted of psychoeducation and scheduled support, also given for 10 weeks via the Internet. Significant within-group effects were expected for both conditions, but the effects for the treatment group were expected to be larger on measures of depression.
Methods
Ethics statement
The study was approved by the Regional Ethics Board of Linköping, Sweden. Signed informed consent was obtained from all participants via the online treatment platform.
Participants and recruitment
The protocol for this trial and supporting CONSORT checklist are available as supporting information; see Checklist S1 and Protocol S1. Participants were recruited nationally through an advertisement in a major Swedish newspaper two weeks before the treatment began. Additional participants were recruited from a waiting list for another treatment trial for depression. The study was approved by the Regional Ethics Board of Linköping, Sweden. Written informed consent was obtained from all participants during the online screening. Inclusion criteria for the study were a) being at least 18 years old, b) having a total score in the range of 15 to 35 on the self-rated version of the Montgomery-Åsberg Depression Rating Scale (MADRS-S) [9], c) no assessed risk of suicidality (see below for details), d) if on medication, unchanged dosage of psychiatric medication during the three months preceding the screening, e) no concurrent psychological treatment, f) not having other primary disorders that needed different treatments or that could be affected negatively by the treatment, g) a diagnosis of MDD according to the DSM-IV, with current acute episode of depression or an episode in partial remission.
Applicants to the study were instructed to complete an online screening containing demographical questions and the outcome measures described below. A participant was contacted for a telephone-based diagnostic interview if he or she had completed the screening and met the initial inclusion criteria. In the telephone interview, diagnostic questions about depression and anxiety disorders were asked in addition to questions about use of medications and psychological treatments. Additionally, an assessment of suicidal ideation was conducted.
Six final semester-M.Sc. clinical psychology students who had been trained in the diagnostic procedures conducted the interviews. To ensure reliability and quality in the procedure, a psychiatrist was available for consultation during the entire assessment phase. Before a participant was included, the psychiatrist and the senior researcher reviewed the screening results and the interview protocol. Figure 1 shows the participant flow throughout the trial and reasons for exclusion. The demographic data are presented in Table 1.
Abbreviations: MADRS-S: Montgomery-Åsberg Depression Rating Scale-Self-rated version; M.I.N.I.: Mini-International Neuropsychiatric Interview; ITT: Intention-to-treat.
Outcome measures
Primary outcome measure.
The primary outcome measure was the Beck Depression Inventory-II (BDI-II) [10] that was administered pre-treatment, on a weekly basis during the entire treatment phase, at post-treatment and also 10 months after the treatment had ended.
Secondary outcome measures.
Other outcome measures were collected at pre-treatment, post-treatment and at a 10-month follow-up. The results from the online screening were used as pre-treatment assessment. In addition to the BDI-II, measures of depression included the self-rated version of the Montgomery-Åsberg Depression Rating Scale (MADRS-S) [9] and the 9-item Patient Health Questionnaire Depression Scale (PHQ-9) [11]. Two measures of anxiety were used-the Beck Anxiety Inventory (BAI) [12] and the 7-item Patient Health Questionnaire Generalized Anxiety Disorder Scale (GAD-7) [13]. Finally, life quality was measured using the Quality of Life Inventory (QOLI) [14].
Clinician-administered measures.
Psychiatric diagnoses (from the DSM-IV) were assessed using the Mini-International Neuropsychiatric Interview (M.I.N.I.) [15]. The M.I.N.I. is a diagnostic interview that, in contrast to several other diagnostic interviews, is completely structured, making it appropriate for other assessors than experienced psychiatrists [15]. At post-assessment, another structured telephone interview was conducted. The purpose of the interview was to give an estimation of global improvement, measured by the 7-point version of the Clinical Global Impression-Improvement (CGI-I) scale [16]. All interviews were conducted by the six psychology students described above, who at post-treatment were not blind to participant's condition. During the 10-month follow-up period, the participants were assessed with the CGI-I once again. All follow-up interviews were conducted by a final semester-M.Sc. clinical psychology student.
Procedure
For those participants included in the study, the results from the online screening were used as pre-treatment assessment. The outcome measures that were collected pre-treatment were also collected at post-treatment and at follow-up. All measures used have been shown to have good psychometric properties, with internal consistencies of at least α = .79. Details of this can be found in the respective references of the outcome questionnaires. The measures were administered via the Internet, which has been shown to be a valid format for questionnaires regarding depression and anxiety [17], [18].
The participants were allocated to the psychodynamic treatment or to the active control condition in a 1∶1 ratio using block randomisation. An independent person, separate from the staff conducting the study, handled the randomisation using an online randomisation tool.
Interventions
Psychodynamic treatment and therapists.
The psychodynamic treatment was given as guided self-help, with minimal text-based guidance provided on a weekly basis [7]. In all, there were nine treatment modules, totalling 167 pages of text. Participants were given gradual access to the self-help modules and had continuous online support from a therapist using a secure online messaging system, similar to encrypted e-mail.
The treatment modules were largely derived from the self-help book Make the leap [19] that is based on psychodynamic principles. To make the material suitable for depression, the text was adapted and an extra chapter was written, which contained a psychodynamic understanding of how depression is developed and maintained [20]. The overall focus of the treatment was on teaching the client how to see and break unhelpful affective, cognitive and behavioural patterns. The treatment was called SUBGAP, which stands for (1) Seeing unconscious patterns that contribute to emotional difficulties, (2) Understanding these patterns, (3) Breaking such unhelpful patterns, and (4) Guarding Against Patterns and/or relapses [19]. A detailed description of the treatment is provided in Figure 2. All treatment modules ended with an encouragement for the participants to try out the SUBGAP strategies described in the particular module and write to the therapists about the experiences from this. The therapists gave feedback on the clients' experiences and administered the gradual access to the modules. In general, feedback was given on Mondays, but the therapists were available to answer additional questions within 24 hours.
Scheduled online supportive treatment.
The group that served as the active control group received psychoeducation and scheduled online support delivered in the same online environment as the psychodynamic treatment. Similarly, this intervention lasted for 10 weeks. During the first week of support, all participants received text material on depression. The text was 15 pages long and contained general information about depression, including DSM-IV criteria, epidemiology and treatment alternatives. All participants were assigned to an individual therapist who provided the support. Every Monday the participants were contacted by the therapist and were asked questions about the previous week. The therapists were instructed to give support, but not to use any specific psychological techniques other than basic therapeutic skills such as empathic listening and asking further question to help the clients to express their experiences and emotions. In addition to the scheduled online support, the participants could send messages to the therapists at any time during the week and were then given response within 24 hours during weekdays. The intervention given to this group was similar to how non-directive supportive therapies have been described [4]. Non-directive supportive therapies have been shown to be effective for depression, but significantly less effective than other psychotherapies [4].
After the treatment period had ended, the participants in the support group were crossed over to treatment. They could then choose between the psychodynamic treatment and a previously developed CBT treatment [21]. The results from this treatment period are, however, outside the scope of this study.
Therapists.
The therapists were six final-semester students from a five-year M.Sc. clinical psychologist programme. All therapists had completed their clinical training as well as 16 weeks of internship. Each therapist was responsible for 7 to 8 treatment participants from the treatment group and an equal number of participants from the control group. Therapists were randomly allocated to participants, with the restriction of not having more than 8 participants from each group. For the entire duration of the study the therapists received continuous supervision from an experienced psychotherapist with psychodynamic orientation, who had previous experience of the psychodynamic treatment manual. Typically, supervision consisted of examination of specific online interactions as well as more general therapeutic issues. Clients from both groups were discussed during supervision. During treatment, the therapists also had the possibility to consult the psychiatrist, e.g. on medication issues or if a participant expressed suicidal ideation.
Data analyses
All analyses were performed using SPSS 19 (SPSS, Inc., Chicago, IL). Independent t-tests and χ2-tests were used to test for group differences in demographics, pre-treatment data and in clinical significant improvement. Differences between the psychodynamic treatment and the structured support were primarily investigated by modelling interaction effects of group and time. In order to adhere to the intention-to-treat principle, the continuous outcome variables were analysed using mixed effects models, given their ability to handle missing data [22]. All analyses used Maximum Likelihood estimation. Random intercept models were selected for all measures except for the BDI-II. Group, time and their interaction were included as predictors in these models. For the BDI-II, where weekly measures were available, several models were compared using available information criteria, and the model with best fit was chosen. This model included a fixed linear effect of time with a random intercept and slope. The covariance between the random intercept and slope was not significant, so it was not included in the model. Error terms across time were modelled with a first-order autoregressive covariance structure with heterogeneous variances. Differences in average rates of growth between the two groups were examined by a fixed effects interaction between group and time. Between-group differences at post-treatment were analysed using independent t-tests. Power analysis indicated an 89% chance of detecting a between-group effect size of d = 0.60 (α level = 0.05).
To investigate recovery after treatment and at follow-up, the BDI-II was used. Recovery was defined as a post-treatment BDI-II score ≤10. This definition is in line with previous clinical trials on depression (e.g. [23], [24]). Participants who did not provide post-treatment data or follow-up data were classified as non-recoverers.
Within- and between-group effect sizes (Cohen's d) were calculated by dividing the differences in means by the pooled standard deviations [25]. The between-group effect sizes can be interpreted as follows: an effect size in the range of 0.20–0.49 is small, while 0.50–0.79 is moderate, and an effect size over 0.80 is large [26].
Results
The treatment group and the support group did not differ significantly on any of the pre-treatment measures (all t's<1.47, all p's>.14). Additionally, there were no significant differences between the groups on any demographic data or current/past treatment with medication and/or psychological treatment. Results from the mixed-effects model analyses are presented below. Means, standard deviations and effect sizes within and between groups for all self-report measures are presented in Table 2.
Attrition and adherence
Four out of 92 participants (4.3%) did not provide post-treatment data. Three out of these and two additional participants (totalling 5.4%) were unreachable for the telephone interview and were classified as unimproved according to the CGI-I. In the 10-month follow-up, 39 participants from the treatment group (84.8%) provided data on the self-report measures and 38 (82.6%) were reached for the telephone interview. Once again, those unreachable were classified as unimproved on the CGI-I. Follow-up data from the control group was also collected, but is reported elsewhere.
The number of completed treatment modules was used as a measure of adherence in the treatment group. A module was considered to be finished only if the weekly discussion of the module was sent to the therapist. Two participants (4.3%) did not start the treatment at all. One (2.1%) stopped after the first module, one after the third, three (6.5%) after the fourth, two after the fifth and one after the sixth. In total, 36 out of 46 participants (78.3%) in the treatment group finished all modules.
Primary outcome measure
As illustrated in Figure 3, the psychodynamic treatment group displayed continuous within-group improvements throughout the trial on the BDI-II. As seen in Table 2, the effect size between the groups at post-treatment was large (Cohen's d = 1.11). There was a substantial within-group effect size in the structured support group as well, indicating an effect in both groups. Mixed-effect model analyses showed a significant interaction effect of group and time on the BDI-II (F(1, 109.8) = 37.2, p<.001). The post hoc t-test was significant (t(86) = 5.23, p<.001).
Secondary outcome measures
Measures of depression.
As seen in Table 2, there were large between-group effect sizes on the MADRS-S and the PHQ-9 as well. Mixed models analyses showed significant interaction effects of group and time (F(1, 92.1) = 15.2, p<.001 and F(1, 90.7) = 13.1, p<.001, for the MADRS-S and the PHQ-9 respectively). Post-hoc t-tests were all significant (all t's>4.05, all p's<.001).
Measures of anxiety and quality of life.
On the GAD-7, the mixed models analysis showed a significant interaction effect of group and time (F(1, 90.8) = 6.92, p<.05). The post-hoc t-test was significant (t(86) = 2.42, p<.05). The mixed models analyses conducted on the BAI and the QOLI did not reveal any significant interaction effects, although the p-values were close to significant (F(1, 90.0) = 3.28, p = .074 and F(1, 88.1) = 3.07, p = .083, for the BAI and the QOLI respectively. As seen in Table 2, both groups had substantial within-group effects on the BAI.
Recovery after treatment
There were differences in recovery rates (post-treatment BDI-II score of ≤10) between the groups at post-treatment. The intervention group had a significantly larger proportion of participants who recovered after treatment (n = 16; 34.8%) than the control group (n = 4; 8.7%), χ2(N = 92, df = 1) = 9.2, p<.01. At the 10 month follow-up, 25 out of 46 participants (54.3%) from the treatment group had recovered.
Clinical global improvement and adverse events
Of the 92 participants randomised, 42 from the treatment group and 45 from the control group were reached for a post-treatment telephone interview that gave an estimate of the clinical global improvement on the CGI-I [16]. Unreachable participants were classified as unimproved. In the treatment group, 24 participants (52.2%) were much or very much improved while this was only true for 13 (28.3%) in the support group. The difference was significant, χ2(N = 92, df = 1) = 5.47, p<.05. Two participants from each group were minimally worse and one from the control group was much worse and therefore classified as adverse events. Of the two treatment group participants, one did not begin treatment, while the other reported feeling worse since she did not understand how to make use of the material. In the 10-month follow-up interview, 27 participants (58.7%) were classified as much or very much improved.
Therapist time
As expected, the average therapist time per participant and week was larger in the treatment group compared to the support group (13.2 minutes compared to 4.5 minutes, t(90) = 8.57, p<.001). Importantly, this was the time that the therapists were logged on to the online treatment platform, i.e., not including time for between-session reflection.
Discussion
This trial tested the efficacy of a 10-week psychodynamic treatment for depression, given as guided self-help over the Internet. The treatment was compared to an active control in the form of a supportive treatment that contained psychoeducation and scheduled online contacts. The main finding was that participants who received psychodynamic treatment improved more than those who received the scheduled support. Between-group effect sizes on measures of depression were large and the treatment effects were maintained at the 10-month follow-up. The results indicate that psychodynamic guided self-help is effective in the treatment of depression and that it is possible to deliver psychodynamic therapy via the Internet.
When developing the psychodynamic guided self-help treatment, we made the assumption that the core ingredients of psychodynamic theory could be retained. Internet-based treatments still involves therapist contact, albeit different from face-to-face treatment. The treatment manual aimed to preserve the psychodynamic principles and was derived from a book by an experienced psychoanalyst [19].
One fundamental aspect of psychodynamic psychotherapy is the therapeutic relationship [27]. A therapist contact of 10 to 15 minutes per week and client may seem insufficient to establish a strong relationship. However, the therapists acted in a personal manner and aimed to build a strong therapeutic alliance, e.g. by using supportive techniques and by answering any messages within 24 hours. Although alliance was not directly measured in this study, it is known that a strong therapist-client alliance can be established in Internet-based treatments [28], even when the therapist contact is brief [29]. Consequently, this treatment seems to be more supportive than expressive, using the supportive-expressive distinction established by Luborsky [30]. Since all communication in the guided self-help treatment was in a medium similar to e-mail, it was neither possible for the therapists to do any in-session exploration of the clients' affective experience nor any interpretation of transference processes, which are interventions which distinguishes psychodynamic face-to-face treatments from e.g. cognitive-behavioural therapy [27].
The psychodynamic guided self-help treatment had a continuous focus on identifying affective, cognitive and interpersonal patterns that had led to problematic behaviour for the client. Furthermore, several treatment modules had an emphasis on past experiences as well as on interpersonal experiences. These are also typical elements of psychodynamic therapy [27]. While not including homework in the classical sense, the treatment modules contained descriptions of techniques that the participants were encouraged to apply in their own lives. This focus and encouragement of activities outside-of-session is traditionally associated with cognitive-behavioural treatments [31], but in actual practice many psychodynamic practitioners tend to use homework [32].
Ten weeks are indeed an unusually short duration for a psychodynamic treatment, making it reasonable to question if such a treatment is possible to conduct in such a short time. It is possible that the decrease in symptoms may have continued for the treatment group if the treatment had been longer than 10 weeks. However, results from the Sheffield studies have shown comparable effects between 8- and 16-week psychodynamic therapy, indicating that psychodynamic treatment indeed can be as short as 10 weeks and that longer therapy may not necessarily result in larger effects [33].
There are limitations that must be addressed. First, the participants were recruited from the community and we cannot be sure that this treatment would work in a clinical setting, e.g. an outpatient psychiatric facility. However, mean depression severity as measured by the BDI-II at intake (M = 26.51) is close to the limit of 29 that Beck proposes for severe depression [10], and the fact that a majority of the participants had a comorbid anxiety disorder seems to parallel clinical reality [34]. A second related concern was the large number of participants who had college- or university level education. This might bias generalizability of the results, since it is possible that guided self-help is especially well suited for educated clients. However, there are data indicating that 50% of patients seeking psychotherapy have some college education [35] and that educated patients may be more inclined to seek help for mental health problems [36].
As a third limitation, we allowed the participants to have a parallel treatment of medication if stable for three months, which was true for 25% of the participants. This is common procedure in previous trials from our group (e.g., [21]). Medication status was unrelated to change on the BDI-II in this trial, but we cannot rule out that there was a small additive effect of medication in the sample.
A fourth limitation that needs to be addressed concerns the therapists in the study who all were psychologists in training, albeit during the last semester of training in a five year program and under regular supervision. There are some indications in the literature that students may be less effective as therapists when conducting face-to-face therapy [37]. Therefore it is not possible to rule out that experienced psychotherapists would have performed even better. However, this can be contrasted with recent indications that a computer technician can conduct Internet-based CBT as good as a clinician [38]. These recent results call for further research on who can conduct guided self-help treatments.
A related concern is that psychologists in training conducted all diagnostic interviews. Although the M.I.N.I. was designed to be administered by non-experts [15], there is a possibility that the study sample could have been more adequately defined. The interviewers were all trained in the procedure, but there were no procedures to ensure inter-rater reliability, which is a further limitation. The lack of blinding is also an important limitation, as it may have biased the results on the clinical interviews. Future studies should make use of more formal training in diagnostic procedures and blinding of assessors.
There are some implications of this study in addition to the aforementioned. While a significant number of randomised controlled trials have been conducted investigating the effects of psychodynamic psychotherapy, there has been a recent call for high quality, adequately powered trials targeting specific disorders [39]. As pointed out by Connolly Gibbons, Crits-Christoph and Hearon [40], there is no trial on depression that demonstrates the superiority of a manualised psychodynamic monotherapy over a control condition or another treatment. In that respect, one implication of this study is that it adds to the empirical base of psychodynamic treatment of depression. This study indicates that psychodynamic treatment might be on par with other treatments for depression. However, before any further conclusions are drawn, the results need to be replicated in other sites. In addition, direct comparisons to other established bona fide treatments (e.g., cognitive–behavioural therapy) need to be conducted.
This study also shows that psychodynamic psychotherapy is possible to deliver as guided self-help, which naturally has implications for dissemination of psychotherapy in general. There is a lack of psychodynamic therapy in other treatment formats than individual and group. Using the Internet format of delivery makes it possible to reach individuals who lack access to therapists nearby, or who simply do not want to meet face-to-face.
The availability of psychodynamic treatments that are possible to deliver via the Internet may also influence therapists' attitudes towards Internet treatments. There are some indications that CBT therapists may have a more positive attitude towards Internet-based treatments than therapists with psychoanalytic orientation [41], [42]. Thus, one implication of this study is that psychodynamic/psychoanalytic therapists in general may become more positive towards this treatment modality, which in the long run could mean that more patients get access to evidence-based treatments.
In summary, this trial demonstrates the efficacy of a treatment that presents psychodynamic principles in self-help text, accompanied by encrypted e-mail contact with a therapist working in a supportive fashion and aiming to build a strong alliance. The findings from this study add to the empirical base of psychodynamic treatments for depression and indicate that it is indeed possible to conduct psychodynamic treatment as guided self-help via the Internet.
Supporting Information
Protocol S1.
Original study protocol, translated to English.
https://doi.org/10.1371/journal.pone.0038021.s002
(DOC)
Acknowledgments
We would like to thank Farrell Silverberg for letting us use his book as a base for the treatment. In addition we extend our thanks to Anna Nyblom for collecting follow-up data, Jonas Almlöv for translation, Thomas Berger and Brjánn Ljótsson for technical support, Hugo Hesser for methodological support and Sarah Vigerland for acting as a CBT-therapist after the treatment was completed.
Author Contributions
Conceived and designed the experiments: RJ SE AH ML SM EP SP MHL AR P. Carlbring P. Cuijpers GA. Performed the experiments: RJ SE AH ML SM EP SP MHL AR P. Carlbring P. Cuijpers GA. Analyzed the data: RJ P. Carlbring P. Cuijpers GA. Wrote the paper: RJ SE AH ML SM EP SP MHL AR P. Carlbring P. Cuijpers GA.
References
- 1. Ebmeier KP, Donaghey C, Steele JD (2006) Recent developments and current controversies in depression. Lancet 367: 153–167.
- 2. Smit F, Cuijpers P, Oostenbrink J, Batelaan N, de Graaf R, et al. (2006) Costs of nine common mental disorders: implications for curative and preventive psychiatry. J Ment Health Policy 9: 193–200.
- 3. Kessler RC, Berglund P, Demler O, Jin R, Koretz D, et al. (2003) The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R). JAMA 289: 3095–3105.
- 4. Cuijpers P, van Straten A, Andersson G, van Oppen P (2008) Psychotherapy for depression in adults: a meta-analysis of comparative outcome studies. J Consult Clin Psychol 76: 909–922.
- 5. Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N (2010) Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One 5: e13196.
- 6. Cuijpers P, Donker T, van Straten A, Li J, Andersson G (2010) Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med 40: 1943–1957.
- 7. Andersson G (2009) Using the Internet to provide cognitive behaviour therapy. Behav Res Ther 47: 175–180.
- 8. Driessen E, Cuijpers P, de Maat SC, Abbass AA, de Jonghe F, et al. (2010) The efficacy of short-term psychodynamic psychotherapy for depression: a meta-analysis. Clin Psychol Rev 30: 25–36.
- 9. Svanborg P, Åsberg M (1994) A new self-rating scale for depression and anxiety states based on the Comprehensive Psychopathological Rating Scale. Acta Psychiatr Scand 89: 21–28.
- 10.
Beck AT, Steer RA, Brown GK (1996) BDI-II, Beck depression inventory-Manual. Boston: Harcourt Brace.
- 11. Kroenke K, Spitzer RL, Williams JB (2001) The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 16: 606–613.
- 12. Beck AT, Epstein N, Brown G, Steer RA (1988) An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol 56: 893–897.
- 13. Spitzer RL, Kroenke K, Williams JB, Lowe B (2006) A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 166: 1092–1097.
- 14. Frisch MB, Cornell J, Villanueva M, Retzlaff PJ (1992) Clinical validation of the quality of life inventory: A measure of life satisfaction for use in treatment planning and outcome assessment. Psychol Assess 4: 92–101.
- 15. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, et al. (1998) The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiat 59: Suppl 2022–33.
- 16.
Guy W (1976) Clinical Global Impressions. ECDEU assessment manual for psychopharmacology, US department of health and human services. Rockville: NIMH.
- 17. Holländare F, Andersson G, Engström I (2010) A comparison of psychometric properties between internet and paper versions of two depression instruments (BDI-II and MADRS-S) administered to clinic patients. J Med Internet Res 12: e49.
- 18. Carlbring P, Brunt S, Bohman S, Austin D, Richards J, et al. (2007) Internet vs. paper and pencil administration of questionnaires commonly used in panic/agoraphobia research. Comput Hum Behav 23: 1421–1434.
- 19.
Silverberg F (2005) Make the leap: a practical guide to breaking the patterns that hold you back. New York: Marlowe & Co.
- 20.
Busch F, Rudden M, Shapiro T (2004) Psychodynamic treatment of depression. Washington, DC: American Psychiatric Pub.
- 21. Andersson G, Bergstrom J, Hollandare F, Carlbring P, Kaldo V, et al. (2005) Internet-based self-help for depression: randomised controlled trial. Br J Psychiatry 187: 456–461.
- 22. Gueorguieva R, Krystal JH (2004) Move over ANOVA: progress in analyzing repeated-measures data and its reflection in papers published in the Archives of General Psychiatry. Arch Gen Psychiat 61: 310–317.
- 23. Dimidjian S, Hollon SD, Dobson KS, Schmaling KB, Kohlenberg RJ, et al. (2006) Randomized trial of behavioral activation, cognitive therapy, and antidepressant medication in the acute treatment of adults with major depression. J Consult Clin Psychol 74: 658–670.
- 24. Kessler D, Lewis G, Kaur S, Wiles N, King M, et al. (2009) Therapist-delivered Internet psychotherapy for depression in primary care: a randomised controlled trial. Lancet 374: 628–634.
- 25.
Borenstein M (2009) Introduction to meta-analysis. In: Chichester , Sussex West, editors. U.K0. Hoboken: John Wiley & Sons.
- 26.
Cohen J (1988) Statistical power analysis for the behavioral sciences. In: Hillsdale , editor. N.J.: L. Erlbaum Associates.
- 27. Blagys MD, Hilsenroth MJ (2000) Distinctive Features of Short-Term Psychodynamic-Interpersonal Psychotherapy: A Review of the Comparative Psychotherapy Process Literature. Clin Psychol-Sci Pr 7: 167–188.
- 28. Knaevelsrud C, Maercker A (2007) Internet-based treatment for PTSD reduces distress and facilitates the development of a strong therapeutic alliance: a randomized controlled clinical trial. BMC Psychiatry 7: 13.
- 29. Klein B, Austin D, Pier C, Kiropoulos L, Shandley K, et al. (2009) Internet-based treatment for panic disorder: does frequency of therapist contact make a difference? Cogn Behav Ther 38: 100–113.
- 30.
Luborsky L (1984) Principles of psychoanalytic psychotherapy: a manual for supportive-expressive treatment. New York: Basic books.
- 31. Blagys MD, Hilsenroth MJ (2002) Distinctive activities of cognitive-behavioral therapy. A review of the comparative psychotherapy process literature. Clin Psychol Rev 22: 671–706.
- 32. Kazantzis N, Lampropoulos GK, Deane FP (2005) A national survey of practicing psychologists' use and attitudes toward homework in psychotherapy. J Consult Clin Psychol 73: 742–748.
- 33. Shapiro DA, Barkham M, Rees A, Hardy GE, Reynolds S, et al. (1994) Effects of treatment duration and severity of depression on the effectiveness of cognitive-behavioral and psychodynamic-interpersonal psychotherapy. J Consult Clin Psychol 62: 522–534.
- 34. Kessler RC, Merikangas KR, Wang PS (2007) Prevalence, comorbidity, and service utilization for mood disorders in the United States at the beginning of the twenty-first century. Annu Rev Clin Psychol 3: 137–158.
- 35. Vessey JT, Howard KI (1993) Who seeks psychotherapy? Psychother-Theor Res 30: 546–553.
- 36. Howard KI, Cornille TA, Lyons JS, Vessey JT, Lueger RJ, et al. (1996) Patterns of mental health service utilization. Arch Gen Psychiat 53: 696–703.
- 37. Cuijpers P, Van Straten A, Warmerdam L, Smits N (2008) Characteristics of effective psychological treatments of depression: a metaregression analysis. Psychother Res 18: 225–236.
- 38. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, et al. (2010) Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PLoS One 5: e10939.
- 39. Gerber AJ, Kocsis JH, Milrod BL, Roose SP, Barber JP, et al. (2011) A quality-based review of randomized controlled trials of psychodynamic psychotherapy. Am J Psychiat 168: 19–28.
- 40. Connolly Gibbons MB, Crits-Christoph P, Hearon B (2008) The empirical status of psychodynamic therapies. Annu Rev Clin Psychol 4: 93–108.
- 41. Mora L, Nevid J, Chaplin W (2008) Psychologist treatment recommendations for Internet-based therapeutic interventions. Comput Hum Behav 24: 3052–3062.
- 42. Wangberg SC, Gammon D, Spitznogle K (2007) In the eyes of the beholder: exploring psychologists' attitudes towards and use of e-therapy in Norway. Cyberpsychol Behav 10: 418–423.