http://orcid.org/0000-0003-1100-0260
Figures
Abstract
Digitally delivered interventions for depression vary in many aspects, including their therapeutic orientation, depth of content, interactivity, individual tailoring, inclusion of multimedia, cost, and effectiveness. However, their effectiveness is rarely examined in intervention-specific meta-analyses. An earlier meta-analysis of eight randomized controlled trials (RCT) demonstrated the effectiveness of a tailored, integrative digital intervention (deprexis), which is delivered via the Internet. This updated meta-analysis of twelve deprexis-specific RCT with a total of N = 2901 participants confirmed the effectiveness of deprexis for depression reduction at post-intervention (g = 0.51, 95% CI: 0.40–0.62, I2 = 26%). Results were analogous when study quality, screening and randomization procedure were taken into account. Clinician guidance, developer-involvement, setting (community vs. clinical), and initial symptom severity did not have statistically significant effects on the effect size, and there was no evidence of publication bias. Thus, these findings demonstrate that deprexis can facilitate clinically relevant reduction of depressive symptoms over 8–12 weeks across a broad range of initial symptom severity, and that the intervention can be combined with other forms of depression treatment. There is now a need to study the intervention’s implementation in routine care settings as well as its long-term effectiveness and cost-effectiveness in diverse cultural and linguistic settings.
Citation: Twomey C, O’Reilly G, Bültmann O, Meyer B (2020) Effectiveness of a tailored, integrative Internet intervention (deprexis) for depression: Updated meta-analysis. PLoS ONE 15(1): e0228100. https://doi.org/10.1371/journal.pone.0228100
Editor: Daniel Vigo, Simon Fraser University, UNITED STATES
Received: September 12, 2019; Accepted: January 7, 2020; Published: January 30, 2020
Copyright: © 2020 Twomey et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data underlying the results presented in the study have been published and are available in the respective journal articles of the twelve individual randomized controlled trials included in this meta-analysis.
Funding: Funding Statement: Gaia AG, Hamburg, provided support in the form of salaries for authors [B.M. and O.B.], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.
Competing interests: Competing Interests Statement: Two of the authors (B.M. and O.B.) are employed by Gaia AG, Hamburg, the company that developed, owns and operates the Internet intervention examined in this article. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
The global treatment gap for depression ranges from 72% in high-income countries to 93% in low-income countries, and initiatives to scale up effective treatments are likely to be cost-effective because untreated depression is often associated with poor health and increased disability risk [1]. Internet-based depression treatments are promising in this regard because they can be scaled up easily, and recent meta-analyses have shown that even self-guided Internet interventions can achieve clinically relevant effects on depressive symptom reduction (g = 0.27, NNT = 6.58) [2], which may be augmented when additional clinician support is provided [3].
Despite such encouraging findings, caution is warranted for several reasons, and sweeping conclusions suggesting that all Internet interventions for depression are effective would be misguided. Of note, meta-analytic evidence showing that such interventions are effective on average does not mean that these interventions are all equally effective or, indeed, that any one intervention is effective at all. Internet interventions for depression differ substantially in their accessibility, therapeutic orientation, depth of content, interactivity, individual tailoring, inclusion of multimedia, self-monitoring tools, homework assignments, and cost [4]. Worryingly, many such interventions for depression are not evidence-based because they are never evaluated in randomized clinical trials (RCTs); this was the case for 20 out of 32 identified programs in a recent review [4].
Given this substantial heterogeneity among Internet interventions for depression, one might expect that they also differ in terms of their effectiveness, although head-to-head trials are rare. In one large trial that included two Internet interventions for depression (Beating the Blues and MoodGYM), neither of them was effective when added to primary care in the United Kingdom [5]. A recent meta-analysis of self-guided depression-focussed Internet interventions yielded a modest average effect of Hedges’ g = 0.27 across all programmes, but the effects of individual interventions ranged from -0.13 to 0.89, and there was significant moderate to high heterogeneity, suggesting that these interventions were not equally effective.
To examine the effectiveness of specific Internet interventions for depression, two intervention-specific meta-analyses have recently been completed. The first of these reported an average effect of g = 0.36 for MoodGYM, an Australian intervention based on cognitive behaviour therapy (CBT). Even though this average effect is above the recently proposed threshold of clinical relevance of d = 0.24 [6], the effect shrank to a non-significant level of g = 0.17 after adjusting for publication bias. Furthermore, moderator analyses suggested that the intervention was more effective in trials set in Australia compared to Europe-based trials, and larger effects were found when face-to-face guidance was provided, compared to remote guidance (telephone or email). Of note, 11 of the 12 included trials examined the intervention with some form of personal guidance, making it impossible to disentangle whether the intervention effects could be attributed to guidance or to the online programme, as such [7].
A second intervention-specific meta-analysis recently reported an average effect of g = 0.54 for deprexis, an integrative depression-focussed Internet intervention developed in Germany. This intervention differs in many ways from MoodGYM but is also broadly based on CBT principles. Unlike MoodGYM, deprexis also contains interactive modules on topics such as positive psychology, interpersonal skills, or coping with difficult childhood memories; more detailed descriptions are provided in several other articles [8–11]. The deprexis intervention is also unique in the sense that it engages users in ‘simulated dialogues’, such that users can continuously select from predefined response options, and then custom-tailors subsequent content to individual user needs and preferences. The deprexis-specific meta-analysis was based on eight RCTs that had been published up to November of 2016.
The purpose of the present meta-analysis is to provide an update of the deprexis-specific meta-analysis, which seems important because four more trials have been published since then, including two that are unique because they examined the utility of adding deprexis adjunctively to inpatient or outpatient depression treatment. Furthermore, one of these new trials used an active control condition (a psychoeducational depression-focussed online programme), which is still rare because most trials of behavioural interventions for psychiatric conditions continue to use waitlist or treatment-as-usual control conditions [12, 13]. As in the previous meta-analysis, we examined several potential moderators of treatment effects, including the setting (clinical, community), developer-involvement, and provision of personal support. Additionally, we examined potential publication bias.
Methods
Eligibility criteria, literature search, data extraction and quality ratings
The present study was modelled closely after the previous deprexis-specific meta-analysis and, therefore, used the same eligibility criteria for study selection and equivalent literature search, data extraction and quality assessment methods [11]. In brief, all studies were eligible in which deprexis was used as an intervention for adults with elevated depression symptoms, and which included any kind of control condition. Waiting lists, delayed programme access, no treatment, treatment as usual (TAU), or active control interventions (computerized or not) were eligible. Only randomized controlled trials (RCTs) that focussed on the outcome of self-reported or clinician-rated depression measures were included, and any language was permitted.
Literature search and data extraction
We used the same search term (“deprexis”) that was used in the previous deprexis-specific meta-analysis [11] and searched five databases: PsycINFO, CINAHL Complete, MEDLINE, Science Direct and Web of Science. The final search was conducted on August 14th, 2019. We hand-searched reference lists of included articles to identify additional articles. Screening of all articles were undertaken by two authors (BM and OB), and data were managed using EndNote X7 (Thomson Reuters Corp.) and word processing software. For each study, we recorded the setting, country, screening method (e.g., questionnaire cut-off or diagnostic interview), demographic data, control condition, randomization method, guidance (none, clinician, technician), outcome measures, assessment time-points, trial dropout at primary endpoint, and treatment outcomes.
Quality assessment
As in the previous deprexis-specific meta-analysis [11], we used three of the seven quality criteria stipulated by the Cochrane Collaboration’s tool for the assessment of risk of bias [14]: random sequence generation, allocation concealment, and completeness of outcome data (when intention-to-treat analyses were used, data were deemed complete). Blinding from intervention knowledge was not used because the experimental conditions did not permit such blinding; blinding of outcome assessment was also not used because only self-report assessments were included; and selective reporting bias as well as other bias were deemed too ambiguous and, as in the previous meta-analysis, were not included as quality criteria. Only information described in the published articles was used in the quality ratings. Of note, the lead author of one of the studies that was included in both the previous deprexis meta-analysis and the current meta-analysis has questioned the quality ratings assigned to that study [15]; however, our response to these concerns has been published [16], and we decided to not alter the original quality ratings for the eight studies included in both meta-analyses.
Statistical analysis
Meta-analysis using a random effects model was performed, and all statistical analyses were conducted with Comprehensive Meta-Analysis (version 3.3.070). We calculated pooled mean effect sizes (Hedges’ g) with 95% confidence intervals (CI); following conventions, we regarded effect sizes of 0.2 as small, 0.5 as moderate, and 0.8 as large. Higgin’s I2 percentages were calculated to estimate heterogeneity and values of 25%, 50% and 75% were regarded as low, moderate and high heterogeneity, respectively. We analysed only data at the post-intervention time-point (8–12 weeks) as this was the only assessment time-point in most trials, and control participants often received access to the intervention after this time-point. As in the previous meta-analysis, when there was more than one study arm in which deprexis was used, the effects of these multiple arms were averaged and entered only once in the analysis to avoid double-counting.
We also performed sensitivity analyses in which we adjusted for study quality, depression screening, randomization method and publication bias. The ‘Trim and Fill’ procedure [17] was used to examine potential publication bias. Subgroup analyses were conducted to examine (a) whether the provision of personal guidance, (b) developer-involvement, (c) the setting from which the sample was recruited (community versus clinical treatment), (d) publication date, or (e) initial depressive symptom severity influenced the results.
Results
Study selection
Fig 1 shows the literature search flow. A total of 80 abstracts were screened after removal of duplicates. Screening agreement rate between the authors BM and OB was 100%. After excluding records because they were not RCTs, 12 studies were included in the meta-analysis [8–10, 18–26].
Study characteristics
Table 1 summarizes the study characteristics. In 5 of the 12 studies (42%), a deprexis developer was involved in the study and listed as an author. Whereas 10 studies recruited participants primarily from the community with invitational advertisements, 2 studies recruited patients from clinical treatment settings, including outpatient psychotherapy and multimodal inpatient depression treatment at a psychosomatic hospital. Seven of the 12 studies formally screened for the presence of elevated depressive symptoms or a diagnosis of a depressive disorder, whereas 5 studies required only subjective feelings of sadness or depression or did not screen participants for depressive symptoms. Sample sizes ranged from 27 to 1013, resulting in a total of 2901 across all studies. Except for one study [20], all other studies included more women than men, and the mean age ranged from 32 to 48.
None of the studies prohibited access to other care options, meaning that deprexis was always used adjunctively to treatment as usual (TAU). Control conditions were, therefore, usually a combination of heterogeneous TAU and waiting list in the sense that control group participants typically received access to deprexis after the post-intervention time-point. In one study [26], an active control condition was used (12 sessions of online psychoeducational material). In two studies [19, 26], all participants received structured depression treatments (outpatient psychotherapy, multimodal inpatient psychotherapy), and deprexis was used adjunctively to these treatments in the intervention groups only.
Self-report instruments of depressive symptom severity served as primary outcome measures in all studies, although some studies also included clinician-rating measures as secondary outcomes. Post-intervention data collection time-points ranged from 8 to 12 weeks, and study dropout rates in the interventions conditions ranged from 6% to 56%, with a weighted average of 27.84%. Eight of the 12 studies met all three quality criteria. We did not change the quality ratings reported for the eight studies in the earlier deprexis-specific meta-analysis [11].
Of the four new studies, the one study with lower quality ratings was somewhat of an outlier in several aspects: (a) it was published in German, (b) it was described by the authors as a pilot study rather than a full-scale RCT, (c) it was not registered in a trial registry, (d) it did not report intention to treat (ITT) analyses but only analyses based on completer data, (e) it was statistically underpowered because only 14 and 13 participants were in the intervention and control groups, respectively, which corresponds to statistical power of 0.24 for the detection of an effect of d = 0.5, based on a post-hoc power-analysis performed with G*Power 3.1 [27]). The descriptions of the randomization process and the concealment of allocation were not detailed enough to permit positive ratings on these dimensions, in our judgment.
Effectiveness of deprexis for reducing depressive symptoms
As shown in the forest plot (Fig 2), comparisons from 12 studies demonstrated the effectiveness of deprexis for depressive symptoms at post-intervention, with an effect size of g = 0.51 (95% CI: 0.40–0.62) and low heterogeneity (I2 = 26%).
Sensitivity analyses showed that removing lower quality studies (k = 3; Fuhr et al., 2018; Meyer et al., 2009; Moritz et al., 2012) did not reduce the effect size (g = 0.52, 95% CI: 0.38–0.66; I2 = 41%). Removing the two studies with weighted randomization did reduce the effect size slightly but not significantly (g = 0.42, 95% CI: 0.32–0.51; I2 = 0%). The funnel plot was symmetrical, indicating no publication bias (Fig 3).
Subgroup analyses showed that effect sizes were slightly larger when deprexis was provided with some form of clinician guidance, but the difference between guided versus unguided intervention use was not statistically significant. The effect size in studies with guidance (k = 5) was g = 0.57, 95% CI: 0.36–0.78; I2 = 66%, whereas the effect in studies without guidance (k = 7) was g = 0.47, 95% CI: 0.32–0.62; I2 = 0%. The effect size did not differ significantly between studies that involved intervention developers (k = 5) compared to those that did not (k = 7). The effect size in studies with developer-involvement was g = 0.57, 95% CI: 0.37–0.76; I2 = 62%, whereas in studies without developer-involvement it was g = 0.45, 95% CI: 0.30–0.60; I2 = 0%. Of note, 3 of 5 studies (60%) with developer-involvement also included clinician guidance, whereas this was true of only 2 of the 7 studies (29%) without developer-involvement. Thus, the slightly but non-significantly larger effects in studies with developer-involvement could be attributed to the provision of clinician support, which was also associated with slightly but non-significantly larger effects.
The effect size in the two studies that recruited patients from clinical settings (outpatient psychotherapy or inpatient depression treatment) did not differ significantly from studies that recruited participants from the community. The effect size in studies with participants recruited from the community was g = 0.52, 95% CI: 0.39–0.66; I2 = 39%, whereas it was g = 0.47, 95% CI: 0.24–0.70; I2 = 0% in studies with participants from clinical settings. The effect sizes observed in the four newly included studies was g = 0.46, 95% CI: 0.26–0.67; I2 = 0%, which did not differ significantly from the 8 studies already included in the previous meta-analysis, g = 0.53, 95% CI: 0.38–0.69; I2 = 52%. The effect size in eight studies that included samples with mild (k = 1) or moderate baseline depression severity (k = 7) was g = 0.49, 95% CI: 0.34–0.65; I2 = 43%, whereas in four studies with severely depressed samples, it was g = 0.55, 95% CI: 0.37–0.73; I2 = 0%. These effects did not differ significantly from each other.
Discussion
Summary of main findings
This meta-analysis of twelve studies confirmed the effectiveness of deprexis for the reduction of depressive symptoms over a period of 8 to 12 weeks, with an effect size of g = 0.51 (95% CI: 0.40–0.62). This effect size is well above the threshold for clinical significance, which has been proposed at the level of d = 0.24 [6]. The effect of g = 0.51 can be converted to a number-needed-to-treat (NNT) of 3.55, which may be a metric more familiar to many psychiatrists than Hedges’ g or Cohen’s d [28]. Thus, between 3 and 4 individuals would have to be treated with deprexis in order to achieve clinically relevant depression reduction that would not have occurred otherwise.
The effect size estimate is robust in the sense that sensitivity analyses showed that removing lower quality studies or studies with weighted randomization did not significantly alter this effect. Additionally, there was no publication bias as the funnel plot was symmetrical, and there was no developer-bias, as studies that were conducted independently did not differ significantly from studies in which developers were involved.
In line with the earlier deprexis meta-analysis, this updated meta-analyses showed a slight but not statistically significant difference between studies with versus without clinician guidance (d = 0.57 vs. 0.47). Thus, this body of evidence indicates that deprexis is associated with clinically relevant effects on depression reduction, even when no clinician guidance is provided. Further research is required to clarify under which conditions the provision of personal support could augment the intervention’s effectiveness.
In contrast to the previous meta-analysis, the present study also showed clinically relevant effects when deprexis was used as an adjunctive treatment to outpatient psychotherapy or multimodal inpatient depression treatment. Indeed, the effect size observed in these studies conducted in clinical treatment settings did not differ from that observed in studies with participants from community settings. In one of these studies, control group patients not only received inpatient treatment, which included face-to-face psychotherapy, but also an active control programme that included 12 sessions of depression-related psychoeducational content [26]. In that study, statistically significant and clinically relevant intervention effects on depression reduction were shown at discharge from the hospital and at 3 and 6 months follow-up [26, 29]. Consistent with meta-analytic evidence [30], deprexis was found to be equally effective in samples with mild to moderate and even severe depressive symptom severity. Thus, the intervention is relevant for patients with a broad range of symptom severity, and it can be combined with other treatments, including inpatient and outpatient psychotherapy as well as antidepressant medication. In severe depression, the combination of deprexis and antidepressants may yield particularly large effects [9].
Limitations and strengths
With 12 studies and a total of 2901 participants, this is the largest meta-analysis to date of the Internet intervention deprexis, and one of the largest of any depression-focused Internet intervention. Of note, even though the studies were conducted with a wide range of samples in various settings, heterogeneity was low (I2 = 26%), suggesting that this intervention has a relatively uniform effect across settings and populations. Whereas previous findings had been limited to adults recruited from community settings, the present analyses also included two methodologically sound studies with samples recruited from clinical treatment settings. The evidence showed that using deprexis adjunctively to outpatient therapy or inpatient depression treatment is effective, with effect sizes resembling those observed in community-based studies. A finding that is particularly relevant to clinicians with limited time is that deprexis appears to be effective even without the provision of clinician support; further research is needed to confirm this tentative yet promising finding. A limitation of the present analyses is that the focus was limited to short-term outcome, as we only analysed data collected between 8 and 12 weeks after baseline. Several studies have shown longer term effects of deprexis [8, 9, 23, 29], but for the purposes of this meta-analysis there was still an insufficient amount of data to examine the stability of intervention effects over six months or beyond.
The deprexis intervention is currently available in nine languages, but 11 of the 12 studies included in this meta-analysis were conducted in German-speaking countries and only one was set in the United States [10]. Of note, the post-treatment effect in that study was the second largest of all included studies (g = 0.81), even though only minimal personal support was provided (on average, one brief phone call and one e-mail over a period of eight weeks, only to participants who failed to engage with deprexis or complete questionnaires across a two-week period). Thus, further research is needed to replicate these promising results and examine the effectiveness of deprexis in different linguistic and cultural settings.
Comparisons with other studies
The effect size of g = 0.51 (NNT = 3.55) we observed in this meta-analysis is in line with a previous deprexis-specific meta-analysis that included only 2/3 of the 12 studies we examined. The data showed that the new studies did not yield different effect sizes than earlier studies, on average, suggesting that the intervention effect has not declined in recent years. By contrast, there is some evidence suggesting that the effects of certain psychotropic drugs or face-to-face CBT may be declining in more recent studies [31–33].
The current results also compare favourably in the context of other depression-focussed Internet interventions. For example, a meta-analysis on the effectiveness of MoodGYM yielded a smaller effect size of g = 0.36 (NNT = 5), which was reduced to a non-significant g = 0.17 (NNT = 10.4) after correcting for publication bias [7]. Additionally, this meta-analysis found that larger effects were only observed for MoodGYM in studies that involved face-to-face guidance, whereas effects were small (g = 0.23, NNT = 7.7) when remote guidance by telephone or email was provided, and there was no evidence to suggest that the programme was effective without any guidance. However, guidance does not always enhance the effects of Internet interventions, as several trials have not found larger effects when guidance was provided, and a meta-analysis suggested that the effects of guidance may be less pronounced than often assumed [34–36]. In our meta-analysis, the effect sizes for deprexis were in the clinically relevant range regardless of whether no guidance or some guidance was provided.
There is also broader evidence to suggest that an effect size of g = 0.51 (NNT = 3.55) compares favourably to other depression interventions, whether delivered via the Internet or not. For example, an individual patient-level meta-analysis of self-guided Internet interventions for depression yielded an average effect of g = 0.27 (NNT = 6.58) [2]. However, there was considerable heterogeneity (I2 = 71%), and study-specific effect sizes varied substantially. Indeed, 5 of the 13 studies in the meta-analysis by Karyotaki and colleagues had examined deprexis, and their average effect size was g = 0.58 (NNT = 3.14), contrasting with g = 0.28 (NNT = 6.41) for the other interventions. An effect of g = 0.51 compares favourably even to treatment with antidepressant medication or different types of psychotherapy, especially after correcting for publication bias and other methodological biases, such as investigator allegiance and type of control group, which may artificially inflate effects [6, 37–39]. For example, after adjusting for methodological biases, the effect of psychotherapy for depression has recently been estimated at g = 0.31 (NNT = 5.75) The same effect size of g = 0.31 has been reported for antidepressant medication after adjusting for publication bias [40], and the effect of combined psychotherapy and pharmacotherapy for depression has been estimated at g = 0.46 (NNT = 3.91) [6].
When comparing these effect sizes, it is important to keep in mind that control conditions often differ in trials investigating medication, psychotherapy, or Internet interventions, and these differences in comparators may influence the magnitude of effect sizes. Specifically, whereas antidepressants are often compared to pill placebo conditions, psychotherapy and Internet interventions are often compared to treatment as usual (TAU) or wait-list conditions [41–43]. In part, this may be due to the problem that constructing a “psychotherapy placebo” is fraught with conceptual difficulties, such as developing a seemingly plausible psychotherapeutic intervention that actually contains no “active ingredients” of psychotherapeutic effectiveness [44, 45]. Nevertheless, it is possible to compare psychotherapy or Internet interventions to more active control conditions than TAU or wait-list, and two deprexis studies have done so. In both of these studies, the effect of adjunctive use of deprexis was compared with relatively intensive forms of depression treatment (outpatient or inpatient depression treatment), and both trials yielded clinically relevant effects in favour of deprexis [19, 26]. Moreover, one of these studies [26], which was developer-independent, used an active control intervention (12 psychoeducational modules on depression), suggesting that the intervention effects are not limited to comparisons with TAU or wait-list control conditions.
The average dropout rate of 27.8% observed in this meta-analysis was lower than that reported in the MoodGYM meta-analysis (41.3%). Conceivably, dropout rates were relatively low because deprexis, as a tailored programme, adapts its content to individual users, which might provide a personalized experience that motivates them to continue. Of note, though, dropout rates varied drastically between different studies. In one study that involved weekly email contacts with clinicians, the dropout rate was zero as every participant completed post-intervention assessments, whereas in another study of mostly male slot machine gamblers with comorbid depression, a dropout rate as high as 56% was reported. A previous individual patient meta-analysis that included data from several deprexis studies suggested that male gender, lower educational level and comorbid anxiety predicted dropout risk [46], whereas a large deprexis trial showed that older age and greater depressive symptom severity were associated with reduced dropout risk [47]. Despite these intriguing emerging findings, more research is needed to illuminate the factors that predict adherence and dropout risk. Clinicians wishing to use deprexis in their practice would be well advised to consider these findings, anticipate adherence barriers, and take measures to minimize dropout risk.
Implications for clinical practice and research
In the previous deprexis meta-analysis, a call to action was to conduct more research on deprexis in patient samples drawn from clinical settings rather than general community samples. As noted above, this has now been addressed in two studies, and one of these recently reported that the clinically relevant effects observed at three months were still noted and even larger at 6 month follow-up [29]. Nevertheless, more research on the programme’s effectiveness in routine clinical settings, on the long-term stability of effects, health-economic effects, and on the applicability in different populations and global regions is still needed. Emerging evidence suggests that deprexis can be cost effective in the sense that its use may reduce the need for other costly treatments over time [48, 49]. Additionally, more is becoming known about treatment moderators, mediators, response predictors and patterns of change [50–54]. With this increasingly differentiated body of evidence, it should become possible to deploy interventions such as deprexis in such a way that clinical benefits for individual patients are maximized while minimizing costs and staff burden.
Given the fact that the vast majority of people affected by depression still do not have access to effective treatments, these findings suggest that effective and scalable interventions, such as deprexis, should be disseminated more broadly in order to help mitigate the treatment gap and improve the global quality of depression care [1]. Because deprexis is a fully automated digital intervention that does not burden clinicians, it can be implemented easily in different care settings. Given the robust evidence for deprexis, as shown in this updated meta-analysis, the challenge now is to implement the programme in relevant depression care settings across the globe, to continue to evaluate its effectiveness in the field, and to develop it further in order to maximize clinical benefits.
References
- 1. Chisholm D, Sweeny K, Sheehan P, Rasmussen B, Smit F, Cuijpers P, et al. Scaling-up treatment of depression and anxiety: a global return on investment analysis. The Lancet Psychiatry. 2016;3(5):415–24. pmid:27083119
- 2. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA psychiatry. 2017;74(4):351–9. pmid:28241179
- 3. Andersson G, Carlbring P, Titov N, Lindefors N. Internet Interventions for Adults with Anxiety and Mood Disorders: A Narrative Umbrella Review of Recent Meta-Analyses. The Canadian Journal of Psychiatry. 2019:0706743719839381.
- 4. Renton T, Tang H, Ennis N, Cusimano MD, Bhalerao S, Schweizer TA, et al. Web-based intervention programs for depression: a scoping review and evaluation. Journal of medical Internet research. 2014;16(9):e209. pmid:25249003
- 5. Gilbody S, Littlewood E, Hewitt C, Brierley G, Tharmanathan P, Araya R, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. Bmj. 2015;351:h5627. pmid:26559241
- 6. Cuijpers P, Turner EH, Koole SL, Dijke A, Smit F. What is the threshold for a clinically relevant effect? The case of major depressive disorders. Depression and anxiety. 2014;31(5):374–8. pmid:24677535
- 7. Twomey C, O’Reilly G. Effectiveness of a freely available computerised cognitive behavioural therapy programme (MoodGYM) for depression: Meta-analysis. Australian & New Zealand Journal of Psychiatry. 2017;51(3):260–9.
- 8. Meyer B, Berger T, Caspar F, Beevers C, Andersson G, Weiss M. Effectiveness of a novel integrative online treatment for depression (Deprexis): randomized controlled trial. Journal of medical Internet research. 2009;11(2):e15. pmid:19632969
- 9. Meyer B, Bierbrodt J, Schröder J, Berger T, Beevers CG, Weiss M, et al. Effects of an internet intervention (Deprexis) on severe depression symptoms: randomized controlled trial. Internet Interventions. 2015;2(1):48–59.
- 10. Beevers CG, Pearson R, Hoffman JS, Foulser A, Shumake J, Meyer B. Effectiveness of an internet intervention for depression in a US adult sample: A parallel-group pragmatic randomized controlled trial. Journal of Consulting and Clinical Psychology. 2017.
- 11. Twomey C, O’Reilly G, Meyer B. Effectiveness of an individually-tailored computerised cognitive behavioural therapy programme (Deprexis) for depression: a meta-analysis. Psychiatry Research. 2017.
- 12. Gold SM, Enck P, Hasselmann H, Friede T, Hegerl U, Mohr DC, et al. Control conditions for randomised trials of behavioural interventions in psychiatry: a decision framework. The lancet Psychiatry. 2017;4(9):725–32. pmid:28396067
- 13. Furukawa T, Noma H, Caldwell D, Honyashiki M, Shinohara K, Imai H, et al. Waiting list may be a nocebo condition in psychotherapy trials: A contribution from network meta‐analysis. Acta Psychiatrica Scandinavica. 2014;130(3):181–92. pmid:24697518
- 14.
Higgins JP, Green S. Cochrane handbook for systematic reviews of interventions: John Wiley & Sons; 2011.
- 15. Moritz S. A Reply to "Effectiveness of an individually-tailored computerised CBT programme (Deprexis) for depression: A meta-analysis" by Twomey and colleagues [Psychiatry Res. 256 (2017) 371–377]. Psychiatry research. 2018;263. pmid:29046227
- 16. Twomey C, O'Reilly G, Meyer B. Reply to Dr Moritz. Psychiatry research. 2018;263:283.
- 17. Duval S, Tweedie R. Trim and fill: a simple funnel‐plot–based method of testing and adjusting for publication bias in meta‐analysis. Biometrics. 2000;56(2):455–63. pmid:10877304
- 18. Berger T, Hammerli K, Gubser N, Andersson G, Caspar F. Internet-based treatment of depression: a randomized controlled trial comparing guided with unguided self-help. Cogn Behav Ther. 2011;40(4):251–66. Epub 2011/11/09. pmid:22060248.
- 19. Berger T, Krieger T, Sude K, Meyer B, Maercker A. Evaluating an e-mental health program ("deprexis") as adjunctive treatment tool in psychotherapy for depression: Results of a pragmatic randomized controlled trial. Journal of Affective Disorders. pmid:29154168
- 20. Bücker L, Bierbrodt J, Hand I, Wittekind C, Moritz S. Effects of a depression-focused internet intervention in slot machine gamblers: A randomized controlled trial. PloS one. 2018;13(6):e0198859. pmid:29883479
- 21. Fischer A, Schröder J, Vettorazzi E, Wolf OT, Pöttgen J, Lau S, et al. An online programme to reduce depression in patients with multiple sclerosis: a randomised controlled trial. The Lancet Psychiatry. 2015;2(3):217–23. pmid:26359900
- 22. Fuhr K, Fahse B, Hautzinger M, Gulewitsch MD. Erste Erfahrungen zur Implementierbarkeit einer internet-basierten Selbsthilfe zur Überbrückung der Wartezeit auf eine ambulante Psychotherapie. Psychother Psych Med. 2018;(EFirst). Epub 19.01.2018. pmid:29351710
- 23. Klein JP, Berger T, Schröder J, Späth C, Meyer B, Caspar F, et al. Effects of a Psychological Internet Intervention in the Treatment of Mild to Moderate Depressive Symptoms: Results of the EVIDENT Study, a Randomized Controlled Trial. Psychotherapy and psychosomatics. 2016;85(4):218–28. pmid:27230863
- 24. Moritz S, Schilling L, Hauschildt M, Schröder J, Treszl A. A randomized controlled trial of internet-based therapy in depression. Behaviour research and therapy. 2012;50(7):513–21.
- 25. Schröder J, Brückner K, Fischer A, Lindenau M, Köther U, Vettorazzi E, et al. Efficacy of a psychological online intervention for depression in people with epilepsy: a randomized controlled trial. Epilepsia. 2014;55(12):2069–76. pmid:25410633
- 26. Zwerenz R, Becker J, Knickenberg RJ, Siepmann M, Hagen K, Beutel ME. Online Self-Help as an Add-On to Inpatient Psychotherapy: Efficacy of a New Blended Treatment Approach. Psychotherapy and Psychosomatics. 2017;86(6):341–50. pmid:29131090
- 27. Faul F, Erdfelder E, Lang A-G, Buchner A. G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods. 2007;39(2):175–91. pmid:17695343
- 28.
Cuijpers P. Meta-analyses in mental health research. A practical guide. Amsterdam, the Netherlands: Pim Cuijpers Uitgeverij. 2016.
- 29. Zwerenz R, Baumgarten C, Becker J, Tibubos A, Siepmann M, Knickenberg RJ, et al. Improving the Course of Depressive Symptoms After Inpatient Psychotherapy Using Adjunct Web-Based Self-Help: Follow-Up Results of a Randomized Controlled Trial. Journal of medical Internet research. 2019;21(10):e13655. pmid:31651403
- 30. Bower P, Kontopantelis E, Sutton A, Kendrick T, Richards DA, Gilbody S, et al. Influence of initial severity of depression on effectiveness of low intensity interventions: meta-analysis of individual patient data. Bmj. 2013;346:f540. pmid:23444423
- 31. Leucht S, Corves C, Arbter D, Engel RR, Li C, Davis JM. Second-generation versus first-generation antipsychotic drugs for schizophrenia: a meta-analysis. The Lancet. 2009;373(9657):31–41.
- 32. Ljótsson B, Hedman E, Mattsson S, Andersson E. The effects of cognitive–behavioral therapy for depression are not falling: A re-analysis of Johnsen and Friborg (2015). 2017.
- 33. Johnsen TJ, Friborg O. The effects of cognitive behavioral therapy as an anti-depressive treatment is falling: A meta-analysis. Psychological Bulletin. 2015;141(4):747. pmid:25961373
- 34. Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on Internet-based mental health interventions—A systematic review. Internet Interventions. 2014;1(4):205–15. https://doi.org/10.1016/j.invent.2014.08.003.
- 35. Kordy H, Wolf M, Aulich K, Bürgy M, Hegerl U, Hüsing J, et al. Internet-delivered disease management for recurrent depression: a multicenter randomized controlled trial. Psychotherapy and psychosomatics. 2016;85(2):91–8. pmid:26808817
- 36. Titov N, Dear B, Staples L, Terides M, Karin E, Sheehan J, et al. Disorder-specific versus transdiagnostic and clinician-guided versus self-guided treatment for major depressive disorder and comorbid anxiety disorders: a randomized controlled trial. Journal of Anxiety Disorders. 2015;35:88–102. pmid:26422822
- 37. Cuijpers P, Karyotaki E, Reijnders M, Ebert DD. Was Eysenck right after all? A reassessment of the effects of psychotherapy for adult depression. Epidemiology and Psychiatric Sciences. 2018:1–10. Epub 02/28. pmid:29486804
- 38. Cuijpers P, Smit F, Bohlmeijer E, Hollon SD, Andersson G. Efficacy of cognitive–behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. The British Journal of Psychiatry. 2010;196(3):173–8. pmid:20194536
- 39. Cuijpers P, van Straten A, Bohlmeijer E, Hollon S, Andersson G. The effects of psychotherapy for adult depression are overestimated: a meta-analysis of study quality and effect size. Psychological medicine. 2010;40(2):211–23. pmid:19490745
- 40. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. New England Journal of Medicine. 2008;358(3):252–60. pmid:18199864
- 41. Arnberg FK, Linton SJ, Hultcrantz M, Heintz E, Jonsson U. Internet-delivered psychological treatments for mood and anxiety disorders: a systematic review of their efficacy, safety, and cost-effectiveness. PLoS One. 2014;9(5):e98118. pmid:24844847
- 42. Barth J, Munder T, Gerger H, Nüesch E, Trelle S, Znoj H, et al. Comparative efficacy of seven psychotherapeutic interventions for patients with depression: a network meta-analysis. Focus. 2016;14(2):229–43.
- 43. Benedetti F, Carlino E, Piedimonte A. Increasing uncertainty in CNS clinical trials: the role of placebo, nocebo, and Hawthorne effects. The Lancet Neurology. 2016;15(7):736–47. pmid:27106073
- 44. Kirsch I, Wampold B, Kelley JM. Controlling for the placebo effect in psychotherapy: Noble quest or tilting at windmills? Psychology of Consciousness: Theory, Research, and Practice. 2016;3(2):121.
- 45.
Gaab J, Locher C, Blease C. Placebo and psychotherapy: differences, similarities, and implications. International review of neurobiology. 138: Elsevier; 2018. p. 241–55. https://doi.org/10.1016/bs.irn.2018.01.013 pmid:29681328
- 46. Karyotaki E, Kleiboer A, Smit F, Turner DT, Pastor AM, Andersson G, et al. Predictors of treatment dropout in self-guided web-based interventions for depression: an ‘individual patient data’meta-analysis. Psychological medicine. 2015;45(13):2717–26. pmid:25881626
- 47. Fuhr K, Schröder J, Berger T, Moritz S, Meyer B, Lutz W, et al. The association between adherence and outcome in an Internet intervention for depression. Journal of Affective Disorders. 2018;229:443–9. pmid:29331706
- 48. Gräfe V, Berger T, Hautzinger M, Hohagen F, Lutz W, Meyer B, et al. Health economic evaluation of a web-based intervention for depression: the EVIDENT-trial, a randomized controlled study. Health economics review. 2019;9(1):16. pmid:31175475
- 49. Gräfe V, Greiner W. Internet Based Treatment of Depressive Symptoms–A Health Economic Evaluation of Costs and Benefits. Value in Health. 2017;20(9):A714.
- 50. Fuhr K, Schröder J, Berger T, Moritz S, Meyer B, Lutz W, et al. The association between adherence and outcome in an Internet intervention for depression. Journal of affective disorders. 2017.
- 51. Klein JP, Gamon C, Späth C, Berger T, Meyer B, Hohagen F, et al. Does recruitment source moderate treatment effectiveness? A subgroup analysis from the EVIDENT study, a randomised controlled trial of an internet intervention for depressive symptoms. BMJ open. 2017;7(7):e015391. pmid:28710212
- 52. Klein JP, Späth C, Schröder J, Meyer B, Greiner W, Hautzinger M, et al. Time to remission from mild to moderate depressive symptoms: One year results from the EVIDENT-study, an RCT of an internet intervention for depression. Behaviour research and therapy. 2017;97:154–62. pmid:28797829
- 53. Lutz W, Arndt A, Rubel J, Berger T, Schröder J, Späth C, et al. Defining and Predicting Patterns of Early Response in a Web-Based Intervention for Depression. Journal of medical Internet research. 2017;19(6).
- 54. Schröder J, Berger T, Meyer B, Lutz W, Späth C, Michel P, et al. Impact and change of attitudes toward Internet interventions within a randomized controlled trial on individuals with depression symptoms. Depression and Anxiety. 2018;0(0). pmid:29489038