Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Results of the Optimune trial: A randomized controlled trial evaluating a novel Internet intervention for breast cancer survivors

Abstract

Introduction

After the acute treatment phase, breast cancer patients often experience low quality of life and impaired mental health, which could potentially be improved by offering cognitive behavioural therapy (CBT) and addressing exercise and dietary habits. However, CBT and other behavioural interventions are rarely available beyond the acute treatment phase. Internet-based interventions could bridge such treatment gaps, given their flexibility and scalability. In this randomized controlled trial (RCT), we investigated the effects of such an intervention (“Optimune”) over three months.

Methods

This RCT included 363 female breast cancer survivors (age range = 30–70), recruited from the community, who had completed the active treatment phase. Inclusion criteria were: breast cancer diagnosis less than 5 years ago and acute treatment completion at least 1 month ago. Participants were randomly assigned to (1) an intervention group (n = 181), in which they received care as usual (CAU) plus 12-month access to Optimune immediately after randomization, or (2) a control group (n = 182), in which they received CAU and Optimune after a delay of 3 months. Primary endpoints were quality of life (QoL), physical activity, and dietary habits at three months. We hypothesized that intervention group participants would report better QoL, more physical activity, and improved dietary habits after 3 months.

Results

Intention-to-treat (ITT) analyses revealed significant effects on QoL (d = 0.27, 95% CI: 0.07–0.48) and dietary habits (d = 0.36, 95% CI: 0.15–0.56), but the effect on physical exercise was not significant (d = 0.30; 95% CI: 0.10–0.51).

Discussion

These findings suggest the effectiveness of Optimune, a new CBT-based Internet intervention for breast cancer survivors, in facilitating improvements in quality of life and dietary habits. Efforts to disseminate this intervention more broadly may be warranted.

Trial registration

ClinicalTrials.gov, NCT03643640. Registered August 23rd 2018, https://clinicaltrials.gov/ct2/show/NCT03643640.

Introduction

With an incidence of 2.09 million women each year, breast cancer is the most frequent type of cancer among women worldwide [1]. After the initial acute treatment phase, which often includes surgery, radiation and aggressive combination-chemotherapy, patients typically progress to a monitoring phase, during which they may receive lower intensity treatments with anti-hormones, aromatase-inhibitors or bisphosphonates [2]. Psychological support is often offered during the acute treatment phase but is rarely available in monitoring phases, even though between 25% and 52% of breast cancer survivors experience clinically significant distress, including fatigue, depression, anxiety and impaired quality of life, even years after acute phase treatment [37]. Moreover, psychological distress may be linked with chronic systemic inflammation, which in itself could be a predictor of a worse prognosis over time [810].

Both pharmacological treatments and psychological interventions, such as Cognitive Behaviour Therapy (CBT) and mindfulness meditation, could be used to treat clinically relevant depression or anxiety and potentially reduce inflammation among breast cancer survivors [1120]. Of note, meta-analytic evidence suggests that antidepressants are prescribed for as many as 23% of breast cancer survivors, despite some controversy as to whether selective serotonin re-uptake inhibitors (SSRI) interact with tamoxifen [21, 22]. There is some evidence that certain SSRI might alleviate depression because of their effects on cytokines such as IL-6 and TNF-α [20]. Conversely, it has also been observed that targeting inflammation with NSAIDs and cytokine-inhibitors could alleviate depressive symptoms [19].

Psychological interventions have also been found to improve well-being and potentially reduce inflammation. For example, mindfulness-based stress reduction (MBSR), CBT and supportive-expressive dynamic psychotherapy appear to reduce inflammatory markers (e.g., IL-6, TNF-α) in several diseases with comorbid depression [17, 18, 23]. A meta-analysis reported that mindfulness meditation reduces inflammatory biomarkers such as CRP or NF-κB but may be neutral with regard to other cytokines [16]. One trial also found that a Yoga intervention reduced both depressive symptoms and markers of inflammation [15]. Studies with cancer survivors also suggest that psychological interventions such as CBT and mindfulness-based treatment improve quality of life while reducing inflammation [1214].

Even though CBT and other psychological interventions could be beneficial for breast cancer survivors in the monitoring phase, a major challenge is that systematic survivorship programmes including psycho-oncological treatments are not always available, particularly in rural areas [24]. Barriers such as stigma concerns, scepticism regarding psychotherapy, time constraints, disease-related restrictions, lack of motivation, and perceived lack of necessity can also prevent patients from accessing these treatments [25]. Internet-based psychological interventions are often regarded as promising in this context because they could potentially deliver effective support in a flexible and efficient manner [26, 27]. Many randomized controlled trials (RCT) have demonstrated the efficacy of Internet interventions for a broad spectrum of mental health conditions as well as for illness management and health-related behaviour change [2830]. However, there are also trials showing that some interventions are less effective than others, and some may even be harmful [31, 32]. Thus, it is necessary to examine the effects of each specific Internet intervention in methodologically adequate trials [30, 33].

As described in the study protocol [34], we developed an Internet intervention to provide psychological support for breast cancer survivors, named Optimune. This intervention was developed with the same technology and by the same multidisciplinary team that has developed a range of other evidence-based Internet interventions for conditions such as depression, anxiety disorders, harmful drinking, and psychological support for patients with multiple sclerosis and epilepsy [3541]. Optimune contains 16 modules spanning topics such as stress management, emotion regulation, healthy dieting and regular exercise after breast cancer treatment, sleep management, and CBT techniques to improve mental health (see Methods section for a more detailed description). Given this broad range of content, Optimune can be described as a CBT-based, holistic Internet intervention. Optimune aims to engage users in a broad array of CBT methods and behaviour change techniques (see intervention description below) in order to improve their quality of life and to facilitate the likelihood that they adopt habits that support their immune health, particularly dietary and exercise habits. By engaging in these exercises, users could potentially learn how to manage or reduce stress and how to adopt and maintain healthier dietary and exercise habits.

The goal of this pragmatic RCT was to test effects of Optimune over the course of three months on quality of life (QoL) and on two relevant health behaviours: dietary habits and physical exercise (primary endpoints). Trial success was defined a priori [34] as showing a significant effect on at least one of these endpoints at 3 months because improvements in some modifiable lifestyle factors are better than none, and even partial improvements in health behaviours or quality of life could be valuable outcomes in their own right, which might lead to improvements in immune function or disease prognosis. Secondary outcomes included insomnia, fear of cancer recurrence, cancer-related emotional stress, depression and anxiety symptoms. We also examined the subjective utility of the intervention as well as potential adverse effects.

The intervention is intended to be used adjunctively to other treatments patients may receive; therefore, in this trial it was offered in addition to care-as-usual (CAU). The relevant control group was CAU-only because we were interested in examining intervention effects under routine care conditions [42, 43]. The primary hypothesis was that participants randomized to the intervention, compared to those in the control condition, would have better quality of life as well as healthier dietary habits and physical exercise habits at 3 months. To examine the stability of effects, we also collected data at a follow-up time-point 6 months after baseline.

Methods

The ethics committee of the IfADo—Leibniz-Institut für Arbeitsforschung at the TU (Technical University) Dortmund, Germany, approved of the study.

Study design

In this parallel-groups RCT, participants were randomly assigned to (1) the intervention group (IG), in which they immediately received access to Optimune, on top of CAU, or (2) the control group (CG), in which they received CAU only. Data were collected at baseline, 3 months (time-point of primary interest), and 6 months post-baseline (follow-up). Participants in the CG were offered access to Optimune after 3 months; therefore, the CG is a combination of CAU and wait-list (CAU/WL). Participants were not blinded to group assignment, given the design of the intervention. Simple randomization (no blocks or stratification) was performed by the Principal Investigator with a computer-generated sequence. Concealed allocation to conditions was ensured because research team members who informed participants of the randomization result did not have access to any participant data and were not informed of the randomization result until after inclusion of a participant.

Recruitment and assessment

Participants were recruited in Germany from a broad range of settings, including Internet advertisements, treatment clinics, patient associations, support groups, and health insurance companies. Potentially interested women were referred to the study homepage, where they could sign up with their name and email address to request further information. Potential participants then received an email invitation with detailed information about the study and an invitation to an online baseline assessment. All participants were required to provide online informed consent, and those who were potentially eligible had to provide a discharge letter from their oncology treatment centre or clinic in order to verify diagnoses and therapies. Final inclusion into the study occurred after research team members had confirmed receipt of such a letter and had checked all inclusion and exclusion criteria. Randomization took place immediately after inclusion. In case participants assigned to the IG failed to register to Optimune within one week after randomization, they were contacted by the blinded technical support team. Email invitations were sent for the 3 and 6 months assessments, and up to two reminders were sent to those who did not respond to the initial invitation.

Inclusion criteria

Inclusion criteria were female sex, having received a breast cancer diagnosis less than 5 years ago, and having completed acute treatment for breast cancer (such as surgery, chemotherapy or radiation treatment) at least 1 month prior to study entry. Concurrent treatment with anti-hormone therapy such as tamoxifen, aromatase-inhibitors or bisphosphonates was permissible. Participants were required to be between 30 and 70 years of age, to provide informed consent, to be able to speak and read German, and to provide treatment discharge letter to verify diagnoses and treatments.

Intervention: Optimune

The CBT-based, holistic Internet-based intervention Optimune was developed by a team of clinical psychologists, CBT therapists, physicians, software engineers, graphic artists, and professional speakers, among others, affiliated with GAIA in Hamburg, Germany. GAIA is an SME (small to medium enterprise) specialized on the development of digital treatment programmes, with a track record of more than 20 such programmes, many of which have been evaluated in independent RCTs [3541, 44]. Optimune is based primarily on established CBT techniques targeting depression, anxiety, and fatigue. The intervention also engages users in therapeutic techniques that have been shown to have beneficial effects on immune system functioning and inflammation, including sleep and stress management (e.g., mindfulness-based techniques) and health behaviour change (dietary habits and physical activity advice). The dimensions addressed in the programme are consistent with the German treatment guidelines for breast cancer survivors. The programme also includes a total of 240 citations to relevant scientific literature, with patient-friendly summaries (annotated bibliography). All therapeutic techniques are conveyed via the format of a “simulated dialogue” in which users read information or listen to audio clips and then have to choose one of several predefined response options. Subsequent content is continuously tailored based on these user responses. The programme also contains illustrations and photographs, pdf-summaries, and daily text messages that are intended to remind and motivate patients to use the programme. Optimune was developed with a responsive web-design approach and can be accessed via a password-protected, secure (https-encrypted) website (https://optimune.broca.io) with any computer or smartphone equipped with a contemporary web browser. Patient and expert feedback was sought continuously during the development process, and content as well as functions were iteratively refined based on this feedback. The intervention was produced on a proprietary software platform (broca®) developed by GAIA. It uses cloud computing with fast global access and is securely hosted in an ISO-27001-certified data centre located in Germany.

After registration with a personal 12-digit code, users can access Optimune for a period of one year, even though it is assumed that they can complete the programme much sooner. A total of up to 16 modules is offered, although there is no expectation or requirement for users to finish all modules. There is no fixed or generic sequence in which modules must be completed. The intervention uses algorithm-driven sequences to guide users through the programme, and they are invited to flexibly explore the content whenever they wish. A general recommendation offered to users is to complete at least 1 to 2 sessions of around 30 minutes per week. Module length varies depending on factors such as reading speed, individual path taken through the module, desire to explore greater or lesser depth, and decisions to listen or not listen to optional audio clips. Users are also encouraged to discontinue using the intervention if they feel that it is not helpful.

The 16 modules can be grouped into four content domains: (1) psychological well-being, (2) dietary coaching, (3) physical activity and exercise, and (4) sleep management. Functions, purpose and time-frame of the intervention are covered in an introductory module. Subsequent modules cover some psychoeducational content but primarily CBT-based exercises to enhance emotion regulation, improve well-being, ward off depression and anxiety, optimize coping with common concerns and hassles associated with breast cancer and its treatment, adhere to a healthy diet, exercise regularly, and use CBT techniques to overcome insomnia. Evidence-based behaviour change techniques such as goal setting, action planning, mental contrasting (of pros and cons or goal attainment obstacles), mental imagery exercises, case examples to encourage modelling, problem-solving, rehearsing implementation intentions, and positive self-statements are used in every module. Dimensions used to custom-tailor content include variables such as current stress level, dietary habits (e.g., vegetarian diet), or fitness and exercise habits and preferences. A more detailed description of the functions and content of Optimune is provided in the study protocol [34].

Outcome measures

Primary endpoints.

The primary endpoints were (1) overall quality of life, measured with the total score of the World Health Organization Quality of Life Questionnaire (WHOQOL-BREF, 26 items) [45]); (2) physical exercise, measured with the total score of the International Physical Activity Questionnaire (IPAQ, 27 items) [46]); and (3) dietary habits, measured with the total score of the Food Quality Questionnaire (FQQ, 20 items, see section below for further details). Trial success was defined a priori as demonstrating a significant intervention effect on at least one of these three primary endpoints [34].

Development of the FQQ. Whereas the WHOQOL-BREF and the IPAQ are well validated questionnaires, the FQQ was developed for this trial as a face-valid self-report measure of current dietary habits, with an emphasis on selected foods that are thought to either facilitate or reduce inflammation. The FQQ items were developed based on consensus discussions among the research team members, several of whom were involved in the development of Optimune. Two main criteria had to be met in order for an item to be included: (a) There had to be agreement that the respective food item was regarded as clearly healthy or unhealthy, according to information presented in Optimune and in line with current evidence [4749], and (b) each food item had to be common and easily available throughout Germany. Food items for which there was controversy or lack of clear consensus as to whether they can be considered healthy or unhealthy in terms of their effects on inflammation were excluded (e.g., dairy products, red wine, meat). After review of potential alternative questionnaires with established validity, the research team concluded that no current questionnaire met these criteria, which justified the development of the FQQ.

The FQQ contains two 10-item subscales: (1) healthy foods and (2) unhealthy foods. Instructions are: “I eat (or drink/consume)”. Respondents are asked to indicate on a 4-point Likert-type scale how often they typically eat each of 20 items, which are presented in pseudo-random order. Response options are (0) very rarely, (1) rather rarely, (2) rather often, and (3) very often. This study represents an initial effort to examine the reliability and validity of the FQQ; internal consistency and factor structure are presented in the results section. The German version of the FQQ is available upon request from the authors and can be used freely by others.

The 10 items composing the FQQ healthy foods subscale are: (1) Kernels and seeds (e.g. sesame, linseed, pine and sunflower seeds), (2) Pome fruit (e.g. apple, pear, quince), (3) Stone fruit (e.g. apricot, peach, cherry, olive, plum), (4) Berries (e.g. raspberry, blackberry, blueberry, currant), (5) Fruit vegetables (e.g. cucumber, zucchini, tomato, bell pepper), (6) Onion vegetables (e.g. onion, garlic, leek), (7) Dried or fresh herbs (e.g. parsley, coriander, rosemary, thyme, dill), (8) Fresh or powdered spices (e.g. pepper, cinnamon, ginger, turmeric, cumin), (9) Olive oil, (10) Fatty fish (e.g. herring, sprat, salmon, mackerel, tuna).

The 10 items composing the FQQ unhealthy foods subscale are: (1) White bread or white flour buns, (2) Cookies or biscuits, (3) Pastries or sweet cakes (e.g. muffins, poppy seed snails, almond croissants) (4) Milk chocolate or chocolate bars, (5) Candy or sweets (e.g. gummy bears, hard candy, liquorice, marshmallows), (6) Processed or pre-packaged foods (e.g. instant soups, canned foods, microwave dishes, pre-packaged dishes), (7) Hamburgers or cheeseburgers, (8) White-flour pasta dishes, (9) French fries (chips), (10) Sweetened drinks (e.g. lemonade, cola, sweetened fruit spritzer).

In addition to computing these two subscales, we also constructed a total FQQ score which constituted one of the three primary endpoints. For the total FQQ score, all items of the unhealthy food subscale were reverse-scored, such that higher values indicate greater consumption of healthy and less consumption of unhealthy foods.

Secondary endpoints.

Secondary outcomes were: (1) Cancer-related fatigue (Brief Fatigue Inventory, 9 items, BFI-9 [50, 51]), (2) Cancer-related emotional impact (Intrusion scale of the Impact of Event Scale-Revised, 7 items, IES-R [52]), (3) Depression (Patient Health Questionnaire, 9 items, PHQ-9 [53]), (4) Anxiety (Generalized Anxiety Disorder, 7 items, GAD-7 [54]), (5) Fear of progression (Fear of Progression questionnaire, 12 items, PA-F12 [55]), (6) Insomnia symptoms (Insomnia Severity Index, 7 items, ISI [56]), and (7) Subjective usefulness of the programme (single item, Net promoter score [57]), plus items to assess potential negative effects of the programme (23 items, Inventory for the Assessment of negative effects of psychotherapy—online version [German Inventar zur Erfassung negativer Effekte von Psychotherapie, INEP-ON] [58]. Further detail on all outcome measures and their psychometric properties is provided in the study protocol [34].

Sample size calculation and analysis

As described in the study design paper [34], an a priori power analysis was performed with g*power (Version 3.1.9.2) [59] and showed that a sample size of 346 participants would be required to detect a small to medium effect (Cohen’s d = 0.35) with a power of 0.80 and an alpha level of 0.0167 (Bonferroni adjustment because of the three primary endpoints; 0.05/3). The anticipated effect size of Cohen’s d = 0.35 was deemed realistic based on previous reviews of psychosocial outcomes in cancer patients [6062]. Therefore, we aimed to enrol 180 participants per group.

Missing data were replaced using multiple imputation (100 imputations, 50 iterations) based on sociodemographic data and available outcomes. Intention to treat (ITT) analyses were performed with data from all randomized participants. Results of per protocol (PP) analyses are presented in the S1 and S2 Tables and used data from all control group participants plus data from intervention group participants who had used the intervention for at least 60 minutes on at least four separate occasions. Analyses of Covariance (ANCOVA) were conducted for the primary outcomes to test for between-group differences at the 3-month time-point. Baseline values of the respective outcome were used as covariates in these analyses. For analyses involving the primary outcomes, a Bonferroni adjustment was used to address multiplicity and control for family-wise error rate (p < 0.05/3 = p < 0.017) [63]. For all other analyses, a p-level of 0.05 (two-tailed) was used. A sensitivity analysis was conducted for the primary outcomes using a conservative multiple imputation approach (reference-based multiple imputation using the jump to reference [J2R]) assumption rather than the missing at random (MAR) assumption. To examine potential dose-response relationships, correlational coefficients between usage and pre-post change in primary outcome variables were inspected. Statistical analyses were performed with R-Studio 1.3 (incl. mice package 3.11.0), BlueSky Statistics 7.0, and with STATA-SE Version 16.

Results

Description of trial participants

Participants were recruited between October 25, 2018 (first patient in) and April 6, 2020 (last patient out). Demographic characteristics are presented in Table 1. The mean age of the women participating in this study was 50 years (range = 30–70). Most participants had attained a relatively high educational level, with 42% holding a university degree. The majority (70%) of participants were working full-time (31%) or part-time (39%), and most were married (69%) or in a relationship (14%). A minority of the women in the sample (29%) were currently in psychotherapy. The prototypical participant was a 50-year-old, well educated, married woman who is not undergoing psychotherapy and had not experienced tumour recurrence in the preceding three months.

Intervention delivery and drop-out

In total, N = 363 participants were randomized to the IG (n = 181) or the CG (n = 182). The drop-out rate at 3 months (T1) in the total sample was 16% (22% in the IG and 9% in the CG). The study flowchart (Fig 1) provides further detail on reasons for dropout.

thumbnail
Fig 1. Study flowchart.

Trial design and participant flow. TAU (Treatment as Usual), WL (waitlist).

https://doi.org/10.1371/journal.pone.0251276.g001

Psychometric properties of the FQQ

Internal consistency of the total FQQ (Cronbach’s alpha = 0.81) as well as the two subscales healthy foods (alpha = 0.75) and unhealthy foods (alpha = 0.80) was acceptable to good. A moderate inverse correlation between the healthy and unhealthy foods subscales indicated that, as expected, participants who tend to eat healthy foods are somewhat less likely to also consume unhealthy foods, and vice versa (r = -0.32, p < 0.001). A principal components analysis (PCA) with varimax rotation, which is recommended for such exploratory analyses [64], suggested that four factors could be distinguished: (1) Vegetables and fatty fish (7 items), (2) Sweets (4 items), (3) Fast food (6 items), (4) Fruits and berries (3 items). When variables based on these factors were entered into a second-order PCA with varimax rotation, a clear two-factor solution emerged: (1) Healthy foods (vegetables and fatty fish; fruits and berries), and (2) unhealthy foods (sweets; fast food). The total (20 items) FQQ scale and both subscales (10 items each) were approximately normally distributed, with no problematic skew or kurtosis (values ranged from -0.44 to 0.76). Further details of these analyses are available from the authors. In summary, given the high levels of internal consistency, plausible inverse correlation between the subscales, interpretable factor structure, and normal distribution, the FQQ was deemed to be an acceptable outcome measure for the analyses reported below.

Study outcomes

Primary outcomes.

ITT analyses showed significant effects on two of the three primary outcome scales at T1 (Table 2): Quality of life (95% CI: 0.07–0.48) and dietary habits (95% CI: 0.15–0.56). The third primary outcome, physical exercise, did not attain significance (95% CI: 0.10–0.51). Post-treatment between-group effect sizes ranged from 0.27 to 0.36 (Cohen’s d), corresponding to an NNT of 6.58 to 5.00 [65]. Fig 2 depicts the pre to post changes in primary outcome scales in both groups. Table 1 provides a summary of the subscale analyses of the primary outcome measures, revealing the relatively strongest effects for psychological quality of life (d = 0.42), healthy dietary habits (d = 0.36), and aerobic exercise (d = 0.32). Per-protocol (PP) analyses confirmed significant intervention effects on quality of life and dietary quality (see S1 Table).

thumbnail
Fig 2. Primary endpoints.

Effects of Optimune on quality of life, dietary habits and physical exercise habits. Mean total scores (error bars represent SEM) are shown for each outcome measure before the intervention (T0, baseline) and at the end of the intervention (T1, 3 months). Treatment had significant effects on (A) quality of life and (C) dietary habits, whereas there was no effect on (B) physical exercise. MET (metabolic equivalent task, in minutes per week), IG (intervention group, n = 181), CG (control group, n = 182).

https://doi.org/10.1371/journal.pone.0251276.g002

Secondary outcomes.

As summarized in Table 3, significant intervention effects were observed in the ITT analyses for four of the six secondary outcome scales: insomnia, cancer-related fatigue, general anxiety symptoms, and depressive symptoms. By contrast, intervention effects were not significant for cancer-related emotional stress and fear of tumour progression. PP analyses (see S2 Table) suggested a highly similar pattern of results, with significant effects on five of the six outcome scales (all but general anxiety).

Stability of effects in the IG and improvements in the CG after 3 months.

We inspected the stability of intervention effects by examining changes in the IG between 3 and 6 months, using data from the ITT analyses. We first considered data from IG participants only. Given that CG participants could access the programme after 3 months, their data were examined separately.

Among IG participants, the majority of treatment effects remained stable or improved further between 3 and 6 months (see S3 Table). Specifically, such stability was observed with regard to total QoL (also physical, psychological, and environment subscales), three of four physical exercise scales (anaerobic, walking, and sitting), the two dietary habits subscales, cancer-related emotional stress, and depressive symptoms. IG participants experienced further improvements between 3 and 6 months on several scales: the social quality of life subscale, total dietary habits, insomnia symptoms, general anxiety symptoms, and fear of cancer progression. However, participants in the IG also reported some reductions in physical activity (total scale and aerobic subscale) between 3 and 6 months.

Among CG participants, who received access to the intervention after 3 months, significant improvements were observed on most scales (see S3 Table): Total quality of life (also the physical, psychological, and environment subscales), total dietary habits (also both subscales), insomnia, cancer-related fatigue and emotional distress, depressive symptoms, general anxiety, and fear of cancer progression. The only exception was that physical activity levels remained stable among CG participants during this period. Overall, improvements were observed across a broad range of outcomes after control group participants were able to access the intervention.

Supplemental analyses suggesting immunological response.

At baseline, the proportion of participants reporting at least one episode of a respiratory infection, common cold or influenza did not differ between the CG (41%) and IG (40%), χ2(1) = 0.77, p = 0.83. At 3-months, 38% of IG participants and 48% of CG participants reported at least one such episode, but this was not a statistically significant difference, χ2 (1) = 3.28, p = 0.08. At baseline, the proportion of participants reporting at least one unplanned physician visit did not differ between the IG (56%) and the CG (54%), χ2 (1) = 0.14, p = 0.75. However, at 3-months, 43% of IG participants versus 56% of CG participants reported at least one unplanned physician visit, which was a significant difference, χ2 (1) = 5.30, p = 0.02.

Adverse effects and user satisfaction.

The evaluation of adverse events showed that an undesirable event related to the use of Optimune was reported by nine patients. In eight cases, this was increased distress (e.g., feeling upset because the content reminded the patient of the cancer diagnosis, feeling pressured to do exercises or change habits, feeling frustrated because response options did not match the personal situation). In one case, a patient reported an unintended reduction of body weight, which she attributed to eating less meat, sweets, juices and chips. None of the reported events were judged to be serious adverse events. Control group participants were not asked about adverse events. Participants were also asked to indicate whether they would recommend Optimune to a friend or colleague (“net promoter score” or NPR [57]). Responses were made on an 11-point scale ranging from 0 (not at all likely) to 10 (very likely). Of the 134 participants who responded to this item, 9% were not satisfied (0 to 3), 16.4% were somewhat satisfied (4 to 6), and 74.6% were satisfied to very satisfied (7 to 10). The mean of 7.71 (SD = 2.59) indicated that participants were rather likely to recommend the programme to others. Using the traditional approach to interpreting the NPR, a score of 24.6% was computed, which can be regarded as good.

Dose-response effects.

Participants in the IG used Optimune for a mean number of 25.7 days (SD = 33.9). Frequency of intervention use did not correlate, however, with the amount of change in primary outcomes between baseline and post-treatment (3 months).

Discussion

Summary of main results

Optimune, the Internet intervention examined here, facilitated significant improvements in patients’ quality of life and in healthy dietary habits. More tentative support was found for improvements in physical exercise habits, but this effect did not reach statistical significance. Nevertheless, trial success had been defined a priori as reaching significance on at least one of the three primary outcomes; therefore, this study provided first evidence of the efficacy of Optimune, a novel Internet intervention designed to improve mental health and facilitate change in health behaviours relevant to immune function among breast-cancer survivors.

A more fine-grained analysis of the primary outcome subscales revealed several notable details. Firstly, with respect to QoL, intervention effects were strongest for the psychological QoL subscale, suggesting that patients experienced improvements in their mental health, self-esteem, and cognitive abilities (e.g., memory and concentration). A somewhat weaker but statistically significant effect was observed for the physical aspects of QoL, suggesting that the intervention improves QoL aspects such as activities of daily living, mobility, work capacity, and dependence on medical. By contrast, intervention effects on the social relations and environment QoL domains were smaller, indicating the facets such as personal relationships, financial resources, or the home environment are less likely to be affected by Optimune, as one might expect. In sum, these findings suggest that Optimune targets mental as well as physical health components of QoL, which is consistent with the intervention’s goal of reducing psychological stress, enhancing mental health, and bolstering physical health by improving immune functioning.

With regard to the second primary outcome, dietary habits, intervention effects were stronger for improvements in healthy dieting rather than reductions in unhealthy dieting. In concrete terms, patients in the IG reported eating more healthy foods, such as vegetables, fruit and fatty fish, but their tendency to also reduce consumption of unhealthy foods (e.g., high-sugar foods, highly industrially processed “junk food” or “fast food”) was less pronounced. The goal of Optimune was to improve dietary habits by consuming more foods that improve healthy immune system functioning while also reducing immune-compromising foods, so this goal seems to be at least partially achieved.

With regard to the outcome exercise habits, there was some evidence that patients in the IG tended to increase their level of aerobic activity. This is in line with the conceptual goals of Optimune, as regular aerobic exercise seems to reduce inflammatory markers, such as CRP, TNF-α, and Interleukin-6 [66], and reducing chronic inflammation may improve the prognosis of breast cancer survivors [10].

Optimune also yielded improvements in several secondary outcomes, including reductions in insomnia, cancer-related fatigue, depression, anxiety, and fear of tumour progression. Furthermore, few negative reactions to the intervention were observed, and no serious adverse events occurred. The majority of the users were satisfied with Optimune and likely to recommend it to others.

Finally, exploratory findings suggested that the intervention might affect immunological parameters. That is, fewer intervention than control group participants reported at least one unplanned physician visit during the 3 months of intervention usage, whereas there was no such difference in the preceding 3 months. Compared to control group participants, intervention group participants also reported fewer respiratory infections at three months but not at baseline; however, this was not a significant difference. Taken together, these findings suggest that the intervention might facilitate improvements in immune status, although studies with greater statistical power will be needed to examine such potential effects and, if confirmed, to study underlying biological mechanisms, including chronic inflammation.

Significance of findings in light of previous research

Although it is difficult to define the exact threshold for a clinically relevant effect [67, 68], the majority of effects observed in this trial may be clinically relevant. For example, evidence suggests that effects of d = 0.24 can be regarded as clinically relevant with respect to psychosocial outcomes such as depression [69]. Applying this criterion, all of the intervention effects on primary outcomes and several effects on secondary outcomes can be deemed clinically relevant.

The effects observed in this trial are generally in line with effects observed for other behavioural interventions for cancer patients on psychosocial outcomes, such as distress and quality of life [6062]. For example, one meta-analysis reported small to medium effects of various CBT-based interventions on distress and pain in breast cancer patients; however, after adjusting for differences in sample size and treatment format (individual versus group), effect sizes for larger and group CBT formats were small [60]. Another systematic review reported that mindfulness-based stress reduction interventions had an average effect of Cohen’s d = 0.29 on QoL, which is similar to the effect we observed [61].

Of note, it has been argued that the value of an intervention does not depend solely on the magnitude of its effect size but also on its capacity to reach large numbers of individuals, because interventions that are easily scalable could improve population health even if effect sizes are modest [27]. As an Internet intervention that does not require personal support and can be used on any computer or smartphone with a web browser, Optimune can be regarded as a prototype of a highly scalable intervention.

In line with several meta-analyses, this trial confirmed that CBT can be delivered effectively via an Internet intervention without the provision of personal support or guidance [30, 40, 70, 71]. Previous meta-analyses have shown that self-guided Internet interventions achieve average effects of d = 0.27, for example, when depression is targeted as the primary endpoint [71]. Personal guidance can enhance overall treatment effectiveness, but the incremental effects of support provision appear to be surprisingly small, and the qualification of the support providers might make little difference (e.g., highly qualified clinicians vs. less qualified technicians) [7274]. Additionally, self-guided or minimally supported Internet interventions may be more cost-efficient and easier to implement, enhancing their potential to impact health on a population level [25, 62]. From a methodological perspective, it is also important to examine the effects of Internet interventions without the potential confounder of personal support by research staff, which may only be available when the intervention is implemented in routine care settings. In Germany, the Federal Institute for Drugs and Medical Devices, which evaluates whether digital health interventions are deemed reimbursable, also mandates trials to demonstrate that interventions are effective without the provision of services such as clinician guidance [75].

Breast cancer survivors often do not receive the psychosocial support they require, particularly once they have moved past the acute treatment phase [24], and many of them recognize the advantages and opportunities provided by Internet-based support, such as flexible remote access [76]. In line with this trial, several other studies have now demonstrated the effectiveness of Internet interventions for breast cancer survivors on outcomes such as quality of life, pain severity, fear of tumour progression, fatigue and insomnia [7779]. Digital interventions also have the potential to improve exercise and dietary habits among breast cancer survivors, according to meta-analytic evidence [80]. However, although the potential of such interventions has been shown repeatedly, the methodological quality of many studies is poor (e.g., small pilot studies), and many such interventions are narrow in focus (e.g., targeting only insomnia) and may not be available in routine practice. The implementation of evidence-based digital interventions remains a formidable challenge and requires the consideration of multiple barriers on different levels (e.g., organizational and care structures, patient-related, provider-related, reimbursement) [81].

Strengths and limitations

Advantages of this trial were that it was adequately powered to detect clinically relevant effects, that diagnoses at study entry were confirmed by clinicians, that a follow-up assessment was included to examine the stability of effects, that an array of well validated outcome measures was used, and that potential adverse effects were specifically examined. Several limitations must also be acknowledged. For example, this trial did not include an alternative specific active treatment comparator, although CAU control groups are generally favoured in pragmatic trials, which aim to examine the effects of behavioural interventions in realistic and heterogeneous routine care conditions [43]. Another limitation is that a novel questionnaire was used to assess dietary habits. Although an effort was made to ensure that the content of this questionnaire was aligned with current evidence [4749] and with the conceptual content of Optimune, and initial psychometric characteristics are promising, further studies are needed to establish its construct validity. However, the success of Optimune in this trial did not depend on the result obtained with this novel questionnaire, as the effect on quality of life was significant. We also acknowledge the limitation that outcomes were measured exclusively by patient self-reports, whereas biological indicators of health or immune status could increase objectivity. Another limitation is that we collected little data on variables such as time since breast cancer diagnosis, stage of disease, and prior treatments as well as current therapy, which makes it difficult to compare this sample with others. Finally, we note that this sample was relatively highly educated and included individuals who were motivated and able to engage with an internet intervention. Patients from under-represented minorities, older adults, rural populations, and those with lower health and computer literacy may be more difficult to reach with interventions such as the one we examined. Therefore, the results may not be generalizable to the entire population of breast cancer survivors. However, we note that Optimune is intended to be used by patients who have access to the internet and who have the cognitive capacities to engage with it.

Conclusion and future directions

In conclusion, this trial provided evidence for the safety and efficacy of Optimune, a new CBT-based, holistic Internet intervention for breast cancer patients. The intervention did not cause serious adverse effects and can therefore be regarded as safe, it was associated with few negative emotional reactions, and it led to statistically significant and clinically relevant improvements in quality of life and dietary habits. The intervention also led to improvements in several secondary outcomes, including insomnia, depression, anxiety, fear or tumour progression, and cancer-related emotional distress. Finally, compared to CG participants, IG participants reported fewer unplanned physician visits over 3 months, which suggests that it might facilitate improvements in general health and possibly immune status. We conclude that Optimune could be disseminated more broadly to breast cancer patients, many of whom do not routinely receive psychosocial support or CBT. It would be desirable to replicate these effects and to examine whether they are mediated by immunological processes, such as reductions in inflammation. Given its scalability, efficiency, flexibility and ease of implementation, we anticipate that Optimune has the potential to improve the quality of life and general health of breast cancer patients. Moreover, we anticipate that versions of Optimune could be developed in other languages and for other target populations, given that improvements in immune status, mental and physical health, as well as dietary and exercise habits are beneficial for a broad spectrum of the general population.

Supporting information

S1 File. Sensitivity analyses (reference-based multiple imputation).

https://doi.org/10.1371/journal.pone.0251276.s001

(DOCX)

S1 Table. PP analysis of primary endpoints.

https://doi.org/10.1371/journal.pone.0251276.s004

(DOCX)

S2 Table. PP analysis of secondary endpoints.

https://doi.org/10.1371/journal.pone.0251276.s005

(DOCX)

S3 Table. Follow-up results.

Changes between 3 and 6 months (CG accessed intervention after 3 months).

https://doi.org/10.1371/journal.pone.0251276.s006

(DOCX)

Acknowledgments

We acknowledge the contributions of the multidisciplinary development team at Gaia, which designed and produced the Internet intervention examined in this study, as well as the contributions of the patients, relatives of patients, and treatment providers who provided feedback and valuable suggestions in the intervention development process.

References

  1. 1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: a cancer journal for clinicians. 2018;68(6):394–424.
  2. 2. Kreienberg R, Albert U, Follmann M, Kopp I, Kühn T, Wöckel A. Interdisziplinäre S3-Leitlinie für die Diagnostik, Therapie und Nachsorge des Mammakarzinoms. Senologie-Zeitschrift für Mammadiagnostik und-therapie. 2013;10(03):164–92.
  3. 3. Abrahams H, Gielissen M, Schmits I, Verhagen C, Rovers M, Knoop H. Risk factors, prevalence, and course of severe fatigue after breast cancer treatment: a meta-analysis involving 12 327 breast cancer survivors. Annals of Oncology. 2016;27(6):965–74. pmid:26940687
  4. 4. Bower JE, Ganz PA, Desmond KA, Bernaards C, Rowland JH, Meyerowitz BE, et al. Fatigue in long-term breast carcinoma survivors: a longitudinal investigation. Cancer. 2006;106(4):751–8. pmid:16400678
  5. 5. Maass SW, Roorda C, Berendsen AJ, Verhaak PF, de Bock GH. The prevalence of long-term symptoms of depression and anxiety after breast cancer treatment: a systematic review. Maturitas. 2015;82(1):100–8. pmid:25998574
  6. 6. Von Ah D, Kang DH. Correlates of mood disturbance in women with breast cancer: patterns over time. Journal of Advanced Nursing. 2008;61(6):676–89. pmid:18302608
  7. 7. Mehnert A, Hartung TJ, Friedrich M, Vehling S, Brähler E, Härter M, et al. One in two cancer patients is significantly distressed: Prevalence and indicators of distress. Psycho-oncology. 2018;27(1):75–82. pmid:28568377
  8. 8. Bower JE, Ganz PA, Aziz N, Fahey JL. Fatigue and proinflammatory cytokine activity in breast cancer survivors. Psychosomatic medicine. 2002;64(4):604–11. pmid:12140350
  9. 9. Collado-Hidalgo A, Bower JE, Ganz PA, Cole SW, Irwin MR. Inflammatory biomarkers for persistent fatigue in breast cancer survivors. Clinical cancer research. 2006;12(9):2759–66. pmid:16675568
  10. 10. Pierce BL, Ballard-Barbash R, Bernstein L, Baumgartner RN, Neuhouser ML, Wener MH, et al. Elevated biomarkers of inflammation are associated with reduced survival among breast cancer patients. Journal of Clinical Oncology. 2009;27(21):3437–44. pmid:19470939
  11. 11. Faller H, Schuler M, Richard M, Heckl U, Weis J, Küffner R. Effects of psycho-oncologic interventions on emotional distress and quality of life in adult patients with cancer: systematic review and meta-analysis. Journal of Clinical Oncology. 2013;31(6):782–93. pmid:23319686
  12. 12. McGregor BA, Antoni MH. Psychological intervention and health outcomes among women treated for breast cancer: a review of stress pathways and biological mediators. Brain, behavior, and immunity. 2009;23(2):159–66. pmid:18778768
  13. 13. Carlson LE, Speca M, Patel KD, Goodey E. Mindfulness-based stress reduction in relation to quality of life, mood, symptoms of stress, and immune parameters in breast and prostate cancer outpatients. Psychosomatic medicine. 2003;65(4):571–81. pmid:12883107
  14. 14. Bower JE, Crosswell AD, Stanton AL, Crespi CM, Winston D, Arevalo J, et al. Mindfulness meditation for younger breast cancer survivors: a randomized controlled trial. Cancer. 2015;121(8):1231–40. pmid:25537522
  15. 15. Kiecolt-Glaser JK, Bennett JM, Andridge R, Peng J, Shapiro CL, Malarkey WB, et al. Yoga’s impact on inflammation, mood, and fatigue in breast cancer survivors: a randomized controlled trial. Journal of Clinical Oncology. 2014;32(10):1040. pmid:24470004
  16. 16. Black DS, Slavich GM. Mindfulness meditation and the immune system: a systematic review of randomized controlled trials. Annals of the New York Academy of Sciences. 2016;1373(1):13. pmid:26799456
  17. 17. Walsh E, Eisenlohr-Moul T, Baer R. Brief mindfulness training reduces salivary IL-6 and TNF-α in young women with depressive symptomatology. Journal of consulting and clinical psychology. 2016;84(10):887. pmid:27281371
  18. 18. Moreira FP, de Azevedo Cardoso T, Mondin TC, de Mattos Souza LD, Silva R, Jansen K, et al. The effect of proinflammatory cytokines in Cognitive Behavioral Therapy. Journal of neuroimmunology. 2015;285:143–6. pmid:26198931
  19. 19. Köhler O, Benros ME, Nordentoft M, Farkouh ME, Iyengar RL, Mors O, et al. Effect of anti-inflammatory treatment on depression, depressive symptoms, and adverse effects: a systematic review and meta-analysis of randomized clinical trials. JAMA psychiatry. 2014;71(12):1381–91. pmid:25322082
  20. 20. Hannestad J, DellaGioia N, Bloch M. The effect of antidepressant medication treatment on serum levels of inflammatory cytokines: a meta-analysis. Neuropsychopharmacology. 2011;36(12):2452–9. pmid:21796103
  21. 21. Juurlink D. Revisiting the drug interaction between tamoxifen and SSRI antidepressants. British Medical Journal Publishing Group; 2016. pmid:27694573
  22. 22. Sanjida S, Janda M, Kissane D, Shaw J, Pearson SA, DiSipio T, et al. A systematic review and meta-analysis of prescribing practices of antidepressants in cancer patients. Psycho-Oncology. 2016;25(9):1002–16. pmid:26775715
  23. 23. da Silva GDG, Wiener CD, Barbosa LP, Araujo JMG, Molina ML, San Martin P, et al. Pro-inflammatory cytokines and psychotherapy in depression: Results from a randomized clinical trial. Journal of psychiatric research. 2016;75:57–64. pmid:26802811
  24. 24. Zimmermann-Schlegel V, Hartmann M, Sklenarova H, Herzog W, Haun MW. Accessibility, Availability, and Potential Benefits of Psycho-Oncology Services: The Perspective of Community-Based Physicians Providing Cancer Survivorship Care. The oncologist. 2017;22(6):719. pmid:28438888
  25. 25. Mohr DC, Ho J, Duffecy J, Baron KG, Lehman KA, Jin L, et al. Perceived barriers to psychological treatments and their relationship to depression. Journal of clinical psychology. 2010;66(4):394–409. pmid:20127795
  26. 26. Andersson G. Internet interventions: past, present and future. Internet interventions. 2018;12:181–8. pmid:30135782
  27. 27. Kazdin AE, Blase SL. Rebooting psychotherapy research and practice to reduce the burden of mental illness. Perspectives on psychological science. 2011;6(1):21–37. pmid:26162113
  28. 28. Bendig E, Bauereiß N, Ebert DD, Snoek F, Andersson G, Baumeister H. Internet-based interventions in chronic somatic disease. Deutsches Ärzteblatt International. 2018;115(40):659. pmid:30381130
  29. 29. Carlbring P, Andersson G, Cuijpers P, Riper H, Hedman-Lagerlöf E. Internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: an updated systematic review and meta-analysis. Cognitive Behaviour Therapy. 2018;47(1):1–18. pmid:29215315
  30. 30. Andersson G, Carlbring P, Titov N, Lindefors N. Internet interventions for adults with anxiety and mood disorders: a narrative umbrella review of recent meta-analyses. The Canadian Journal of Psychiatry. 2019;64(7):465–70. pmid:31096757
  31. 31. Ebert D, Donkin L, Andersson G, Andrews G, Berger T, Carlbring P, et al. Does Internet-based guided-self-help for depression cause harm? An individual participant data meta-analysis on deterioration rates and its moderators in randomized controlled trials. Psychological Medicine. 2016;46(13):2679–93. pmid:27649340
  32. 32. Bücker L, Schnakenberg P, Karyotaki E, Moritz S, Westermann S. Diminishing effects after recurrent use of self-guided internet-based interventions in depression: randomized controlled trial. Journal of medical Internet research. 2019;21(10):e14240. pmid:31579014
  33. 33. Cuijpers P, Cristea IA. How to prove that your therapy is effective, even when it is not: a guideline. Epidemiology and Psychiatric Sciences. 2015;25(5):428–35. Epub 09/28. pmid:26411384
  34. 34. Holtdirk F, Mehnert A, Weiss M, Meyer B, Watzl C. Protocol for the Optimune trial: a randomized controlled trial evaluating a novel Internet intervention for breast cancer survivors. Trials. 2020;21(1):1–11. pmid:31996235
  35. 35. Beevers CG, Pearson R, Hoffman JS, Foulser A, Shumake J, Meyer B. Effectiveness of an internet intervention for depression in a US adult sample: A parallel-group pragmatic randomized controlled trial. Journal of Consulting and Clinical Psychology. 2017.
  36. 36. Berger TUA.; Krieger T.; Stolz T.; Schulz A.; Vincent A.; Moser C.T.; et al. Effects of a transdiagnostic unguided Internet intervention ("velibra") for anxiety disorders in primary care: Results of a randomised controlled trial. Psychological Medicine. 2016;in press.
  37. 37. Meyer B, Bierbrodt J, Schröder J, Berger T, Beevers CG, Weiss M, et al. Effects of an internet intervention (Deprexis) on severe depression symptoms: randomized controlled trial. Internet Interventions. 2015;2(1):48–59.
  38. 38. Meyer B, Weiss M, Holtkamp M, Arnold S, Brückner K, Schröder J, et al. Effects of an epilepsy-specific Internet intervention (Emyna) on depression: Results of the ENCODE randomized controlled trial. Epilepsia. 2019. pmid:30802941
  39. 39. Pöttgen J Rona M-M; Wendebourg Janina-Maria; Feddersen Lena; Lau Stefanie; Köpfke Sascha; Meyer Björn; et al. A self-guided online fatigue management program for patients with multiple sclerosis: Randomized controlled trial. Journal of Neurology, Neurosurgery, and Psychiatry. 2018.
  40. 40. Twomey COR G.; Bültmann O.; Meyer B. Effectiveness of a tailored, integrative Internet intervention (deprexis) for depression: Updated meta-analysis. Manuscripts currently under review (submitted to PLOS ONE). 2019.
  41. 41. Zill J, M.; Christalle, E.; Meyer, B.; Härter, M.; Dirmaier, J. Effectiveness of a novel Internet Intervention for the reduction of alcohol consumption in adults (Vorvida): Results of a randomized controlled trial. Deutsches Ärzteblatt International. 2019.
  42. 42. Ford I, Norrie J. Pragmatic trials. New England journal of medicine. 2016;375(5):454–63. pmid:27518663
  43. 43. Freedland KE, King AC, Ambrosius WT, Mayo-Wilson E, Mohr DC, Czajkowski SM, et al. The selection of comparators for randomized controlled trials of health-related behavioral interventions: recommendations of an NIH expert panel. Journal of clinical epidemiology. 2019;110:74–81. pmid:30826377
  44. 44. Zwerenz R, Becker J, Knickenberg RJ, Siepmann M, Hagen K, Beutel ME. Online Self-Help as an Add-On to Inpatient Psychotherapy: Efficacy of a New Blended Treatment Approach. Psychotherapy and Psychosomatics. 2017;86(6):341–50. pmid:29131090
  45. 45. Group W. Development of the World Health Organization WHOQOL-BREF quality of life assessment. Psychological medicine. 1998;28(3):551–8. pmid:9626712
  46. 46. Lee PH, Macfarlane DJ, Lam TH, Stewart SM. Validity of the international physical activity questionnaire short form (IPAQ-SF): A systematic review. International Journal of Behavioral Nutrition and Physical Activity. 2011;8(1):115. pmid:22018588
  47. 47. Katz DL, Meller S. Can we say what diet is best for health? Annual review of public health. 2014;35:83–103. pmid:24641555
  48. 48. Mourouti N, Panagiotakos DB, Kotteas EA, Syrigos KN. Optimizing diet and nutrition for cancer survivors: A review. Maturitas. 2017;105:33–6. pmid:28545906
  49. 49. Schwedhelm C, Boeing H, Hoffmann G, Aleksandrova K, Schwingshackl L. Effect of diet on mortality and cancer recurrence among cancer survivors: a systematic review and meta-analysis of cohort studies. Nutrition reviews. 2016;74(12):737–48. pmid:27864535
  50. 50. Mendoza TR, Wang XS, Cleeland CS, Morrissey M, Johnson BA, Wendt JK, et al. The rapid assessment of fatigue severity in cancer patients: use of the Brief Fatigue Inventory. Cancer. 1999;85(5):1186–96. pmid:10091805
  51. 51. Radbruch L, Sabatowski R, Elsner F, Everts J, Mendoza T, Cleeland C. Validation of the German version of the brief fatigue inventory. Journal of pain and symptom management. 2003;25(5):449–58. pmid:12727043
  52. 52. Creamer M, Bell R, Failla S. Psychometric properties of the impact of event scale—revised. Behaviour research and therapy. 2003;41(12):1489–96. pmid:14705607
  53. 53. Kroenke K, Spitzer RL. The PHQ-9: a new depression diagnostic and severity measure. Psychiatric annals. 2002;32(9):509–15.
  54. 54. Löwe B, Decker O, Müller S, Brähler E, Schellberg D, Herzog W, et al. Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Medical care. 2008:266–74. pmid:18388841
  55. 55. Herschbach PDP, Berg DPP, Duran G, Waadt S, Henrich G, der Isar KR, et al. Entwicklung und Evaluation eines Therapieprogrammes zur Bewältigung von Progredienzangst bei chronisch Kranken.
  56. 56. Morin CM, Belleville G, Bélanger L, Ivers H. The Insomnia Severity Index: psychometric indicators to detect insomnia cases and evaluate treatment response. Sleep. 2011;34(5):601–8. pmid:21532953
  57. 57. Keiningham TL, Aksoy L, Cooil B, Andreassen TW, Williams L. A holistic examination of Net Promoter. Journal of Database Marketing & Customer Strategy Management. 2008;15(2):79–90.
  58. 58. Ladwig I, Rief W, Nestoriuc Y. Welche Risiken und Nebenwirkungen hat Psychotherapie?-Entwicklung des Inventars zur Erfassung Negativer Effekte von Psychotherapie (INEP). Verhaltenstherapie. 2014;24(4):252–63.
  59. 59. Faul F, Erdfelder E, Lang A-G, Buchner A. G* Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior research methods. 2007;39(2):175–91. pmid:17695343
  60. 60. Tatrow K, Montgomery GH. Cognitive behavioral therapy techniques for distress and pain in breast cancer patients: a meta-analysis. Journal of behavioral medicine. 2006;29(1):17–27. pmid:16400532
  61. 61. Musial F, Büssing A, Heusser P, Choi K-E, Ostermann T. Mindfulness-based stress reduction for integrative cancer care—a summary of evidence. Complementary Medicine Research. 2011;18(4):192–202. pmid:21934319
  62. 62. Ledesma D, Kumano H. Mindfulness-based stress reduction and cancer: a meta-analysis. Psycho-Oncology: Journal of the Psychological, Social and Behavioral Dimensions of Cancer. 2009;18(6):571–9. pmid:19023879
  63. 63. Vickerstaff V, Ambler G, King M, Nazareth I, Omar RZ. Are multiple primary outcomes analysed appropriately in randomised controlled trials? A review. Contemporary Clinical Trials. 2015;45:8–12. pmid:26215934
  64. 64. Tabachnick BG, Fidell LS, Ullman JB. Using multivariate statistics: Pearson Boston, MA; 2007.
  65. 65. Cuijpers P. Meta-analyses in mental health research. A practical guide2016.
  66. 66. Zheng G, Qiu P, Xia R, Lin H, Ye B, Tao J, et al. Effect of aerobic exercise on inflammatory markers in healthy middle-aged and older adults: a systematic review and meta-analysis of randomized controlled trials. Frontiers in aging neuroscience. 2019;11:98. pmid:31080412
  67. 67. Wise EA. Methods for analyzing psychotherapy outcomes: A review of clinical significance, reliable change, and recommendations for future directions. Journal of personality assessment. 2004;82(1):50–9. pmid:14979834
  68. 68. Keefe RSE, Kraemer HC, Epstein RS, Frank E, Haynes G, Laughren TP, et al. Defining a clinically meaningful effect for the design and interpretation of randomized controlled trials. Innov Clin Neurosci. 2013;10(5–6 Suppl A):4S–19S. pmid:23882433.
  69. 69. Cuijpers P, Turner EH, Koole SL, Dijke A, Smit F. What is the threshold for a clinically relevant effect? The case of major depressive disorders. Depression and anxiety. 2014;31(5):374–8. pmid:24677535
  70. 70. Andrews G, Basu A, Cuijpers P, Craske M, McEvoy P, English C, et al. Computer therapy for the anxiety and depression disorders is effective, acceptable and practical health care: An updated meta-analysis. Journal of anxiety disorders. 2018. pmid:29422409
  71. 71. Karyotaki E, Riper H, Twisk J, Hoogendoorn A, Kleiboer A, Mira A, et al. Efficacy of self-guided internet-based cognitive behavioral therapy in the treatment of depressive symptoms: a meta-analysis of individual participant data. JAMA psychiatry. 2017;74(4):351–9. pmid:28241179
  72. 72. Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on Internet-based mental health interventions—A systematic review. Internet Interventions. 2014;1(4):205–15.
  73. 73. Dirkse D, Hadjistavropoulos HD, Alberts NA, Karin E, Schneider LH, Titov N, et al. Making Internet-delivered cognitive behaviour therapy scalable for cancer survivors: a randomized non-inferiority trial of self-guided and technician-guided therapy. Journal of Cancer Survivorship. 2020;14(2):211–25. pmid:31853727
  74. 74. Titov N, Andrews G, Davies M, McIntyre K, Robinson E, Solley K. Internet treatment for depression: a randomized controlled trial comparing clinician vs. technician assistance. PloS one. 2010;5(6):e10939. pmid:20544030
  75. 75. (BfArM) FIfDaMD. Das Fast-Track-Verfahren für digitale Gesundheitsanwendungen (DiGA) nach § 139e SGB V: Ein Leitfaden für Hersteller, Leistungserbringer und Anwender https://www.bfarm.de/SharedDocs/Downloads/DE/Service/Beratungsverfahren/DiGA-Leitfaden.pdf;jsessionid=FB509AE07217176119516E54F08EEA55.1_cid354?__blob=publicationFile&v=42020.
  76. 76. Cox A, Lucas G, Marcu A, Piano M, Grosvenor W, Mold F, et al. Cancer Survivors’ Experience With Telehealth: A Systematic Review and Thematic Synthesis. J Med Internet Res. 2017;19(1):e11. pmid:28069561
  77. 77. Galiano-Castillo N, Cantarero-Villanueva I, Fernández-Lao C, Ariza-García A, Díaz-Rodríguez L, Del-Moral‐Ávila R, et al. Telehealth system: A randomized controlled trial evaluating the impact of an internet-based exercise intervention on quality of life, pain, muscle strength, and fatigue in breast cancer survivors. Cancer. 2016;122(20):3166–74. pmid:27332968
  78. 78. Hummel SB, van Lankveld JJ, Oldenburg HS, Hahn DE, Kieffer JM, Gerritsma MA, et al. Efficacy of internet-based cognitive behavioral therapy in improving sexual functioning of breast cancer survivors: results of a randomized controlled trial. Journal of Clinical Oncology. 2017;35(12):1328–40. pmid:28240966
  79. 79. Ritterband LM, Bailey ET, Thorndike FP, Lord HR, Farrell-Carnahan L, Baum LD. Initial evaluation of an Internet intervention to improve the sleep of cancer survivors with insomnia. Psycho-Oncology. 2012;21(7):695–705. pmid:21538678
  80. 80. Roberts AL, Fisher A, Smith L, Heinrich M, Potts HWW. Digital health behaviour change interventions targeting physical activity and diet in cancer survivors: a systematic review and meta-analysis. Journal of Cancer Survivorship. 2017;11(6):704–19. pmid:28779220
  81. 81. Drozd F, Vaskinn L, Bergsund HB, Haga SM, Slinning K, Bjørkli CA. The Implementation of Internet Interventions for Depression: A Scoping Review. J Med Internet Res. 2016;18(9):e236. pmid:27608548