Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Can we learn to manage stress? A randomized controlled trial carried out on university students

  • Dalia Saleh ,

    Roles Conceptualization, Data curation, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft

    dalia_saleh84@yahoo.com

    Current address: Department of Counseling Psychology, Tishreen University, Latakia, Syria

    Affiliations EA4430 CLIPSYD, UFR SPSE, Paris Nanterre University, Nanterre, France, Counseling Psychology, Tishreen University, Latakia, Syria

    ⨯
  • Nathalie Camart,

    Roles Conceptualization, Data curation, Formal analysis, Supervision, Writing – review & editing

    Current address: Department of Psychology, Paris Nanterre University, Nanterre, France

    Affiliation EA4430 CLIPSYD, UFR SPSE, Paris Nanterre University, Nanterre, France

    ⨯
  • Fouad Sbeira,

    Roles Conceptualization, Data curation, Supervision, Writing – review & editing

    Current address: Department of Counseling Psychology, Tishreen University, Latakia, Syria

    Affiliation Counseling Psychology, Tishreen University, Latakia, Syria

    ⨯
  • Lucia Romo

    Roles Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Supervision, Writing – review & editing

    Current address: Department of Psychology, Paris Nanterre University, Nanterre, France

    Affiliations EA4430 CLIPSYD, UFR SPSE, Paris Nanterre University, Nanterre, France, CMME, Centre Hospitalier Sainte Anne, Unité Inserm U864, CPN, Paris, France

    ⨯

Abstract

In our research, we examined the efficacy of an Internet-based stress management program. Our interest in evaluating this type of intervention is based on the increasing accessibility of the Internet today, the growth of Internet-based interventions for various psychopathological problems, and the observation that despite the prevalence of stress among university students, only a fraction of students ever seek professional help. Methodology: “I’m managing my stress” (“Je gère mon stresse”), an Internet-based self-help program composed of four sessions, was examined in this study. The aforementioned program is based on cognitive-behavioral therapy and was inspired by the “Funambule” program in Quebec. Four questionnaires (Perceived Stress Scale, Rosenberg Self-Esteem Scale, Scale of Satisfaction in Studies, and General Health Questionnaire) uploaded online were answered thrice: during “preintervention”, “postintervention”, and “follow-up” stages, the latter of which occurred three months after the intervention. The sample comprised 128 university students, with the majority being women (81.25%). The subjects were divided randomly into two groups (an experimental group and a control group that did not follow the program). Results: The self-esteem scores of the control group were significantly higher than those of the experimental group at the preintervention stage, but this difference disappeared at the postintervention and follow-up stages. There were also significantly lower scores on the General Health Questionnaire subfactors of somatic symptoms and anxiety/insomnia in the experimental group than in the control group during the postintervention stage, though no differences were observed before the intervention. These differences no longer remained after three months. ANOVA revealed significant effects of the intervention over time in the experimental group. Effects were observed at both the postintervention and follow-up stages for self-esteem, perceived stress, satisfaction in studies, and in the somatic symptoms, anxiety and insomnia and severe depression aspects of the General Health Questionnaire (Cohen’s d = 0.38 to 4.58). In contrast, no effects were observed in the control group. Conclusion: This type of Internet-based program has the ability to reach a large number of students due to its rather short format and accessibility. It has already shown improvements in terms of the levels of perceived stress, psychological distress and satisfaction with studies. The option of online interventions could appeal specifically to students who do not seek professional help. However, even though these results are promising at the postintervention stage, they are limited, as indicated by the lack of significant differences between the two groups after the initial three months of follow-up. We still, specifically, need to improve this intervention program and, generally, need more research to address the methodological problems raised by this type of intervention.

Trial registration: ISRCTN registry, ISRCTN13709272

Citation: Saleh D, Camart N, Sbeira F, Romo L (2018) Can we learn to manage stress? A randomized controlled trial carried out on university students. PLoS ONE 13(9): e0200997. https://doi.org/10.1371/journal.pone.0200997

Editor: Jacobus P. van Wouwe, TNO, NETHERLANDS

Received: March 9, 2017; Accepted: July 3, 2018; Published: September 5, 2018

Copyright: © 2018 Saleh et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The data underlying this study can not be made publicly available as the data are protected by the National Commission on Informatics and Liberty (CNIL), the national data protection authority for France. CNIL is an independent French administrative regulatory body whose mission is to ensure that data privacy law is applied to the collection, storage, and use of personal data. As the database of this study was authorized by the CNIL, we cannot make available data without prior agreement of the CNIL. More information can be found here: https://www.cnil.fr/en/home. In addition, the researchers of this paper can also be contacted for assistance requesting data (Dalia Saleh: dalia_saleh84@yahoo.com, Nathalie Camart: nathalie_camart@hotmail.com, Lucia Romo: romodesprez@gmail.com).

Funding: The incentives (in the form of gift vouchers to all participants who fully participated in the research) in this study were funded by The Scientific Interest Group « Jeu et société ». The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Abbreviations: ESDE, Satisfaction in Studies; GHQ-28, General Health Questionnaire; PSS-10, Perceived Stress Scale; RSES, Rosenberg Self-Esteem Scale

Introduction

University students are a category of people who are particularly vulnerable to stress [1–3] and present, according to scientific literature, high levels of stress [4–9]. They are prone to stress-related issues such as anxiety, depression [10–12], eating disorders [13–15], consumption of psychoactive substances [16,17], and sleep disorders [12,18,19]. The rates of psychological morbidity among university students are higher than those seen in the general population [10,11,20–24].

According to the literature, 83% of university students feel tired [25], 60% have low self-esteem [26] and 15% harbor suicidal thoughts [27]. In fact, the prevalence of several mental health issues among university students is always high, regardless of the issue. For example, the mental health distress rate ranges from 21% to 82% [26,28–30], the depression rate between 13% and 53% [31–34], anxiety between 34% and 47% [34–36], and stress between 33% and 79% [8,23,37,38]. More than half of students suffer from at least one mental health problem according to a study undertaken by Zivin and his team. In their study, 50% of students who declared having mental health issues, such as depression, anxiety and suicidal thoughts, did not seek help [39].

The link between stress and psychological distress has been theoretically presented in Lazarus and Folkman’s transactional model in 1984 [4]. Their work is often applied in stress-management programs [40].

Finding efficient strategies to reinforce the feeling of personal efficacy and the efficient management of difficulties in certain groups such as students are essential to help these groups adapt to challenges and maintain a good standard of living [41]. Different types of interventions have already proven their efficacy [42–44].

The Internet has become an essential tool in the field of “self-help” interventions in mental health. These interventions can be defined by the fact that they allow online access to a therapeutic-aimed program [45]. This type of intervention via the Internet has several goals, such as reducing the risks leading to targeted problems [45], such as the levels of stress, anxiety and depression [46]; raising the level of well-being [47]; enhancing adaptation strategies [48]; and even increasing weekly physical activity [49]. The advantages of these online trials are their accessibility, continuous availability, confidentiality, and discretion, especially for people who do not want to seek medical help in a health center, as well as the opportunity to extend the program to a large population in an economical way [45].

Online application studies in this field are being developed to treat several problems, such as panic disorder [50], depression [51–53], anxiety [54–56], insomnia [57], posttraumatic stress [58], alcohol abuse [59], binge drinking [60], social phobia [61,62], and behavioral problems [63]. Stress management is one of the applications that has been suggested for diverse population groups [64–67].

Online stress management interventions aim to manage stress as well as other psychopathological issues [45] such as anxiety [68,69], prevention of obesity [70] and psychological distress [71]. In our literature review, according to a meta-analysis conducted in 2016 [45], only six studies had stress management as their principal goal [46–49,72], and only three of these studies were carried out with university students [45,46,48,49].

Despite its clear efficacy and numerous advantages [46,48], this type of intervention for stress management has its limits [73] and a major methodological problem, that is, a high attrition rate [45].

The aim of the current research is to measure the efficacy of an online stress management program for university students based on several mental health variables: self-esteem, perceived stress and its two subfactors (perceived helplessness and perceived self-efficacy), psychological distress and its four subfactors (somatic symptoms, anxiety/insomnia, social dysfunction, severe depression) and satisfaction with studies.

Materials and methods

Design of study

This protocol used a randomized controlled trial to determine the efficacy of an online stress management intervention for university students (Fig 1). The inclusion criteria included the following: being a student at a French university, having mastered the French language, being aged between 18 and 30, and having an e-mail address and access to the Internet.

thumbnail
Download:
Fig 1. Trial schema.

https://doi.org/10.1371/journal.pone.0200997.g001

The subjects were randomly allocated into one of the two groups: an experimental group (those who followed the program) and a control group (those who did not follow the program). The subjects in the control group were informed that they had been randomly assigned to a waiting group, that they would not be following the program during the study, and that they could follow it afterwards if they so wished. The study followed the CONSORT 2010 Statement’s advice about randomized controlled trials [74].

Participants

A total of 142 students voluntarily signed up for baseline assessments, and 16 participants were excluded (aged under 18 or over 30). The sample consisted of 128 university students (64 in the experimental group and 64 in the control group), with a female majority (81.25% in the whole sample, 82.81% in the experimental group and 79.69% in the control group). The mean age of the whole sample was 22.54 years (Standard Deviation or SD: 3), with no difference between the two groups in terms of age. The students were from all academic years, from the first year at university to PhD-level, and from different study programs: Philosophy, Languages, Literature, Economy, Management, Mathematics, Computing, Psychology, Law and Political Science, Medicine, etc. (Table 1).

thumbnail
Download:
Table 1. Participant characteristics n (%).

https://doi.org/10.1371/journal.pone.0200997.t001

Recruitment and procedure

The research was presented as an assessment of a stress management intervention carried out with university students, whose participation was anonymous and voluntary. The recruitment occurred mostly on the Paris Nanterre University website, on which an advertisement for research was published. This allowed publicity in the student newsletter and on the university’s social media outlets. Posters were also put up in the campus and on social media.

Stages of research

The research occurred between November 2015 and June 2016.

Preintervention (baseline/October 2015-January 2016): After providing their consent, the subjects received an Internet link to take part in the research online. Gift vouchers were offered to all participants as an incentive. Intervention “experimental group”/“control group”: In January, the students who were allocated into the experimental group were informed of their group assignment. They then provided their consent to participate in the intervention and perform tasks between sessions. They were invited to visit the website once a week and to spend at least 20 minutes on it per visit. They could visit the page for longer or more frequently if they wished. At the same time, the subjects in the control group were informed that they had been randomly allocated into the control group and that they could follow the program afterwards if they so wished. Postintervention assessment (February 2016): Just after the end of the intervention in March, the participants of both groups were simultaneously invited to answer the battery of questionnaires by email for the first “postassessment”. The "follow-up" (second postassessment) occurred three months later for both groups, in June. Reminder emails were sent in cases of no response.

Intervention (« I’m managing my stress ») (February 2016)

We created an online stress management pilot program entitled “I’m managing my stress”. It was inspired by the “Funambule” program for teenagers developed by Dumont and his team [40] in Canada, in which cognitive-behavioral therapy techniques are used over 8 sessions.

The “Funambule, for balanced stress management” program, developed by Dumont and his team [40,75], is an intervention program designed to help young people aged 12 to 18 manage their stress better. It is undertaken in 8 weekly sessions of at least an hour and a half each. The content of the sessions is organized into 4 parts to target 4 goals: the perception of stress, the body, thoughts, and adaptation strategies [40,75]. The conception of the program was inspired by the transactional theory of stress developed by Lazarus and Folkmans in 1984, which aimed to strengthen protective resources [76]. Beneficial effects and a significant improvement of stress management in the experimental group were noted for participants using this program [75].

The “I’m managing my stress” program that we developed has the same objectives and aims, but as it is a pilot program inspired by “Funambule”, we therefore reworked and adapted it to fit the Internet and our study population (adults, not children as in the original program). The program consists of four sessions, each 20 minutes long, including psycho-education, practical exercises and one to two weekly activities that the participant is asked to complete (prescription of tasks, as is customary in cognitive-behavioral techniques). The goal is for the students to learn easy techniques to help them handle stressful situations better.

The first session is “psycho-education”, with information for the participants on how to identify and understand stress, measure stress levels, and determine stress sources. The second session aims to develop body relaxation using practical exercises adapted for the Internet, such as a contraction and relaxation method or diaphragmatic breathing. The third session is based on cognitive-behavioral techniques, such as Beck’s three columns, the Meichenbaum method, and problem solving. The objective of this session is to develop positive thinking [40]. The fourth session focuses on adaptation strategies such as time management, exam preparation, and the use of DIY anti-stress cards [40,75]. The final session is used to collect the participants’ opinions on the program in order to assess the program.

This program is interactive to the extent that the participants are invited to provide comments and opinions after each exercise and at the end of each session, at every stage of the program.

An email was sent to invite the participants at the start of each session to go online if they had not visited the website spontaneously; the first reminder was sent within three days after the start of the sessions. The students could contact us any time via a specific email address with any questions or complementary information on the program. Responses were sent within 48 hours.

Measures

We split the questionnaires used in our study into two sections. First, the students provided socio-demographic data concerning gender, age, place of residence, current year of study, study program and university of origin. The second part was a collection of questionnaires, including four uploaded ones during the preintervention, postintervention, and follow-up (3 months after the intervention) stages.

  • Stress was assessed with the Perceived Stress Scale (PSS-10) using the 10-item version. This method was established by Cohen and his team [77] It was subsequently translated and validated in French by Bellinghausen and his team. Using a 5-point scale, from 1 (which means “never”) to 5 (“very often”), each variable’s frequency is rated. This scale includes two subfactors: perceived helplessness and perceived self-efficacy [78]. Two thresholds were decided upon: a score over 24 indicated anxiety, and a score over 26 indicated depression [79]. The Cronbach’s alpha coefficient calculated for this assessment was (.84).
  • To measure self-esteem, we decided to use a 10-item version of the Rosenberg Self-Esteem Scale (RSES), which was established by Morris Rosenberg [80] and then later translated and validated in French by Vallieres and his team. The participants were invited to use a 4-point Likert scale format, with their answers to the items ranging from “strongly agree” to “strongly disagree” [81]. Low self-esteem was indicated by a score under 30 [82] In this study, Cronbach’s alpha coefficient was (.9).
  • To evaluate the participants’ satisfaction in their studies, we opted for the 5-item version of the ESDE, developed by Bissonnette and Vallerand, which was translated into French and validated by Vallerand and his team. In this assessment, five items are answered on a scale that ranges from 1 (“strongly disagree”) to 7 (“strongly agree”) [83]. The higher the score, the higher the participant’s satisfaction in their studies [84]. In this study, Cronbach’s alpha coefficient was (.79).
  • We measured psychological distress using the 28-item General Health Questionnaire (GHQ-28), a questionnaire developed by Goldberg [85] and then translated and validated in French by Bolognini and his team [86]. It consists of 4 subscales that evaluate somatic symptoms, anxiety/insomnia, social dysfunction and severe depression [85]. In each of these sections, participants are invited to answer 7 items using a 4-point Likert scale [86]. Psychological distress is indicated by a score greater than or equal to 5 [87]. In this study, Cronbach’s alpha coefficient was (.65).

Ethics

The ethics committee of the Psychological Science and Learning Science Department at the University of Paris Ouest Nanterre La Défense, UFR SPE (Department of Psychology and Education) in May 2015 and the CNIL (National Commission for Computing and Freedom) in November 2014 [n°: 1811031 v 0] gave their approval for our study protocol. We also obtained written consent from each student prior to their participation, as required by the Helsinki Declaration.

The research team obtained written permission from the creators of “Funambule”, giving us the right to take inspiration from their program to create an intervention program adapted to the Internet. A report on the study was sent to the creators of “Funambule” after each stage.

The authors confirm that all ongoing and related trials for this intervention are registered.

Feasibility study

The feasibility of the study was tested in a pilot study between January and March 2015, in which a first group of 18 students (mean age = 23.64, SD = 5.62) underwent an intervention.

During the feasibility study, we noted a high level of attrition, meaning a loss of participants over time. In response to this problem, we decided to offer an incentive in the form of gift vouchers to all participants who fully participated in the research [88].

This approach was based on the results of a study by Fridric and his team [89], who encountered this problem in similar experiments. The participants’ comments also allowed us to improve our program’s form.

Statistics

To allocate the participants into two groups (intervention/experimental and control), we used SPSS (IBM v. 23) random sampling software.

A standard descriptive analysis assessed sample characteristics (in general and by group), and Student’s t-tests and chi-squared test were used to measure the homogeneity between the two groups. To compare the two groups (intervention and control) at each stage (T1 = preassessment, T2 = postassessment and T3 = follow-up), we used the Mann-Whitney U test because the variables: self-esteem, life satisfaction in studies, perceived stress and its two sub factors (perceived helplessness and perceived self-efficacy), psychological distress and its 4 sub factors (somatic symptoms, anxiety/insomnia, social dysfunction, severe depression), did not follow a normal distribution.

To examine intra-group changes in the mental health variables’ scores studied in our research, that is, self-esteem, life satisfaction in studies, perceived stress and its two subfactors (perceived helplessness and perceived self-efficacy), and psychological distress and its 4 subfactors (somatic symptoms, anxiety/insomnia, social dysfunction, severe depression), for each group over time (T1, T2 and T3), we used a repeated measures ANOVA for the variables that followed a normal distribution and the non-parametric equivalent, the Friedman test, for the variables that did not follow a normal distribution. Effect sizes and 95% confidence intervals (95% CI) were calculated using Cohen’s d for the data from participants who completed the questionnaire at baseline and at follow-up. A value of 0.2 is generally interpreted as being suggestive of a small effect size, 0.5 of a medium effect size, and 0.8 of a large effect size [89].

A total of 63 students will be recruited for each group to detect 5% level of significance with 80% power in order to answer the research question (alpha: 0.05; Beta: 0.20 Power: 0.80; The variability (estimated) in (PSS / main variable): 2; Past experience (Feasibility Study), with similar experiments, with similar measurement method (PSS) and similar subjects, suggests that the data will be fairly distributed with an SD of 4; So, n = 2 × [(1.96 + 0.842) 2 × 42] / 22 = 62.809632).

The significance level was fixed at 0.05. All analyses were performed with SPSS (IBM v. 23) and Statistica (v. 12).

Results

Participants’ characteristics

The data in (Table 1) describe the study population, illustrating that the randomization protocol succeeded in balancing the intervention assignments and ensuring equal representation of important characteristics in the study’s population. There were no significant statistical differences between the two groups.

Attrition

A total of 90 participants started the research study after being allocated into the two groups and sending an email confirming their participation after this stage, with 49 participants in the experimental group and 41 in the control group. A total of 55 students finished the study up to the first postassessment (of whom 20 were in the experimental group and 35 in the control group), and 47 students completed the second follow-up postassessment three months later (17 in the experimental group and 30 in the control group).

From the start of the intervention, in the experimental group (first session: 36 participants), there was a rate of general follow-up (for both groups) of 64.94% at the postassessment and 61.04% at the three-month follow-up (Table 2).

thumbnail
Download:
Table 2. Number of participants in each trial.

https://doi.org/10.1371/journal.pone.0200997.t002

The attrition during the intervention went from 36 participants in the first session to 21 participants in the fourth session, with a follow-up rate of 58.33%.

Preliminary analyses

Table 3 shows the means and SDs of the outcome variables at baseline and one-month and three-month follow-ups for the intervention and control groups.

thumbnail
Download:
Table 3. *Summary statistics for all outcome measures at baseline, postintervention and follow-up.

https://doi.org/10.1371/journal.pone.0200997.t003

Table 4 shows the comparisons between the two groups at each of the three stages. We noted only one significant difference at baseline between the two groups (U test: -2.49, p = 0.01): the self-esteem score (M = 28.12, SD = 5.9) was significantly higher in the control group than in the experimental group (M = 25.4, SD = 5.7).

thumbnail
Download:
Table 4. Comparison between both groups (Mann-Whitney test) in three trials.

https://doi.org/10.1371/journal.pone.0200997.t004

At the postassessment stage, there were three significant differences between the two groups in global General Health (U test: -2.16, p = 0.03), with significantly lower scores in the experimental group (M = 10.45, SD = 3.7) than in the control group (M = 12.28, SD = 3.1). Regarding the two subfactors of the GHQ-28, the scores were significantly lower in the experimental group than in the control group for somatic symptoms (U test: -3.6 3, p = 0.001), with (M = 2.1, SD = 1.4) for the experimental group and (M = 3.82, SD = 1.3) for the control group, and for anxiety/insomnia (U test: -2.78, p = 0.04), with (M = 2.45, SD = 2.7) in the experimental group and (M = 3.48, SD = 2.07) in the control group.

The significant difference in self-esteem between the two groups at baseline disappeared at the postassessment and follow-up stages (Table 4)

Intervention effects

Table 5 shows the means, standard deviations and effect sizes (Cohen’s d) of the variables at baseline and at one-month (or immediate) and three-month follow-ups in both the intervention and control groups.

thumbnail
Download:
Table 5. Means, standard deviations and effect sizes (Cohen’s d) of the observed and estimated marginal means for each trial.

https://doi.org/10.1371/journal.pone.0200997.t005

ANOVA (repeated-measures or Friedman) (Table 6) revealed significant effects of the intervention over time for self-esteem (F2,32 = 7.23, p = 0.02); perceived stress (F2,32 = 5.28, p = 0.006) and its two subfactors “perceived helplessness” (F2,32 = 4.51, p = 0.1) and “perceived self-efficacy” (F2,32 = 5.31, p = 0.01); satisfaction in studies (F2,32 = 3.39, p = 0.02); and the global GHQ score (χ2 N = 17, dl = 2 = 15.11, p = 0.001) and three of its subfactors: somatic symptoms (F2,32 = 3.87, p = 0.03), anxiety/insomnia (F2,32 = 7.85, p = 0.001), and social dysfunction (χ2 N = 17, dl = 2 = 17.73, p = 0.001).

thumbnail
Download:
Table 6. ANOVA of repeated measures (F)/ Friedman test (χ2) intra-group.

https://doi.org/10.1371/journal.pone.0200997.t006

However, in the control group, we found significant changes over time regarding self-esteem (F2,58 = 1.18, p = 0.03); and the global GHQ score (χ2 N = 30, dl = 2 = 15.2, p = 0.001) and two of its subfactors: somatic symptoms (F2,58 = 7.25, p = 0.001) and social dysfunction (χ2 N = 30, dl = 2 = 18.01, p = 0.001) (Table 6).

Assessment of the intervention by the participants

To obtain the participants’ opinions on the program, we asked a series of open- and closed-format questions to those who completed all the sessions. Table 7 shows the participants’ answers.

thumbnail
Download:
Table 7. Assessment of the online program by the participants.

https://doi.org/10.1371/journal.pone.0200997.t007

Feedback on the intervention

We received comments throughout the program: after each exercise, after each session, and at the end of the program. These comments varied between positive feedback describing the effect a session or an exercise had on a participant, feedback on the sessions of exercises themselves, and interesting suggestions on how to improve the intervention, for instance, requests for more concrete examples of certain exercises.

Discussion and conclusions

Our work assessed the efficacy of an online stress management program for university students.

We noted an improvement in self-esteem just after the intervention in the group that followed the program, which was maintained after three months of follow-up. As a reminder, before the intervention, self-esteem was higher in the control group than in the experimental group who benefitted from the intervention. However, this difference disappeared after the intervention and after the follow-up assessment three months later.

The results show a significant reduction in the experimental group between the start of the intervention and just after it for the anxiety/insomnia and somatic symptoms of the GHQ-28. However, this difference did not maintain significance after three months of follow-up. (Tables 2 and 3).

The significant decline of psychological distress levels measured by the global GHQ-28 score mirrors the results of the “Funambule” program study [75]. Our results are also similar in terms of the decrease in the level of anxiety [49] and the lack of significant differences between the two groups at follow-up, as also observed in a study by Chiauzzi and his team [49].

The fact that the differences between the groups at follow-up were not maintained could be explained by the fact that the participants in the control group already knew that they would soon be able to benefit from the intervention, which could potentially create positive effects.

The ANOVA analysis of the experimental group showed significant effects of time for all the variables studied in our research. Those variables included self-esteem; perceived stress and its two dimensions, perceived helplessness and perceived self-efficacy; satisfaction in studies; and psychological distress and its four dimensions: somatic symptoms, anxiety/insomnia, social dysfunction and severe depression.

The effect of time for the control group was noted a reduction trend of psychological distress and two of its dimensions, somatic symptoms and social dysfunction. And, on the other hand this effect was also found regarding the level of self-esteem, which also declined between baseline and follow-up three months later (Tables 4 and 5).

Despite the effect of time on both groups, we can note more effects in the group that received the intervention, as there were improvements for most of the studied variables in the experimental group but not the control group. The lack of maintenance of the effects at follow-up three months later in the experimental group (see Fig 2) is in agreement with a study carried out by Zetterqvist and his team [46].

thumbnail
Download:
Fig 2. Mean scores over time based on the experimental condition for both groups and the studied mental health variables.

https://doi.org/10.1371/journal.pone.0200997.g002

Intra-group effect sizes: for the experimental group, large effect sizes (≥0.8) were observed for the measures of self-esteem, perceived stress, satisfaction in studies, and two dimensions of the GHQ-28: somatic symptoms and anxiety/insomnia. Moderate effect sizes (≥0.5) were noted for the social dysfunction dimension of the GHQ-28.

These moderate to large improvements were maintained after 3 months of follow-up. A smaller effect size (0.2) on another GHQ-28 subfactor, severe depression, was noted and maintained at follow-up. The effect sizes show an improvement from small to moderate after three months of follow-up.

However, for the control group, the effect sizes were small (0.2) for the variables of self-esteem, satisfaction in studies, and severe depression, and the effects sizes stayed small three months later. A small effect size for social dysfunction after the intervention increased to a moderate level three months later. For the variable of perceived stress and its two dimensions, somatic symptoms and anxiety/insomnia, the effect size was large and was maintained three months later.

We must highlight the fact that the comparison of effect sizes between the two groups showed a difference in favor of the experimental group, which mirrors the study by Zetterqvist and his team [46]. Regarding the variables mainly targeted by the program, for example, perceived stress, the effect sizes for the experimental group were (-3.443/-3.525) and for the control group, (-1.097 /-1.152). (Table 4).

These results are consistent with those of the study using the “Funambule” program [75], which found improvements in well-being and psychological distress such as anxiety.

These benefits from the intervention of an online stress management program on university students are in accordance with Zetterqvist and his team’s research [46], confirming the fact that an online intervention can have effects similar to those of a face-to-face intervention.

Our results are discordant with the study findings reported by Chiauzai et al. That study did not show any maintenance of significant differences in either group at a six-month follow-up [49]. However, some improvements were noted in this study [49] in terms of self-esteem and depression, in agreement with our results.

To conclude, the advantages of this type of Internet-based intervention relative to other programs lie in it being in a quite short and accessible format, which allows it to reach a larger number of students. Our results show significant effects immediately after the intervention on self-esteem and psychological distress (somatic symptoms and anxiety/insomnia) and an interesting improvement in all the studied variables, such as stress, depression and satisfaction in studies, at a follow-up three months later in the experimental group but not in the control group.

This online program could therefore be offered to students showing problems with stress who have not yet sought professional help. However, even though these results are similar to those obtained in the original program that inspired us, there are time constraints, as indicated by the absence of significant differences between groups three months after the intervention. To better determine the effects of time, a follow-up assessment at six months should be considered. The fact that the control group knows that they may benefit from the intervention should also be kept in mind.

Limitations

Our work has a certain number of limitations: the size of the sample, the measures given by questionnaire only, a follow-up of only 3 months later, and the attacks of November 13th, 2015 in Paris, which could have had a psychological impact on the studied sample.

Moreover, over time there was a loss of participants; we faced this problem during the feasibility study in 2015, which led us to use an incentive, as Fridric and his team recommended [90]. Nevertheless, despite this incentive, our sample remained small. It is therefore still necessary to improve this intervention and pursue research that resolves the methodological problems raised by this type of intervention.

Acknowledgments

We thank the Scientific Interest Group “Jeu et société”

We thank Hassan Njifon Nsangou for correcting the English version of this article.

References

  1. 1. Robotham D, Julian C. Stress and the higher education student: a critical review of the literature. J Furth High Educ. 2006;30(2):107–17.
  2. 2. Lovell GP, Nash K, Sharman R, Lane BR. A cross-sectional investigation of depressive, anxiety, and stress symptoms and health-behavior participation in Australian university students. Nurs Health Sci. 2015;17(1):134–42. pmid:24799077
  3. 3. Crawford G, Burns SK, Chih HJ, Hunt K, Tilley PM, Hallett J, et al. Mental health first aid training for nursing students: a protocol for a pragmatic randomised controlled trial in a large university. BMC Psychiatry. 2015;15:26. pmid:25886615
  4. 4. Abu-Ghazaleh SB, Sonbol HN, Rajab LD. A longitudinal study of psychological stress among undergraduate dental students at the University of Jordan. BMC Med Educ. 2016;16:90. pmid:26968682
  5. 5. Deasy C, Coughlan B, Pironom J, Jourdan D, Mannix-McNamara P. Psychological Distress and Coping amongst Higher Education Students: A Mixed Method Enquiry. PLOS ONE. 2014;9(12):e115193. pmid:25506825
  6. 6. Grant F, Guille C, Sen S. Well-Being and the Risk of Depression under Stress. PLOS ONE. 2013;8(7):e67395. pmid:23840872
  7. 7. Montero-Marín J, Demarzo MMP, Stapinski L, Gili M, García-Campayo J. Perceived Stress Latent Factors and the Burnout Subtypes: A Structural Model in Dental Students. PLOS ONE. 2014;9(6):e99765. pmid:24927260
  8. 8. Vandentorren S, Verret C, Vignonde M, Maurice-Tison S. Besoins d’information en santé des étudiants au service inter-universitaire de médecine préventive de Bordeaux. Santé Publique. 2005;17(1):47–56.
  9. 9. Saleh D, Camart N, Romo L. On what Resources can the Students Rely on: Satisfaction with Life, Self-Esteem and SelfEfficacy. Ann Psychiatry Ment Health. 2016;4(2: 1062).
  10. 10. Moreira JFG, Telzer EH. Changes in family cohesion and links to depression during the college transition. J Adolesc. 2015;43:72–82. pmid:26058003
  11. 11. Walsh JM, Feeney C, Hussey J, Donnellan C. Sources of stress and psychological morbidity among undergraduate physiotherapy students. Physiotherapy. 2010;96(3):206–12. pmid:20674652
  12. 12. Milojevich HM, Lukowski AF. Sleep and Mental Health in Undergraduate Students with Generally Healthy Sleep Habits. PLOS ONE. 2016;11(6):e0156372. pmid:27280714
  13. 13. Luce KH, Crowther JH, Pole M. Eating Disorder Examination Questionnaire (EDE-Q): Norms for undergraduate women. Int J Eat Disord. 2008;41(3):273–6. pmid:18213686
  14. 14. Lavender JM, De Young KP, Anderson DA. Eating Disorder Examination Questionnaire (EDE-Q): Norms for undergraduate men. Eat Behav. 2010;11(2):119–21. pmid:20188296
  15. 15. Torstveit MK, Aagedal-Mortensen K, Stea TH. More than Half of High School Students Report Disordered Eating: A Cross Sectional Study among Norwegian Boys and Girls. PLOS ONE. 2015;10(3):e0122681. pmid:25825877
  16. 16. Lindgren KP, Wiers RW, Teachman BA, Gasser ML, Westgate EC, Cousijn J, et al. Attempted Training of Alcohol Approach and Drinking Identity Associations in US Undergraduate Drinkers: Null Results from Two Studies. PLOS ONE. 2015;10(8):e0134642. pmid:26241316
  17. 17. Rogowska AM. Problematic use of psychoactive substances in undergraduates: a comparison of four patterns of substance use. J Subst Use. 2016;21(3):304–8.
  18. 18. Choueiry N, Salamoun T, Jabbour H, Osta NE, Hajj A, Khabbaz LR. Insomnia and Relationship with Anxiety in University Students: A Cross-Sectional Designed Study. PLOS ONE. 2016;11(2):e0149643. pmid:26900686
  19. 19. Pilcher JJ, Ginter DR, Sadowsky B. Sleep quality versus sleep quantity: Relationships between sleep and measures of health, well-being and sleepiness in college students. J Psychosom Res. 1997;42(6):583–96. pmid:9226606
  20. 20. Adlaf EM, Gliksman L, Demers A, Newton-Taylor B. The Prevalence of Elevated Psychological Distress Among Canadian Undergraduates: Findings from the 1998 Canadian Campus Survey. J Am Coll Health. 2001;50(2):67–72. pmid:11590985
  21. 21. Baykan Z, Naçar M, Cetinkaya F. Depression, anxiety, and stress among last-year students at Erciyes University Medical School. Acad Psychiatry J Am Assoc Dir Psychiatr Resid Train Assoc Acad Psychiatry. 2012;36(1):64–5.
  22. 22. Bewick B, Koutsopoulou G, Miles J, Slaa E, Barkham M. Changes in undergraduate students’ psychological well-being as they progress through university. Stud High Educ. 2010;35(6):633–45.
  23. 23. O’Brien L, Mathieson K, Leafman J, Rice-Spearman L. Level of stress and common coping strategies among physician assistant students. J Physician Assist Educ Off J Physician Assist Educ Assoc. 2012;23(4):25–9.
  24. 24. Stallman HM. Psychological distress in university students: A comparison with general population data. Aust Psychol. 2010;45(4):249–57.
  25. 25. Maniecka-Bryła I, Bryła M, Weinkauf A, Dierks M-L. The international comparative study of the health status of medical university students in Lodz and Hanover. Przegl Lek. 2005;62(3):63–8.
  26. 26. Strenna L, Chahraoui K, Vinay A. Santé psychique chez les étudiants de première année d’école supérieure de commerce: liens avec le stress de l’orientation professionnelle, l’estime de soi et le coping. Orientat Sc Prof. 2009;(38/2):183–204.
  27. 27. Lafay N, Manzanera C, Papet N, Marcelli D, Senon J. Les états dépressifs de la post-adolescence. Résultats d’une enquête menée chez 1521 étudiants de l’université de Poitiers. Ann Med Psychol (Paris). 2003;161(2):147–51.
  28. 28. Humphris G, Blinkhorn A, Freeman R, Gorter R, Hoad-Reddick G, Murtomaa H, et al. Psychological stress in undergraduate dental students: baseline results from seven European dental schools. Eur J Dent Educ. 2002;6(1):22–9. pmid:11872070
  29. 29. Nerdrum P, Rustøen T, Rønnestad MH. Student Psychological Distress: A psychometric study of 1750 Norwegian 1st-year undergraduate students. Scand J Educ Res. 2006;50(1):95–109.
  30. 30. Dyrbye LN, Harper W, Durning SJ, Moutier C, Thomas MR, Massie FS Jr, et al. Patterns of distress in US medical students. Med Teach. 2011;33(10):834–9. pmid:21942482
  31. 31. Boujut E, Koleck M, Bruchon-Schweitzer M, Bourgeois ML. Mental health among students: A study among a cohort of freshmen. Ann Méd-Psychol. 2009;167(9):662–8.
  32. 32. Furr SR, Westefeld JS, McConnell GN, Jenkins JM. Suicide and depression among college students: A decade later. Prof Psychol Res Pract. 2001;32(1):97.
  33. 33. Dahlin M, Joneborg N, Runeson B. Stress and depression among medical students: a cross-sectional study. Med Educ. 2005;39(6):594–604. pmid:15910436
  34. 34. Wong JGWS, Cheung EPT, Chan KKC, Ma KKM, Tang SW. Web-based survey of depression, anxiety and stress in first-year tertiary education students in Hong Kong. Aust N Z J Psychiatry. 2006;40(9):777–82. pmid:16911753
  35. 35. Bayram N, Bilgel N. The prevalence and socio-demographic correlations of depression, anxiety and stress among a group of university students. Soc Psychiatry Psychiatr Epidemiol. 2008;43(8):667–72. pmid:18398558
  36. 36. Shamsuddin K, Fadzil F, Ismail WSW, Shah SA, Omar K, Muhammad NA, et al. Correlates of depression, anxiety and stress among Malaysian university students. Asian J Psychiatry. 2013;6(4):318–23.
  37. 37. Bughi SA, Sumcad J, Bughi S. Effect of brief behavioral intervention program in managing stress in medical students from two southern California universities. Med Educ Online. 2006;11:1–8.
  38. 38. Koochaki GM, Charkazi A, Hasanzadeh A, Saedani M, Qorbani M, Marjani A. Prevalence of stress among Iranian medical students: a questionnaire survey. East Mediterr Health J. 2011;17(7):593–8. pmid:21972483
  39. 39. Zivin K, Eisenberg D, Gollust SE, Golberstein E. Persistence of mental health problems and needs in a college student population. J Affect Disord. 2009;117(3):180–5. pmid:19178949
  40. 40. Dumont M. Funambule Pour une gestion équilibrée du stress [Internet]. Québec: Septembre éditeur; 2012. http://www.septembre.com/livres/funambule-1259.html
    • 41. Leppin AL, Bora PR, Tilburt JC, Gionfriddo MR, Zeballos-Palacios C, Dulohery MM, et al. The Efficacy of Resiliency Training Programs: A Systematic Review and Meta-Analysis of Randomized Trials. PLOS ONE. 2014;9(10):e111420. pmid:25347713
    • 42. de Brouwer SJM, Kraaimaat FW, Sweep FCGJ, Donders RT, Eijsbouts A, van Koulil S, et al. Psychophysiological Responses to Stress after Stress Management Training in Patients with Rheumatoid Arthritis. PLOS ONE. 2011;6(12):e27432. pmid:22162990
    • 43. Lai ESY, Kwok C-L, Wong PWC, Fu K-W, Law Y-W, Yip PSF. The Effectiveness and Sustainability of a Universal School-Based Programme for Preventing Depression in Chinese Adolescents: A Follow-Up Study Using Quasi-Experimental Design. PLOS ONE. 2016;11(2):e0149854. pmid:26921275
    • 44. Lay B, Drack T, Bleiker M, Lengler S, Blank C, Rössler W. Preventing Compulsory Admission to Psychiatric Inpatient Care: Perceived Coercion, Empowerment, and Self-Reported Mental Health Functioning after 12 Months of Preventive Monitoring. Front Psychiatry [Internet]. 2015 [cited 2016 Aug 26];6(161). Available from: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4650287/
    • 45. Saleh D, Camart N, Romo L. Intervention de gestion du stress par Internet chez les étudiants: revue de la littérature. Ann Méd-Psychol Rev Psychiatr. 2016;5.
    • 46. Zetterqvist K, Maanmies J, Ström L, Andersson G. Randomized controlled trial of Internet-based stress management. Cogn Behav Ther. 2003;32(3):151–60. pmid:16291546
    • 47. Vliet HV, Andrews G. Internet-based course for the management of stress for junior high schools. Australas Psychiatry. 2009;43(4):305–9.
    • 48. George DR, Dellasega C, Whitehead MM, Bordon A. Facebook-based stress management resources for first-year medical students: A multi-method evaluation. Comput Hum Behav. 2013;29(3):559–62.
    • 49. Chiauzzi E, Brevard J, Thurn C, Decembrele S, Lord S. MyStudentBody–Stress: An online stress management intervention for college students. J Health Commun. 2008;13(6):555–72. pmid:18726812
    • 50. Gajecki M, Berman AH, Sinadinovic K, Andersson C, Ljótsson B, Hedman E, et al. Effects of Baseline Problematic Alcohol and Drug Use on Internet-Based Cognitive Behavioral Therapy Outcomes for Depression, Panic Disorder and Social Anxiety Disorder. PLOS ONE. 2014;9(8):e104615. pmid:25122509
    • 51. Broman-Fulks J. Internet-Based Interventions for Traumatic Stress-Related Mental Health Problems: A Review and Suggestion for Future Research. 2009;
    • 52. Ebert DD, Zarski A-C, Christensen H, Stikkelbroek Y, Cuijpers P, Berking M, et al. Internet and Computer-Based Cognitive Behavioral Therapy for Anxiety and Depression in Youth: A Meta-Analysis of Randomized Controlled Outcome Trials. PLOS ONE. 2015;10(3):e0119895. pmid:25786025
    • 53. Imamura K, Kawakami N, Furukawa TA, Matsuyama Y, Shimazu A, Umanodan R, et al. Effects of an Internet-Based Cognitive Behavioral Therapy (iCBT) Program in Manga Format on Improving Subthreshold Depressive Symptoms among Healthy Workers: A Randomized Controlled Trial. PLOS ONE. 2014;9(5):e97167. pmid:24844530
    • 54. Hedman E, Andersson G, Lindefors N, Gustavsson P, Lekander M, Rück C, et al. Personality Change following Internet-Based Cognitive Behavior Therapy for Severe Health Anxiety. PLOS ONE. 2014;9(12):e113871. pmid:25437150
    • 55. Hoek W, Schuurmans J, Koot HM, Cuijpers P. Effects of Internet-Based Guided Self-Help Problem-Solving Therapy for Adolescents with Depression and Anxiety: A Randomized Controlled Trial. PLOS ONE. 2012;7(8):e43485. pmid:22952691
    • 56. Morgan AJ, Rapee RM, Bayer JK. Prevention and early intervention of anxiety problems in young children: A pilot evaluation of Cool Little Kids Online. Internet Interv. 2016;4:105–12.
    • 57. Ye Y, Zhang Y, Chen J, Liu J, Li X, Liu Y, et al. Internet-Based Cognitive Behavioral Therapy for Insomnia (ICBT-i) Improves Comorbid Anxiety and Depression—A Meta-Analysis of Randomized Controlled Trials. PLOS ONE. 2015;10(11):e0142258. pmid:26581107
    • 58. Amstadter AB, Broman-Fulks J, Zinzow H, Ruggiero KJ, Cercone J. Internet-based interventions for traumatic stress-related mental health problems: a review and suggestion for future research. Clin Psychol Rev. 2009;29(5):410–420. pmid:19403215
    • 59. Sundström C, Gajecki M, Johansson M, Blankers M, Sinadinovic K, Stenlund-Gens E, et al. Guided and Unguided Internet-Based Treatment for Problematic Alcohol Use–A Randomized Controlled Pilot Trial. PLOS ONE. 2016;11(7):e0157817. pmid:27383389
    • 60. Moore MJ, Soderquist J, Werch C. Feasibility and Efficacy of a Binge Drinking Prevention Intervention for College Students Delivered via the Internet Versus Postal Mail. J Am Coll Health. 2005;54(1):38–44. pmid:16050327
    • 61. Alaoui SE, Ljótsson B, Hedman E, Kaldo V, Andersson E, Rück C, et al. Predictors of Symptomatic Change and Adherence in Internet-Based Cognitive Behaviour Therapy for Social Anxiety Disorder in Routine Psychiatric Care. PLOS ONE. 2015;10(4):e0124258. pmid:25893687
    • 62. Carlbring P, Furmark T, Steczkó J, Ekselius L, Andersson G. An open study of Internet-based bibliotherapy with minimal therapist contact via email for social phobia. Clin Psychol. 2006;10(1):30–8.
    • 63. Hughes JA, Phillips G, Reed P. Brief Exposure to a Self-Paced Computer-Based Reading Programme and How It Impacts Reading Ability and Behaviour Problems. PLOS ONE. 2013;8(11):e77867. pmid:24223125
    • 64. Billings DW, Cook RF, Hendrickson A, Dove DC. A web-based approach to managing stress and mood disorders in the workforce. J Occup Environ Med. 2008;50(8):960–8. pmid:18695455
    • 65. Servant D. Les programmes de gestion du stress au travail sont-ils efficaces? Lett Psychiatre. 2011;7(1):25–8.
    • 66. Jung Y-H, Ha TM, Oh CY, Lee US, Jang JH, Kim J, et al. The Effects of an Online Mind-Body Training Program on Stress, Coping Strategies, Emotional Intelligence, Resilience and Psychological State. PLOS ONE. 2016;11(8):e0159841. pmid:27479499
    • 67. Hoch DB, Watson AJ, Linton DA, Bello HE, Senelly M, Milik MT, et al. The Feasibility and Impact of Delivering a Mind-Body Intervention in a Virtual World. PLOS ONE. 2012;7(3):e33843. pmid:22470483
    • 68. Currie SL, McGrath PJ, Day V. Development and usability of an online CBT program for symptoms of moderate depression, anxiety, and stress in post-secondary students. Comput Hum Behav. 2010;26(6):1419–26.
    • 69. Day V, McGrath PJ. Internet-based guided self-help for university students with anxiety, depression and stress: A randomized controlled clinical trial. Behav Res Ther. 2013;51(7):344–351. pmid:23639300
    • 70. LaChausse RG. My Student Body: Effects of an Internet-Based Prevention Program to Decrease Obesity Among College Students. J Am Coll Health. 2012;60(4):324–30. pmid:22559092
    • 71. Arpin-Cribbie C, Irvine J, Ritvo P. Web-based cognitive-behavioral therapy for perfectionism: A randomized controlled trial. Psychother Res. 2012;22(2):194–207. pmid:22122217
    • 72. Fridrici M, Lohaus A. Stress-prevention in secondary schools: online-versus face-to-face-training. Health Educ. 2009;109(4):299–313.
    • 73. Baumeister H, Reichler L, Munzinger M, Lin J. The impact of guidance on Internet-based mental health interventions—A systematic review. Internet Interv. 2014;1(4):205–15.
    • 74. Schulz KF, Altman DG, Moher D. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c332. pmid:20332509
    • 75. Dumont M, Leclerc D, Massé L, McKinnon S. Étude de validation du programme Funambule: pour une gestion équilibrée du stress des adolescents. Éducation Francoph. 2015;43(2):154.
    • 76. Lazarus RS, Folkman S. Coping and adaptation. Handb Behav Med. 1984;282–325.
      • 77. Cohen S, Kamarck T, Mermelstein R. A global measure of perceived stress. J Health Soc Behav. 1983;24(4):385–96. pmid:6668417
      • 78. Bellinghausen L, Collange J., Botella M., Emery J-L, Albert É. Factorial validation of the French scale for perceived stress in the workplace. Santé Publique. 2009;21(4):365–73. pmid:20101815
      • 79. Collange J, Bellinghausen L, Chappé J, Saunder L, Albert E. Stress perçu: à partir de quel seuil devient-il un facteur de risque pour les troubles anxiodépressifs? Arch Mal Prof Environ. 2013;74(1):7–15.
      • 80. Rosenberg M. Society and the adolescent self-image. [Internet]. Princeton University Press. 1965. 326 p. https://www.vitalsource.com/en-uk/products/society-and-the-adolescent-self-image-morris-rosenberg-v9781400876136
        • 81. Vallieres EF, Vallerand RJ. Traduction et validation canadienne-française de l’échelle de l’estime de soi de Rosenberg. Int J Psychol. 1990;25(2):305–16.
        • 82. Chabrol H, Carlin E, Michaud C, Rey A, Cassan D, Juillot M, et al. Étude de l’échelle d’estime de soi de Rosenberg dans un échantillon de lycéens. Neuropsychiatr Enfance Adolesc. 2004;52(8):533–6.
        • 83. Blais MR, Vallerand RJ, Pelletier LG, Brière NM. L’échelle de satisfaction de vie: Validation canadienne-française du “Satisfaction with Life Scale”. Can J Behav Sci Can Sci Comport. 1989;21(2):210–23.
        • 84. Vallerand RJ, Bissonnette R. Construction et validation de l’Échelle de Satisfaction dans les Études (ESDE). [Construction and validation of the Scale of Satisfaction in Studies]. Can J Behav Sci Can Sci Comport. 1990;22(3):295–306.
        • 85. Goldberg DP, Hillier V. A scaled version of the General Health Questionnaire. Psychol Med. 1979;9(01):139–45.
        • 86. Bolognini M, Bettschart W, Zehnder-Gubler M, Rossier L. The validity of the French version of the GHQ-28 and PSYDIS in a community sample of 20 year olds in Switzerland. Eur Arch Psychiatry Neurol Sci. 1989;238(3):161–8. pmid:2721534
        • 87. Guelfi J-D. L’évaluation clinique standardisée en psychiatrie. Lavaur, France: Éd. médicales Pierre Fabre; 1993. 411 p.
          • 88. Saleh D, Camart N, Sbeira F, Romo L. Internet-Based Stress Management Intervention: Feasibility Study. EC Psychol PSYCHIATRYShort. 2017 Jun;4(1):27–33.
          • 89. Cohen J. A power primer. Psychol Bull. 1992;112(1):155–9. pmid:19565683
          • 90. Fridrici M, Lohaus A, Glass C. Effects of incentives in web-based prevention for adolescents: Results of an exploratory field study. Psychol Health. 2009;24(6):663–75. pmid:20205019