Introduction

Since the first prospective report in 2004 of an inverse association between short sleep and body mass index (BMI) in a cohort of 496 young adults followed for 13 years [1], a growing body of evidence has accumulated on the negative consequences of sleep deprivation on weight, metabolism, and the endocrine system. Chronic sleep deprivation [2] and social jet lag, the modern tendency of living a lifestyle in dissonance with the inherent biological clock [3], have been listed among the emerging factors contributing to the modern obesogenic environment in industrialized societies. A meta-analysis of 45 studies with a total of more than 600,000 children and adults reported an increased risk of obesity in subjects with short sleep duration (odds ratio: 1.55, adults; 1.89, children) [4]. The existence of an inverse relationship between short sleep and BMI is undisputed [5] especially in obese subjects [6], but it remains to be established whether extending sleep would result in weight loss. Prior circumstantial evidence that extending sleep duration and improving sleep hygiene may lead to weight loss has recently been complemented by additional reports providing more direct evidence for a relationship between sleep and weight. A small, controlled, randomized, cross-over study conducted in overweight (average BMI 27 kg/m²) men and women showed that moderate (5.5 h vs. 8.5 h) sleep deprivation limits diet-induced fat loss by approximately 50% or 0.7 kg and is accompanied by a shift in relative substrate oxidation toward oxidation of less fat [7]. Furthermore, a 6 year, large, longitudinal study found that subjects sleeping on average less than 6 h/night gained over time approximately 60% more visceral fat than subjects sleeping more than 9 h/night [8].

After bariatric surgery, sleep quality improves substantially and sleep duration increases by approximately 50 min [9] and better sleep quality is associated with higher chances of success in weight loss programs [10] specifically with more fat loss [11].

The existence of a causal relationship is best tested in randomized, controlled clinical trials. Nevertheless, performing randomized trials of behavioral intervention, presents unique challenges. It is difficult to mask subjects to treatment allocation, compliance with study requirements is problematic and requires a high level of participation, and in a randomized study subjects may behave in dissonance with group allocation. In the case of sleep extension subjects in the control group may also decide to extend sleep duration and vice versa.

The Sleep Extension Study is the first randomized, controlled trial of sleep extension in chronically sleep-deprived (less than 6.5 h per night) obese subjects. The study design has been previously reported [12]. The main study hypothesis was that sleep extension would cause weight loss and induce metabolic and endocrine improvements. Since the potential impact of the interactions between participants and study team on sleep behavior and various biochemical parameters in the early phases of a behavioral study of sleep extension have not been fully characterized, we investigated it in the above chronically sleep-deprived obese population. In a period of about 2.7 months between screening and randomization (i.e., prior to any intervention), we observed substantial improvements in sleep and select biochemical parameters, most prominently in glucose homeostasis. The improvements were transient and most of them disappeared in the absence of intervention about four months after screening. We propose that the observed changes were due to the Hawthorne effect.

Methods

The protocol for this trial and supporting CONSORT checklist are available as supporting information; see Checklist S1 and Protocol S1.

Screening, Randomization and Run-In Study Visits and Consents

Two hundred and forty individuals were screened in person at the screening visit (Visit 1) (Fig. 1). The main goal of this visit was to ensure that participants met eligibility criteria by eliminating those candidates that had social, medical, or other reasons that could be ascertained in the first in-person encounter with the study team and would prevent enrollment into the active study. The screening consent informed prospective participants that sleep duration and quality would be assessed at each visit, described the devices used, and the procedures involved in the study. This consent also mentioned that eligible participants in the subsequent visit would be assigned by chance to an Intervention or a Comparison Group, should they have qualified for the study. Individuals were asked to wear actigraphy monitors, and complete sleep questionnaires and diaries for two consecutive weeks in a month.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Study flow diagram.

https://doi.org/10.1371/journal.pone.0104176.g001

At Visit 2 (randomization visit), 125 of the initially screened 240 individuals were randomized in a 1.3∶1 ratio (Intervention/Comparison), thus assigning 72 to the Intervention Group and 53 to the Comparison Group (Fig. 1). The main goal of this visit was to ensure that participants would be able to comply with the various requirements that the study entailed, based on group allocation. Eligibility was determined based on several parameters of sleep duration such as actigraphy, questionnaires, and diaries. These results were shared with the candidates and those eligible were consented for randomization. The requirements of study participation were described in details in the randomization consent which mentioned that randomization to group assignment would be performed by a computer, based on a specifically designed algorithm. Once the subject was randomized, group assignment was disclosed and the subject was asked to sign another consent specific for the Intervention or for the Comparison Group, depending on the results of the randomization. Subjects in the Intervention Group were instructed to increase sleep duration up to 7.5 h per night, following a personalized sleep plan devised in close collaboration with the subject and based on the information collected up to that point. Strategies included consistent bedtime routine, avoiding caffeine, alcohol, heavy meals and exercise prior to bedtime, creating an environment conducive to sleep, and controlling bedroom light and temperature. The Comparison Group was asked not to modify the existing short sleep habits for the remaining time of the Efficacy Phase of the study. The rest of the two consents were identical. At the Run-In Visit, the sleep results between Visit 2 and Visit 3 were discussed with each participant.

Of all participants randomized, a total of 116 subjects made it to the run-in visit (Visit 3): 50 in the Comparison Group and 66 in the Intervention Group (Fig. 1). The goal of this visit was to verify the level of compliance with the group-specific instructions received at Visit 2, namely to increase their sleep duration (Intervention Group) or to maintain their sleep duration (Comparison Group). In addition, subjects in both groups were required to record a two-week diary of sleep time and were asked not to attempt to lose weight.

Statistical Analysis

The Kolmogorov-Smirnov test was used to assess normality of data; logarithmic transformations were employed for skewed variables (e.g., insulin concentrations) before parametric tests were performed. Statistical tests used to compare pairwise changes between visists included paired Student t-test for difference in mean values, Wilcoxon test for skewed variables and, as appropriate, Chi square or McNemar test for difference in counts and frequency. Mixed models analysis accounting for repeated measures and missing data was used to assess changes between groups and within each group over Visits 1–3. Post-hoc tests were conducted using the Bonferroni correction of significance. P-values were also adjusted for multiple testing according to the independent classes of parameters (n = 4), namely, sleeping, anthropometric, glycemic and lipid characteristics. Analyses were performed using SPSS (version 21, IBM Corp, Armonk, NY, USA). Data are presented as mean±standard deviation (SD) in the tables and standard error of the mean (SEM) in the error bars of Figs. 2 and 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Percent changes from screening in weight and waist circumference (WC), and in systolic and diastolic blood pressure (upper four panels, A–D), and the changes in sleep duration by diaries and by actigraphy watch (AW), and the changes in PSQI, and Epworth scores (lower four panels, E–H).

Error bars represent 1 SEM. Significant changes from screening: *P<0.05, **P<0.01, and ***P<0.001; all adjusted for multiple testing. Significant between group differences (Comparison vs. Intervention) are shown with actual P-value, as appropriate, above or below the pair of columns. The time intervals between Visit 1 and Visit 2, as well as Visit 1 and Visit 3 were not different between the two groups (P = 0.30 and P = 0.36, respectively).

https://doi.org/10.1371/journal.pone.0104176.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Percent changes from screening in glucose parameters and in the prevalence of metabolic syndrome (upper four panels, A–D) and the changes in lipid levels (lower four panels, E–H).

Error bars represent 1 SEM. Significant changes from screening: *P<0.05, **P<0.01, and ***P<0.001; all adjusted for multiple testing. Significant between group differences (Comparison vs. Intervention) are shown with actual P-value, as appropriate, above or below the pair of columns. The time intervals between Visit 1 and Visit 2, as well as Visit 1 and Visit 3 were not different between the two groups (P = 0.30 and P = 0.36, respectively).

https://doi.org/10.1371/journal.pone.0104176.g003

Results

Characterization of the Participants at Screening (Visit 1) in the Full Study Population

Of the 240 individuals screened in person at Visit 1, 125 participants completed Visit 2 (thereby referred to as “full study population”) and 116 completed all three visits. Mean age was 41 years, mean BMI 38 kg/m², approximately 75% were women, and more than half were black at the time of screening (Table 1). Average sleep duration was less than 6 h: 357 min by diaries, and 344 min by actigraphy. Sleep efficiency was 80% and the average score for ESS and PSQI questionnaires was 9. Thus, sleep parameters reflected inadequate sleep, both in quantity and quality at the time of screening.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of Participants at Screening (Visit 1) and Randomization (Visit 2).

https://doi.org/10.1371/journal.pone.0104176.t001

Characterization of the Participants at Screening (Visit 1) According to Subsequent Randomization Status

Since at randomization (Visit 2) participants received different instructions about sleep behavior depending on their group assignment (Intervention vs. Comparison), we could not compare results of the “full study population” at the run-in visit (Visit 3) with results at screening and randomization. In order to follow up on possible changes up to Visit 3, we therefore analyzed the percent changes at Visit 2 and Visit 3 compared to Visit 1 according to their “future” group allocation in the 116 participants who reached the run-in phase (Visit 3).

Table 2 reports subjects' characteristics at screening, categorized according to the two allocation groups. Randomization at Visit 2 was successful in achieving comparable characteristics between the Comparison and Intervention Groups for: demographic, sleep and lipid parameters (Table 2). Differences were found in one anthropometric parameter: the waist circumference was 5 cm larger in the Intervention group. This group also tended to have higher body weight and neck circumference, and had significantly higher fasting glucose concentrations and a worse Quicki index.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Characteristics at screening (Visit 1) of participants who completed randomization.

https://doi.org/10.1371/journal.pone.0104176.t002

Discussion

In our Sleep Extension Study of chronically short-sleeping obese adults, we found that sleep duration, sleep quality, and sleepiness, diastolic blood pressure, and insulin sensitivity markedly improved prior to any intervention in about 2.7 months between screening and randomization. The proportion of subjects with pre-diabetes was halved and the proportion of subjects with metabolic syndrome decreased by one third. Since no life-style modifications had been recommended at that point, we were surprised by the magnitude and consistency of the improvements observed after enrollment across a variety of different parameters. Of importance is that several of these parameters were of biochemical nature and therefore measured objectively.

Between screening and randomization, self-reported sleep duration increased by an average of 30 min, which is a clinically meaningful improvement. For comparison, a drug for the treatment of insomnia was recently approved in the US based on a sleep extension of approximately 14 min [18]. The 8% reduction in fasting glucose, paralleled by a 26% decrease in fasting insulin, and by a 7% improvement in Quicki index, an accepted marker of insulin sensitivity, are also remarkable in our subjects. These changes clearly exceeded the “normal” intra-individual fluctuations for most of the parameters measured, which is estimated to be, for example, approximately 4%, and 21% for fasting glucose and fasting insulin, respectively [19]. Further, in the absence of active intervention, glucose parameters tend to get worse over time and certainly do not improve in subjects with the degree of obesity seen in our study. A possible explanation for improvements in glucose homeostasis parameters is the parallel improvement in sleep. Besides temporal sequence, biological plausibility, magnitude and direction of changes and published reports favor this possibility. A large body of evidence indicates that acute sleep deprivation causes insulin resistance in healthy, lean subjects [20]. A meta-analysis of randomized, controlled, one-year duration studies of diet and exercise for patients with metabolic syndrome reported reduced mean values of approximately 11 mg/dL of fasting glucose [21], quite close to the approximately 8 mg/dL reduction in fasting glucose that we observed here. Lipid parameters also showed statistical improvements between screening and randomization; the changes were, however, of smaller magnitude than changes in glucose parameters and insufficient to cause reversal to normal values in a significant proportion of participants. A relationship between sleep and the lipid concentrations has been reported before. Both short and long sleep were associated in women but not in men with abnormal triglycerides, HDL-C, and LDL-C levels in a large survey conducted in Japan of approximately 4000 adult men and women [22]. Nonetheless, in addition to the documented sleep extension, we should also consider the possibility that the marked metabolic changes observed in our subjects between screening and randomization may have been the consequence of unrecognized life-style changes, e.g., those in diet and exercise, as well. Finally, a 1/3 decline in the proportion of our subjects with metabolic syndrome between screening and randomization was driven by the improvements observed in blood pressure, fasting glucose and waist circumference, rather than by improvements in the lipid components of the metabolic syndrome, namely triglycerides and HDL-C.

Due to the study design, follow-up on the changes beyond randomization could only be analyzed in the two groups (Comparison and Intervention Group) of our study population. Once participants were randomized, improvements in sleep parameters were consistently observed only in the group advised to extend sleep at the time of randomization (Intervention Group). Since these subjects have been made fully aware of the goals of the study and have been consented to change their sleep behavior, these effects are expected and they are understood as a direct result of study intervention. On the other hand, irrespective of group allocation at randomization, the initial improvements in parameters of the glucose homeostasis completely disappeared by the time of the Run-In Visit, about 4 months after screening. This was surprising, since this was observed even in the Intervention Group that sustained sleep improvements at the run-in visit.

While we cannot fully explain the specific cause(s) of sleep and biochemical improvements observed between screening and randomization in our study population, we propose that they are consistent with the Hawthorne effect. Of note is that, to the best of our knowledge, no objectively measurable changes have been reported yet as a consequence of the Hawthorne effect. This effect is named after experiments conducted in the 1930s' at the Hawthorne Works plant of Western Electric by researchers interested in the relationship between productivity and work environment. Surprisingly, contact with the research team, rather than manipulations of the experimental variables resulted in improved productivity [23]. The Hawthorne effect did historically refer to behavioral changes (increased productivity) that derive from being the object of observation, rather than from the specific nature of the modifications made to the environment in which the study subjects operate, such as lighting in the historical example. Schwartz et al. recently described the “pure” effect of participation in another consumer setting [24]. In a study of residential consumers' electricity use, five post-cards were sent to a random sample of customers, notifying them at first and then reminding them of their participation in a study of household electricity use. Being reminded that they were in a study was associated with a 2.7% reduction in monthly use of electricity. When the intervention ended, energy consumption returned to pre-study values. Typically, the Hawthorne effect is quite transient and it disappears once study participation and observation of the subjects are stopped. Since its original description, the Hawthorne effect has been also reported in the field of sociology, psychology, as well as in several fields of clinical medicine. For instance, patients who were aware of being in a study had less post-operative knee pain than subjects who were unaware [25]. Being aware of ongoing monitoring was associated with improvement in decontamination hospital practices [26].

Nevertheless, we need to note that some features of our findings are not typical for the classic Hawthorne effect. Until the Randomization Visit, our subjects were not yet “participating” in the study, they were “candidates” for participation. Hence they were not “naïve” about the goal of the study (sleep extension) and its general finalities by receiving a fair amount of knowledge just from the screening informed consent alone. Further, there may have been a potential bias in the study admission procedure after screening as enrollment was contingent upon compliance with study requirements in terms of collection of information and documentation, in other words, of “good” behavior. It is possible that candidates, especially if committed to be accepted into the study, decided, either consciously or unconsciously, that in order to secure participation, they would strictly adhere to the requirements of recording their sleep, weight and blood pressure, and wearing wrist actigraphy for weeks; thus, they may have changed their behavior in anticipation of study entry. This participant selection procedure was necessary because poor adherence to study requirements per se, whether of behavioral nature or of the necessity to take a study drug (either active drug or placebo) is considered a negative predictor of study outcome. In the Canadian Amiodarone Myocardial Infarction Arrhythmia Trial, poor adherence to study medication, both active and placebo, was associated with a two-fold increase in relative risk of sudden death of 2.1 [27]. Finally, at variance with the classical Hawthorne effect that typically disappears once study participation and observation of the subjects are stopped, our subjects continued to participate and to be observed beyond randomization, yet the initial improvements in sleep, anthropometric and biochemical parameters were largely or completely gone without intervention about 4 months after screening.

Another issue is that, despite supporting evidence from a variety of sources, the existence of the Hawthorne effect is not universally accepted. Criticisms include that the original reports of this effect are “nebulous” and that the effect is both ill-defined and seldom reproducible [28]. To address these criticisms, clinical researchers are more often characterizing observations that would have been previously categorized as “Hawthorne” effect in a different construct called “research participation effect” to include the effects that may derive from patient preferences in the context of the various demands of clinical trials [29]. This may be especially true for the earlier stages of a trial prior to randomization, especially if more than a few days separate these stages and the study is of behavioral nature as in our case.

To further complicate matters, it is a widely accepted practice for ethical and regulatory purposes in randomized clinical trials to recommend to participants, regardless of group allocation, adherence to the “standard of care”. Thus, in weight loss trials, diet and exercise are prescribed in addition to drug and placebo, because these non-pharmacological interventions represent the currently accepted standard of care for obesity. The typical design of a randomized controlled trial does not allow dissecting out how much of the therapeutic effect may be due to the study drug vs. the implementation of the standard of care. This is one of the reasons why the therapeutic effect obtained in randomized controlled trials is seldom comparable to therapeutic effect observed in clinical practice, in which the standard of care is less consistently enforced.

Accepting the validity of the Hawthorne effect, detecting and characterizing its features in different experimental situations is important, especially for behavioral interventions. In clinical research, the Hawthorne effect might at times introduce confounding factors, whereas in clinical practice it may result in beneficial behavioral changes, otherwise difficult to obtain. This effect is operational in longitudinal studies, especially with behavioral interventions, and should be distinguished from the natural history, placebo, and treatment effects [30]. In our study, no “classical” placebo effect was present, as no study drug was administered per protocol. Furthermore, the general direction of the changes was congruent among the different parameters, consistently indicating clinical improvements. Regression to the mean, that is the well-known phenomenon according to which if a variable is extreme on its first measurement, it will tend to be closer to the average on its second measurement and vice versa cannot be completely excluded but it is unlikely, given that our measurement values generally were not extreme.

In addition to “research participation effect” or the Hawthorne effect, our experience might also be conceptualized based on another principle. In quantum physics, the “Observer Effect”, also known as the “Uncertainty Principle”, named after the German scientist Werner Heisenberg who first described it in the 1920's, states that measurements of certain physical systems cannot be made without affecting the very same systems they are meant to describe. In that sense the phenomenon that we have described and characterized is akin to the Heisenberg principle, and carries implications that may revolutionize clinical research as a discipline. We therefore caution that the “gold standard” study design which includes a ‘run-in’ period, previously regarded as neutral, may in fact markedly alter baseline measurements and therefore have important effects on the predetermined sample size and study outcomes, especially for behavioral studies. In spite of the common presence of an initial run-in period, especially in drug trials, there is little empirical evidence establishing its relevance [31].

Independent of the specific nature of the mechanism(s) at play, clinically meaningful improvements were achieved by simply being screened for enrollment into our Sleep Extension Study in sleep and glucose parameters in a chronically short-sleeping obese adult population. Remarkably, almost half of the subjects that had metabolic syndrome at screening were temporarily “cured” by the time they were randomized about three months later. A legitimate question therefore becomes how to reproduce these effects in clinical practice and, possibly, to maintain them over time. Even in pharmacological trials, where an active drug is administered to the intervention group, the importance of the ritual process and of the psychosocial context is being increasingly recognized [32]. The desire of harnessing this effect should, however, be balanced with the accepted right of properly consenting subjects; the inherent tension between these two factors poses challenges that deserve wide discussions by the scientific community. In prospective studies, whether of observational (i.e. cohort studies) or of interventional nature, the designation of the study “baseline” has important implications, as all subsequent modifications are compared to it. Conventionally, the baseline is considered the time of enrollment in case of a prospective cohort study, or the time of randomization. Little attention is usually paid to the characterization of study candidates at the time of the first encounter, other than for reasons of determining eligibility. Even less attention is subsequently devoted to the modifications that take place in the characteristics of the study population between the first encounter and the “official” baseline; in spite of having collected the data, seldom changes between screening and the official start in the study for those candidates that are eventually enrolled are analyzed and shown. This widespread and accepted practice introduces a number of biases that are currently not fully appreciated by investigators, peer reviewers, editors of academic journals, and regulatory agencies.

An inherent limitation of our study was that the nature of the study design did not allow for dissecting out of how much of the observed changes were due to an intentional effect, as compared to something non-intentional that we characterized in the context of the Hawthorne effect. Most likely, such a desirable study design does not exist for the reasons that we have enumerated earlier (i.e., akin to the Heisenberg Principle in Physics). No matter what term we elect to describe the “non-specific” changes observed in our study and in other studies, these perturbations to the investigational systems do occur and ignoring them, as it often has been the case so far, as compared to acknowledging and trying to understand them obviously is unacceptable. Additional limitation include that the sample size was relatively small and that the Intervention Group was slightly more obese than the Comparison Group.

In summary, discovering between screening and randomization “unintended” changes primarily in sleep and glucose homeostasis without specific intervention in an adult obese population has potential implications that may trespass the design of behavioral studies in the field of sleep to include any type of life-style behavioral modification Moreover, our findings have far-reaching implications in clinical research for the design and interpretation of randomized controlled trials in general, and, possibly, in clinical practice for the non-pharmacological management of glucose abnormalities in obese subjects.

Supporting Information

Acknowledgments

We would like to thank Dr. Smita Abraham Baid, NICHD, NIH, for her useful comments. We would like to thank in alphabetical order the following colleagues for their scientific advice and critical suggestions in the development and conduct of the study protocol: Karim Calis, Emmanuel Mignot, Terry Phillips, Nancy Sebring, Duncan Wallace and Elizabeth Wright. We would also like to thank present and past members of the study team: Peter Bailey, Laide Bello, Meredith Coyle, Paula Marincola, Megan M. Mattingly, Patrick Michaels, Svetlana Primma, Angela Ramer, Rebecca Romero, Megan Sabo, Tanner Slayden, Sara Torvik, Elizabeth Widen, Lydia Williams and Sam Zuber. The bioinformatics support of Frank Pierce (Esprit Health) is gratefully acknowledged. Finally, we are grateful to all of our enthusiastic study individuals.