Figures
Abstract
Background
Young people tend to over-estimate peer group drinking levels. Personalised normative feedback (PNF) aims to correct this misperception by providing information about personal drinking levels and patterns compared with norms in similar aged peer groups. PNF is intended to raise motivation for behaviour change and has been highlighted for alcohol misuse prevention by the British Government Behavioural Insight Team. The objective of the trial was to assess the effectiveness of PNF with college students for the prevention of alcohol misuse.
Methodology
Solomon three-group randomised controlled trial. 1751 students, from 22 British Universities, allocated to a PNF group, a normal control group, or a delayed measurement control group to allow assessment of any measurement effects. PNF was provided by email. Participants completed online questionnaires at baseline, 6- and 12-months (only 12-months for the delayed measurement controls). Drinking behaviour measures were (i) alcohol disorders; (ii) frequency; (iii) typical quantity, (iv) weekly consumption; (v) alcohol-related problems; (vi) perceived drinking norms; and (vii) positive alcohol expectancies. Analyses focused on high-risk drinkers, as well as all students, because of research evidence for the prevention paradox in student drinkers.
Principal Findings
Follow-up rates were low, with only 50% and 40% responding at 6- and 12-months, respectively, though comparable to similar European studies. We found no evidence for any systematic attrition bias. Overall, statistical analyses with the high risk sub-sample, and for all students, showed no significant effects of the intervention, at either time-point, in a completed case analysis and a multiple imputation analysis.
Citation: Moreira MT, Oskrochi R, Foxcroft DR (2012) Personalised Normative Feedback for Preventing Alcohol Misuse in University Students: Solomon Three-Group Randomised Controlled Trial. PLoS ONE 7(9): e44120. https://doi.org/10.1371/journal.pone.0044120
Editor: James Coyne, University of Pennsylvania, United States of America
Received: January 28, 2012; Accepted: July 30, 2012; Published: September 12, 2012
Copyright: © Moreira et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: MTM was supported by a fellowship from the Portuguese Foundation for Science and Technology. Other parts of this study were supported by Alcohol Research UK and the European Foundation for Alcohol Research. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. No additional external funding was received for this study.
Competing interests: MTM and RO declare no competing interests. DRF declares that his Department has received funding from the alcohol industry for prevention projects, and that he is a Trustee of the alcohol-industry funded Drinkaware Trust. This does not alter the authors' adherence to all the PLoS ONE policies on sharing data and materials.
Introduction
During university or college years, students can escalate their alcohol use to dangerous levels [1], and student alcohol consumption levels are typically higher than their non-university peers [2]–[6]. Heavy alcohol consumption amongst students impacts on individuals, educational institutions, and society [7], [8]: excessive drinking behaviour amongst students is linked with a range of alcohol related problems, including injuries, unprotected sex, violence, car accidents, and health problems [9], resulting in a significant economic burden for health systems [10].
Social normative feedback is a prevention intervention based on the fact that young people tend to overestimate alcohol consumption amongst their peer group resulting in motivation, or “peer pressure” to drink more to catch up with their peers and be normal. Social norms theory [11] suggests that correcting this misperception will lead young people to attenuate their drinking behaviour. Brief personalised normative feedback (PNF) interventions focus on an individual's own drinking behaviour, providing factual details about personal drinking levels and patterns in comparison with norms for drinking in similar aged peer groups, alongside more general information about alcohol risk and harms [12]–[15]. PNF is intended to raise motivation for behaviour change [16], [17] and has been highlighted as a promising intervention by the British Government Behavioural Insight Team [18], [19]. A recent Cochrane review [20] found some limited evidence for the effectiveness of computer-based PNF in University students though questions remain about the generalizability of these findings to other countries, including the U.K. [21], [22]. There is also a question about whether PNF is effective over and above the simple alcohol screening/assessment test that itself raises awareness about alcohol consumption [23].
The main objective of this study was to examine the effectiveness of computer-based PNF, compared with (i) screening/assessment only, for reducing alcohol-related problems in first and second year UK university undergraduate students. A further consideration is whether PNF would be more effective when used with all students, rather than just those identified as being at higher risk, because of the prevention paradox [24]. The Harvard College Alcohol study of more than 49,000 U.S. students [25] found that most alcohol-related harms arise from those who are not higher risk drinkers, and this finding is supported by evidence from other countries [26]. Therefore, a second objective was to assess the effectiveness of whole population PNF versus PNF for high-risk drinkers only.
Methods
The protocol for this trial and supporting CONSORT checklist are available as supporting information; see Checklist S1 and Protocol S1.
Design
The study utilised a Solomon three group randomised controlled trial design [27], with one intervention and two control groups to control separately for intervention and for measurement effects. A more conventional Solomon four group design was not possible given that the intervention required baseline information in order to personalise feedback to each participant. The International Standard Randomised Controlled Trial Number for this study is: ISRCTN30784467. The intervention group received brief personalised normative feedback (PNF) within a few weeks of completing the baseline assessment, which comprised an online questionnaire with demographic questions and an assessment of drinking behaviour. The main control group undertook a baseline alcohol assessment, and the delayed control group only provided demographic details at baseline. The intervention and main control group were followed up at 6- and 12-months, and the delayed control group at 12-months only.
Participants
Eligible participants were undergraduate university students (first and second year students; academic year 2008/9) from U.K. universities, recruited through university information systems (posters, email messages, bulletin boards) and through online social networking sites. Participants were incentivised to respond through entry into a prize draw, with prizes including games consoles, mp3 players and store vouchers.
Ethics
Ethics approval for this study was provided by Oxford Brookes University Research Ethics Committee (UREC No. 2006/28). Participants gave their informed consent to participate by completing the web-based questionnaire.
Intervention
The brief personalised normative feedback (PNF) emailed to each participant in the intervention group one to two weeks post-baseline comprised the results of their drinking behaviour assessment compared with, in an easy to read graphical format, average levels of drinking amongst their student peer group. The feedback also provided general information about alcohol and how it might affect them at their current drinking levels, including how long it could take to return to a zero blood alcohol level after a typical drinking occasion. Information was also provided on the money that they might be spending annually on alcohol and also the calories they might be consuming at their current drinking levels. Details of recommended sensible drinking levels were also provided. The composition of the PNF was informed by online normative feedback provided by the Centre for Addiction and Mental Health (CAMH), Toronto [28], and the Brief Alcohol Screening and Intervention for College Students (BASICS) [29] programme. A specimen feedback is available to download [30].
Objectives
The main objective of the trial was to assess the effectiveness of PNF with risky drinking college students for the prevention of alcohol misuse. A second objective was to assess the relative effectiveness of whole population (universal) versus screening and brief intervention (SBI; targeted) normative feedback in reducing alcohol related problems.
Outcomes
A copy of the online questionnaire is available for download [30]. Baseline demographic questions asked about university year, age, gender, living arrangements and weight.
Respondents completed the Alcohol Use Disorders Identification Test (AUDIT) which is a 10-item scale with good validity that is designed to assess hazardous and harmful drinking [31]. The AUDIT score was specified as the primary outcome variable. In another study we have shown that a small change in AUDIT score may have an important impact on population levels of alcohol disorders [32]. The AUDIT scale is scored between 0 and 40, with a higher score indicating higher levels of drinking.
Frequency of alcohol consumption was assessed with one question asking how often the respondent drank, with responses ‘Never’, ‘1–2 per year’, ‘Once a Month’, ‘Twice a Month’, ‘Once a Week’, ‘Twice a Week’, or ‘Daily’. Responses were calculated as number of drinking days per month; for example ‘Never’ scored 0, ‘Once a Week’ scored 4.5, and ‘Daily’ scored 30.
Usual quantity of alcohol consumption was assessed with one question asking how many drinks/units a respondent usually consumed on a drinking occasion, with responses 0, 1, 2, 3, 4, 5, 6, or 7 or more.
Respondents also completed a drinking diary about a “typical week”, where they indicated the number of drinks/units they usually drink on each day of the week, and this was used to calculate the number of units consumed each week. Respondents were then categorised according to whether they were drinking more than 14 units a week for women and more than 21 units a week for men [33]. In the analysis, respondents were scored 1 if they were over these levels, otherwise 0.
Young people who engage in one problem behaviour (e.g. alcohol misuse) are also more likely to engage in other problem behaviours [34]. Therefore, other problems were measured in a newly-developed self-reported scale with nine possible problems, listed on a yes/no scale: 1. Blackout or memory lapse; 2. Been embarrassed by your actions; 3. Been in a fight; 4. Engaged in unprotected sex; 5. Missed a lecture/class; 6. Required emergency medical treatment; 7. Sustained an injury; 8. Trouble with local or campus authorities; 9. Received unwanted sexual advances. Responses were summed to provide an alcohol-related problems score (range 10 to 20 with a higher score indicating more problems). The internal consistency for this scale was alpha = 0.71.
Perceived norms were measured using an adaptation of the two versions of the Drinking Norms Rating Form [35]. We calculated normative misperception from two variables: the number of alcoholic drinks (“yourself” and “students in your year”) respondents felt were on average consumed at parties or social events. The difference between “yourself” and “students in your year” ranged between −7 and +5, with a negative score indicating that respondents thought that other students were drinking more.
Positive alcohol expectancies were measured using an abbreviated form of the Alcohol Expectancies Questionnaire for Adolescents (AEQ-A) [36]. This self-report questionnaire assesses positive expectancies related to alcohol consumption (social changes, cognitive improvement, sexual enhancement and relaxation) with 21 statements and a true/false answer format. Scores ranged between 0 and 21, with a higher score indicating more positive expectancies.
Several measures were included in the questionnaire to be used as covariates in analyses, including demographic factors (age, gender) as well as baseline consumption. Social desirability responsiveness was also assessed using the short form of the Marlowe Crown scale 2 [37].
Sample Size
We needed an achieved sample size of 900 hazardous drinkers (150 per gender per group), based on effect size estimates of a mean difference in AUDIT score of 1.9 (s.d. = 6) for females [38], and power = .9 and α = .05 (2-tailed tests). Taking account of known prevalence rates for hazardous drinkers in this population, and with cautious estimates for participation and attrition rates, we aimed to recruit 4000 students.
Randomization – sequence generation
Participants who provided informed consent were individually randomised to intervention or control groups via concealed centrally-allocated computer generated random numbers, with a 1∶1∶1 allocation ratio, without stratification or blocking on any variables.
Randomization – allocation concealment
The computer-based randomisation ensured that researchers and participants were not aware of allocated group.
Randomization – implementation
Participants were recruited via a web site that provided trial information and enabled informed consent. Once consent was given, participants were randomized by computer. There was no direct human involvement in the randomization process.
Blinding
At randomisation participants were not aware of the nature of the intervention, but full participant blinding was not possible. Similarly, the postgraduate researcher who emailed the standardised PNF to the intervention group participants was not blind to group allocation. All questionnaires were completed remotely via web-based forms, and so outcome assessment was blinded.
Statistical Methods
Data from the remotely completed online questionnaires were automatically entered and stored in a web based server. At the end of the trial a computer science specialist de-encrypted the data and all personal identifying details were removed from the dataset.
All analyses were intention-to-treat (ITT) and all participants were followed up regardless of their compliance with the intervention. Analyses were performed on the full sample (all students) and also with a higher risk sub-sample to test the prevention paradox prediction. Higher risk drinkers were those who, at baseline, scored 8 or more on the AUDIT scale (the cut-off for hazardous drinking) and also drank more than the recommended weekly consumption limits.
The collected scores about the “Frequency of drinking” and the “Quantity of drinking” were analysed with a mixing distribution of the Poisson regression with a Gamma mixture for panel data using the Stata 10.1 (StataCorp, College Station, Texas). For the proportions of students exceeding 21 units a week (men) or 14 units a week (women), we used generalized linear mixed models with the xtlogit procedure. All other scores (Audit, Alcohol-related Problems, Drinking Norms and Positive Expectancies) were analyzed with linear/log-linear regression using xtmixed (StataCorp). All models included a random intercept to account for clustering within participant (same university) as well as fixed effects for group, follow-up assessment and their interaction. The interactions were removed from the analysis due to insignificant effects. The results are presented as relative risks, odds ratios and difference in regression coefficients, respectively.
We fitted two different random effect models to assess an intention-to-treat hypothesis. In the first random effects model we treated all missing values as missing at random (MAR). Random effects models [29], [30] allow the incorporation of all participants with at least 1 follow-up observation. This model will yield unbiased estimates of the treatment effect under the assumption that values are missing at random. In the second random effects model we carried out a sensitivity analysis using multiple imputation. A chained equation imputation model was fittedsimultaneously for all outcomes as well as baseline AUDIT score, age, and sex, to create imputed complete data sets. Values were imputed for all participants, even for those with no post-baseline data. The following assumptions [39] guided the multiple imputation: 1. all randomised participants were followed-up even if they did not participate in the intervention (i.e. they withdrew from the allocated treatment); 2. We performed a main analysis of all observed data that are valid under a plausible assumption about the missing data; and 3. We performed sensitivity analyses to explore the effect of departures from the assumption made in the main analysis.
Results
Participant flow
The sample at baseline (N = 2611) consisted of first and second year students enrolled in UK universities. Recruitment, follow-up and attrition are described in Figure 1. Follow-up rates were low, with only 50% responding at 6-months and only 40% responding at 12-months. We were not able to collect information about reasons for non-completion of follow-up questionnaires.
Recruitment
Students were recruited into the study at the beginning of the academic year 2007/8, and followed up for 6- and 12-months.
Baseline data
Sample characteristics are described in Table 1. Table 2 describes participants lost to follow-up, with very similar rates in both intervention and the main control groups, suggesting that there was no differential or systematic bias in attrition.
Numbers analysed
Intention to treat was applied in all analyses. In the completed case analyses (i.e. follow-up respondents only) for high risk drinkers 104/116 universities and 550/1187 individuals were followed up and analysed at 6-months, and 97/116 universities and 444/1187 individuals at 12-months. In the full population analyses 111/122 universities and 876/1751 individuals were followed up and analysed at 6-months, and 107/122 universities and 718/1751 individuals at 12-months.
Outcomes and estimation
We have not included the delayed control group in attrition assessment or statistical analysis of effects because there were no statistical differences between the main control and the delayed control groups at 12-month follow-up for any drinking behaviour measures (frequency of drinking, χ2 = 6.29, df = 6, p = 0.39; usual quantity of alcohol, t = 0.075, df = 699, p = 0.94; AUDIT, t = 0.63, df = 699, p = 0.53; alcohol-related problems, t = −0.181, df = 699, p = 0.86; perceived drinking norms, t = −0.609, df = 699, p = 0.54). This analysis indicates that there was no effect on drinking behaviour of the baseline measures and questions about alcohol use and problems.
Six- and twelve-month follow-up analysis in the high risk sub-sample showed no effects of the intervention, at either time-point (Tables 3 and 4) and in both the completed case analysis and the multiple imputation analysis, for most outcomes. Only one outcome measure, weekly drinking, showed a significant effect at 6-months with an odds ratio (OR) of 0.417 (95% CI 0.223, 0.781), but this had disappeared by 12-months (OR = 0.710, 95% CI 0.435, 1.160). The 12-month significant effect remained in the multiple imputation analysis, but this should be regarded with caution given the very high attrition rates. Moreover, this one effect should not be over-interpreted given the consistent pattern of no effect across most outcome measures and time points.
In the full sample analysis there was a similar pattern of no effect of the intervention (Tables 5 and 6) at either time-point and in both the completed case analysis and the multiple imputation analysis. Again, only weekly drinking had a significant effect at six-months (OR = 0.440, 95% CI 0.245, 0.788), but not at 12-months (OR = 0.770, 95% CI 0.495, 1.197). Overall, there is a consistent pattern of no effect across most outcome measures and time points.
Discussion
We have found that a UK web-based personalised normative feedback (PNF) intervention did not motivate students to reduce their alcohol intake: students who received the brief personalised PNF intervention did not have lower AUDIT scores, drink less alcohol or have fewer alcohol related problems than control group students at 6- or 12-month follow-ups. Our results fail to replicate the findings of the New Zealand, Australian and U.S. trials of brief, web-based, social normative feedback [40]–[42]. But our results are generally in agreement with other work from the UK and Sweden, where randomised trials aimed at assessing the effectiveness of an electronic web-based personalised feedback intervention have found little or no effect for most outcome measures [43]–[46].
Interpretation
The innovative design and sampling aspects of this study: the Solomon three-group design to test for measurement effects, and the whole sample and higher risk sub-sample analyses to test the prevention paradox prediction, were illuminating. We found no evidence that simply asking people about their drinking behaviour, in the absence of any intervention, led to any change in behaviour. Similarly, we found no evidence that a population wide intervention was more effective than a targeted intervention with a higher risk sub-sample of students but, given that we found no effects of the PNF intervention across a range of alcohol outcomes, this is not surprising.
Generalizability
The reason(s) for the inconsistent findings between our study and some other studies is not clear. One line of thought is that the accumulating evidence on social normative feedback is an example of initially positive research findings ultimately being found to be false [47]. Another possibility is that the UK offers a more challenging context for effectiveness because of local contextual factors that are not present in other countries, for example it may be that the prevalence of excessive drinking behaviours in the UK, and the cultural acceptance of heavier drinking, provides a more hostile environment for success of normative feedback interventions. Alternatively, the intervention we used, based on studies from other countries, may not have been sufficiently developed for the UK population. In this study personalised normative feedback was presented to university students via email with information about the norms for the “average” student in their university, as in previous research [48] where similar proximal referent norms were used. Research has demonstrated that when compared with more distal referents, proximal referents are more effective for preventing student alcohol misuse and related problems [35], [49], [50]. On the other hand, a study in Canada [51] found that the social norm materials that were designed to be the least comparable had the greatest impact, and that Canadian students were more familiar with American drinking norms because of media influence, with television being the most significant medium for the spread of youth culture.
We should also point out that, like many other studies that have reported the effects of brief personalised normative feedback, the intervention contained more than just the normative feedback. We also provided financial feedback, health information and advice on where help can be obtained. The interaction between these different components, and how they are presented to participants, may be important in determining effectiveness. Clues as to the differences in effect across different studies might lie in the differences in intervention content and delivery.
Attrition in this study was lower than in other European research on email- or web-based social normative feedback interventions with university students [44]–[46] though higher than in some Australian or U.S. studies [40], [42]. There is a potential risk to internal validity from the low follow-up rates achieved in our study, although students unavailable for follow-up were similar across groups with regard to sex, age and baseline drinking status, and the multiple imputation sensitivity analysis did not produce any marked or systematic changes in treatment effect sizes or significance. But, due to the low follow-up rates, we cannot absolutely rule out the possibility that differential attrition in relation to unmeasured and uncontrolled confounders could have affected the results.
One possible reason for the low retention rates in our study is that the personalised feedback given in our study was delayed by a few weeks, as we needed to collect and analyse all baseline data to provide a normative comparison dataset for personalised feedback to each participant. It is conceivable that this could have influenced retention rates, but we regard this as unlikely given more immediate feedback was provided in similar studies that had similar or poorer retention rates [44]–[46] and similar findings of limited/no effectiveness.
A more serious issue is in relation to external validity, specifically generalizability [22]. It is hard to see how the sub-set of students that were retained in the study are generally representative of the whole student population, so inferences from sample to the pre-specified study population are problematic. In fact, the low-follow-up rates add to the problem with understanding how generalizable these study results are given the recruitment methods used: only those students who saw the study adverts (via email, student information systems or Facebook) were able to participate and, as the most effective recruitment strategy was Facebook with 78% of all participants, it is unclear how representative these participants are of the general student body. So, one possible explanation for our null results in this study is that we obtained results on a different group of students than other studies that have found significant effects.
On the other hand, this study was a large pragmatic randomised trial with design, sampling, recruitment and follow-up characteristics that are similar to other large European trials [44]–[46]. All these other European trials have had low follow-up rates from those randomised and assessed at baseline, and with similar non-significant effects to our study. An important conclusion to draw out from our study, alongside these other European studies, is that the acceptability and viability of recruiting students into a brief personalised feedback intervention, outside of a university medical centre screening programme [40] or a mandated student alcohol education programme [42], seems to be low.
Overall evidence
In conclusion, our results show no evidence for the effectiveness of personalised normative feedback for the prevention of alcohol misuse and alcohol-related problems in a UK student population. The applicability of personalised normative feedback to populations and settings that are different from those in the trials whether positive effects have been found is uncertain [22]; moreover an overall conclusion of no effect for this sort of intervention [47] cannot be discounted.
Acknowledgments
Grateful thanks to Dr Gil Lopes for developing the web-based technology for this study.
Author Contributions
Conceived and designed the experiments: MTM DRF. Performed the experiments: MTM. Analyzed the data: RO DRF MTM. Wrote the paper: DRF MTM RO.
References
- 1. Schulenberg JE, Maggs JL (2002) A developmental perspective on alcohol use and heavy drinking during adolescence and the transition to young adulthood. Journal of Studies on Alcohol 54–70.
- 2. Kypri K, Cronin M, Wright CS (2005) Do University students drink more hazardously than their non-student peers? Addiction 100: 713–714.
- 3. Kypri K, Langley J, McGee R, Saunders J, Williams S (2002) High Prevalence, persistent hazardous drinking among New Zealand tertiary students. Alcohol Alcohol 37: 457–464.
- 4. Pickard M, Bates L, Dorian M, Greig H, Saint D (2000) Alcohol and drug use in second-year medical students at the University of Leeds. Med Educ 34: 148–150.
- 5. Underwood B, Fox K (2000) A survey of alcohol and drug use among UK based dental undergraduates. Br Dent J 18: 314–317.
- 6. Webb E, Ashton C, Kelly P, Kamali F (1998) An update on British medical students' lifestyles. Med Educ 32: 325–331.
- 7. Ham LS, Hope DA (2003) College students and problematic drinking: a review of the literature. Clin Psychol Rev 23: 719–759.
- 8. Wechsler H, Davenport A, Dowdall G, Moeykens B, Castillo S (1994) Health and behavioural consequences of binge drinking in college: a national survey of students at 140 campuses. JAMA 272: 1672–1677.
- 9. Wechsler H, Davenport A, Dowdall G, Moeykens B, Castillo S (1994) Health and behavioural consequences of binge drinking in college: a national survey of students at 140 campuses. JAMA 272: 1672–1677.
- 10.
Anderson P, Baumberg B (2006) Alcohol in Europe. London: Institute of Alcohol Studies.
- 11. Perkins HW (2002) Social Norms and the Prevention of Alcohol Misuses in Collegiate Contexts. Journal of Studies on Alcohol 14: 164–172.
- 12. Larimer ME, Cronce JM, Lee CM, Kilner JR (2004/2005) Brief interventions in college settings. Alcohol Research and Health 28: 94–104.
- 13. Saunders JB, Kypri K, Walters ST, Laforge RG, Larimer MT (2004) Approaches to brief intervention for hazardous drinking in young people. Alcoholism: Clinical & Experimental Research 28: 322–329.
- 14. Walters ST, Neighbors C (2005) Feedback interventions for college alcohol misuse: What, why and for whom? Addictive Behaviors 30: 1168–1182.
- 15. White HR (2006) Reduction of alcohol-related harm on United States college campuses: The use of personal feedback interventions. International Journal of Drug Policy 17: 310–319.
- 16. Agostinelli G, Brown JM, Miller WR (1995) Effects of normative feedback on consumption among heavy drinking college students. Journal Drug Education 25: 31–40.
- 17. Sobell LC, Cunningham JA, Sobell MB, Agrawal S, Gavin DR, et al. (1996) Fostering self-change among problem drinkers: A proactive community intervention. Addictive Behaviors 21: 817–833.
- 18.
Cabinet Office Behavioural Insights Team (2011) Behavioural Insights Team Annual update 2010–11 London, UK: Cabinet Office. Available: http://www.cabinetoffice.gov.uk/sites/default/files/resources/Behaviour-Change-Insight-Team-Annual-Update_acc.pdf. Accessed 23 Sep 2011.
- 19. Marteau TM, Ogilvie D, Roland M, Suhrcke M, Kelly MP (2011) Judging nudging: can nudging improve population health? BMJ 342: 263–265.
- 20.
Moreira T, Smith L, Foxcroft D (2009) Social norms interventions to reduce alcohol misuse in University or College students Cochrane Database of Systematic Reviews, 3.
- 21.
Foxcroft DR, Tsertsvadze A (2011) Universal multi-component prevention programs for alcohol misuse in young people. Cochrane Database of Systematic Reviews: Art. No.: CD009307. DOI: 009310.001002/14651858.CD14009307.
- 22. Fernandez-Hermida JR, Calafat A, Becoña E, Tsertsvadze A, Foxcroft DR (2012) Assessment of generalizability, applicability and predictability (GAP) for evaluating external validity in studies of universal family-based prevention of alcohol misuse in young people: systematic methodological review of randomized controlled trials. Addiction doi: 10.1111/j.1360-0443.2012.03867.x [Epub ahead of print].
- 23. McCambridge J, Kypri K (2011) Can simply answering research questions change behaviour? Systematic review and meta analyses of brief alcohol intervention trials. PLoS ONE 6: e23748.
- 24. Rose G (1981) Sick individuals and sick populations. BMJ 282: 1847–1851.
- 25. Weitzman ER, Nelson TF (2004) College student binge drinking and the “prevention paradox”: Implications for prevention and harm reduction. Journal of Drug Education 34: 247–266.
- 26. Danielsson A-K, Wennberg P, Hibell B, Romelsjö A (2011) Alcohol use, heavy episodic drinking and subsequent problems among adolescents in 23 European countries: does the prevention paradox apply? Addiction (online early) DOI: 10.1111/j.1360-0443.2011.03537.x.
- 27.
Campbell D, Stanley J (1963) Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin.
- 28.
Centre for Addiction and Mental Health (2008) Evaluate Your Drinking. Toronto: Centre for Addiction and Mental Health. Available: http://www.camh.net/About_Addiction_Mental_Health/Drug_and_Addiction_Information/evaluate_your_drinking.html. Accessed 22 Sep 2011.
- 29.
Dimeff LA, Baer JS, Kivlahan DR, Marlatt GA (1999) Brief Alcohol Screening and Intervention for College Students Basics: A Harm Reduction Approach. New York: The Guilford Press.
- 30.
Moreira MT, Foxcroft DR (2009) SNF UK trial materials. Available: https://docs.google.com/folder/d/0B6qFfRezLfa4SlYyWVB6MHZwTEU/edit.
- 31.
Babor TF, Biddle-Higgins JC, Saunders JB, Monteiro MG (2001) AUDIT: The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Health Care. Geneva, Switzerland: World Health Organization.
- 32. Foxcroft DR, Kypri K, Simonite V (2009) Bayes' Theorem to estimate population prevalence from Alcohol Use Disorders Identification Test (AUDIT) scores. Addiction 104: 1132–1137 doi: 1110.1111/j.1360-0443.2009.02574.x.
- 33.
NHS Information Centre Lifestyles Statistics (2009) Statistics on Alcohol: England, 2009. UK: The Health and Social Care Information Centre.
- 34.
Jessor R, Jessor SL, editors(1977) Problem Behavior and Psychosocial Development: A Longitudinal Study of Youth. New York: Academic Press.
- 35. Baer JS, Stacy A, Larimer M (1991) Biases in the Perception of Drinking Norms among College Students. J Stud Alcohol 52: 580–586.
- 36. Webb JA, Baer PE, Caid CD, McKelvey RS, Converse RE (1992) Development of an Abbreviated Form of the Alcohol Expectancy Questionnaire for Adolescents. Journal of Early Adolescence 12: 441–456.
- 37. Strahan RF (2007) Regarding some short forms of the Marlowe-Crowne Social Desirability Scale. Psychol Rep 100: 483–488.
- 38. Johnsson KO, Berglund M (2006) Comparison between a cognitive behavioural alcohol program and post-mailed minimal intervention in high-risk drinking university freshmen: results from a randomised controlled trial. Alcohol and Alcoholism 41: 174–180.
- 39. Carpenter J, Pocock SJ (2011) Strategy for intention to treat analysis in randomised trials with missing outcome data. BMJ 342: d40.
- 40. Kypri K, Hallett J, Howat P, McManus A, Maycock B, et al. (2009) Randomized controlled trial of proactive web-based alcohol screening and brief intervention for university students. Archives of Internal Medicine 169: 1508–1514.
- 41. Kypri K, Saunders J, Williams S, McGee R, Langley J, et al. (2004) Web-based screening and brief intervention for hazardous drinking: a double-blind randomized controlled trial. Addiction 99: 1410–1417.
- 42. Lovecchio CP, Wyatt T, DeJong W (2010) Reductions in drinking and alcohol-related harms reported by first-year college students taking an online alcohol education course: A randomized trial. Journal of Health Communication 15: 805–819.
- 43. Bewick BM, Trusler K, Mulhern B, Barkham M, Hill AJ (2008) The feasibility and effectiveness of a web-based personalised feedback and social norms alcohol intervention in UK university students: A randomised control trial. Addictive Behaviors 33: 1192–1198.
- 44. Bewick BM, West R, Gill J, O'May F, Mulhern B, et al. (2010) Providing web-based feedback and social norms information to reduce student alcohol intake: a multisite investigation. Journal of Medical Internet Research 12: e59.
- 45. Ekman DS, Andersson A, Nilsen P, Stahlbrandt H, Johansson AL, et al. (2011) Electronic screening and brief intervention for risky drinking in Swedish university students: randomized controlled trial. Addictive Behaviors 36: 654–659.
- 46.
McCambridge J, Bendtsen P, Bendtsen M, Karlsson N, Nilsen P (2011) RCT of the effectiveness of electronic mail based alcohol intervention with university students: dismantling the assessment and feedback components. Paper presented to the Society for the Study of Addiction Annual Meeting, York 2011. Available: http://wwwaddiction-ssaorg/2011/McCambridgeJ%20Presentationppt. Accessed 24 May 2012. Archived by WebCite® at http://wwwwebcitationorg/67tcZheSn).
- 47. Ioannidis JPA (2005) Why most published research findings are false. PLoS Medicine 2: 696–701.
- 48. Perkins HW, Meilman PW, Leichliter JS, Cashin JR, Presley CA (1999) Misperceptions of the Norms for the Frequency of Alcohol and Other Drug Use on College Campuses. American Journal of College Health 47: 253–258.
- 49. Lewis MA, Neighbors C (2004) Gender-specific misperceptions of college student drinking norms. Psychology of Addictive Behaviors 18: 334–339.
- 50. Borsari B, Carey KB (2003) Descriptive and injunctive norms in college drinking: A meta-analytic integration. Journal of Studies on Alcohol 64: 331–341.
- 51. McShane KE, Cunningham JA (2003) The role of relevancy in normative feedback for unversity students' drinking patterns. Addictive Behaviors 28: 1523–1528.