UK evidence

A recent UK study of risk factors for under-reporting of alcohol consumption was not able to explore non-response bias in detail but recognised there could be value in using the continuum of resistance model in a UK sample in future research [6]. There is limited evidence from the UK but that which exists suggests that alcohol consumption estimates may be biased due to non-response. A 1987 study by Crawford interviewed adults (18+) from three areas in Great Britain about alcohol consumption in their own homes (n = 2,349) [7]. Among men, those who were difficult to contact (requiring 4+ calls) consumed 20.8 drinks a week compared with 16.8 drinks among those who required 1–3 calls (P = 0.02) [7]. In 2002, Lynn and Clarke found that the prevalence of heavy drinking was higher among participants in households that required extended efforts to recruit them (either difficult to contact or who temporarily refused) compared with households that were interviewed after standard efforts [8].

A 2013 study protocol by Gray and colleagues describes how non-response bias can be explored using record linkage [9]. This suggested the downward trend in alcohol consumption in recent years is partially attributable to falling response rates (a trend seen in many health surveys), with fewer heavy drinkers responding over time [9]. Surveys like the Scottish Health Survey (SHeS) use response probability weighting to make them nationally representative, but these weights are based on limited socio-demographic information. The full study published in 2014 found participants in the SHeS experienced lower rates of alcohol-related harm than the general population in Scotland [10]. SHeS participants experienced about two-thirds of the rate of alcohol-related harm compared with the general population (RR 0.69, 95% CI: 0.61–0.76), as well as a lower risk of all-cause mortality (RR 0.89, 95% CI: 0.83–0.96), which was more pronounced in more deprived areas [10]. If the occurrence of alcohol-related harm is higher in the general population than among survey participants, this suggests non-participants may drink more alcohol than survey participants.

International evidence

Dawson and colleagues [11] recently noted that a number of longitudinal studies have found no difference between those who continue to participate and those who are non-participants at later stages or waves [12, 13], or between early-responding participants and late-responding participants [14, 15]. However numerous other studies have identified important differences between participants and non-participants at later stages [11, 16, 17], or between early responders and late responders or those who required more extensive efforts to recruit [4, 18–21]. There is also evidence from Finland that non-participants are more likely to die from alcohol-related diseases, injuries and poisonings than survey participants [22]. Previous studies using the continuum of resistance model in New Zealand [4] and Canada [18] found higher rates of heavy drinking among late responders, however no studies have used this model in the UK.

In this study we examine how survey estimates of alcohol consumption differ by the number of contact attempts, using the continuum of resistance model. We hypothesise that alcohol consumption will be higher among participants who required more contact attempts. Using the number of contact attempts to distinguish between early- and late-responding participants, we are then able to adjust survey estimates of alcohol consumption for non-response bias using the assumptions of the continuum of resistance model.

Data

The Health Survey for England (HSE) is an annual cross-sectional survey, representative of the adult population (aged 16+) living in private households in England [23]. Full details of the HSE methodology are published annually as reports [24–26]. This study uses three consecutive years of survey data: 2011, 2012 and 2013. Response rates and sample sizes are shown in Table 1, with additional detail available in annual published reports [24–26].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Response rate by survey year for HSE 2011–13.

https://doi.org/10.1371/journal.pone.0170892.t001

As in previous years, a computer-assisted personal interview (CAPI) collected information on number of drinking days in the last week and alcohol consumption on the heaviest drinking day in the last week (using a beverage-specific recent recall method), and also on average weekly drinking over the past 12 months (using a beverage-specific quantity frequency method) [27].

In addition to the Health Survey for England dataset that is publicly-available through the UK Data Archive, an additional variable on the number of contact attempts was obtained from the National Centre for Social Research (NatCen) for the purposes of this study. This variable forms part of the HSE paradata set which is held by NatCen.

Ethics statement

Addresses sampled for the HSE are sent an advance letter and leaflet introducing the survey and stating that an interviewer will be calling to seek permission to interview. It is clear in the advance letter and information leaflet that participation in the survey is entirely voluntary, and that participants may decline to answer individual questions, withdraw or stop at any time, or refuse any particular measurement if they wish to do so. Interviewers will often repeat this information in their introductions and when they are setting up appointments, and throughout the interview as necessary. Individual interviews are conducted with adults who give verbal informed consent. The HSE informed consent procedures, information leaflets and questionnaires are scrutinised by a National Health Service (NHS) ethics committee each year. Ethical approval for the HSE survey years 2011–13 was obtained from the Oxfordshire A Research Ethics Committee: 2011 reference 10/H0604/56, 2012 reference 10/H0604/56, 2013 reference 12/SC/0317. This study is a secondary analysis of previously collected data and so additional ethical approval was not required.

Analysis

The number of contact attempts for a productive interview ranged from 1–18; however 90% of interviews were completed within 8 contact attempts. The mean number of calls was 4.4 (SD 2.8). We split the number of contact attempts into 2 categories: 1–6 calls (approximately 80% of the sample) and 7+ (slightly under 20% of the sample). Other previous studies have used two groups of respondents in the same way (e.g. [18]). Descriptive statistics assessed demographic and drinking characteristics by the number of attempts to contact (1–6; 7+), with Chi-squared and t-tests to identify differences that were statistically significant. Associations between number of contact attempts and the odds of drinking more than daily and weekly guidelines (as used by the UK Government until January 2016) were modelled using logistic regression, stratified by sex due to the different drinking patterns between men and women. Estimates of alcohol consumption were then adjusted for non-response bias using the probability approach also used by Zhao and colleagues in a 2009 paper [18] and by Rowland and Forthofer in the 1990s [28]. This follows the continuum of resistance model by assuming that non-participants are similar to the survey participants who were most difficult to contact (i.e. 7+ calls). The equation used states that the true prevalence in the population is the sum of the prevalence among participants and non-participants, weighted by the proportions of participants and non-participants: where the estimated prevalence among non-participants is assumed to be the same as the prevalence among the survey participants who required 7+ calls to elicit a response

All analyses (differences between 1–6 and 7+ contact attempts; and the odds of drinking more than the daily- and weekly-guidelines) were conducted using the HSE interview weight. This weight is computed by NatCen and intends to make the sample representative of the population living in private households in England by adjusting for survey design and demographic predictors of non-response (including sex, household type, geographical area and household social class) [26].

There are two different sources of missing data in cross-sectional surveys such as the HSE. Firstly, individuals living at the sampled addresses who were not surveyed because they did not participate at any stage (unit non-response). We dealt with unit non-response in the present study by using the HSE interview weight described above. The second kind of missing data arises among survey participants, and is due to non-response to certain questions (item non-response). All analyses in the present study used complete-case analysis. That is, in the adjusted regression models, cases were included only if they had non-missing or complete values for all covariates. Missing data in the items of alcohol consumption may have affected the magnitude of the estimates of alcohol consumption shown in the present study–if the item non-response was associated with drinking behaviour. However, item non-response to the items on alcohol consumption was low, and the main focus of the present study was on comparing the estimates of alcohol consumption before and after using the assumptions of the continuum of resistance model. The analyses were conducted in SPSS version 22 (SPSS Inc., Chicago, Illinois, US). Syntax is available on request from the corresponding author.

Key findings

This study found participants in the Health Survey for England who took more attempts to contact have different socio-demographic characteristics and different drinking patterns than participants who took fewer attempts to contact. Most notably, the prevalence of heavy episodic alcohol consumption, and levels of average weekly alcohol consumption were higher for those who took more attempts to contact, although other measures of alcohol consumption showed less variation by the number of contact attempts. The measures of alcohol consumption which increased the most when the estimates were further adjusted according to the assumptions of the continuum of resistance model were measures most associated with acute harms (i.e. heavy episodic drinking) and chronic harms (i.e. total weekly consumption). The decrease in the average number of drinking days per week is consistent with a more hazardous pattern, since it suggests higher volumes are being consumed on each occasion. Our findings lend support to extended efforts to recruit and follow up participants in research studies in order to reduce bias in estimates of drinking.

Strengths

To our knowledge, this is the first study using the assumptions of the continuum of resistance model to assess non-response bias in survey estimates of alcohol consumption in the UK. This also represents a unique use of existing data: the HSE is publicly available through the UK Data Archive, and the paradata (including data on the number of contact attempts) is routinely collected by NatCen, and is available from NatCen on application to their Data Release Panel.

We used three consecutive years of a large, nationally representative survey (the HSE) which is the source of some of England’s national statistics on alcohol drinking. The response rate for the HSE is in line with other similar surveys (for example the SHeS), therefore it may be that similar findings apply in other populations where drinking patterns are similar.

Limitations

It is difficult to compare the findings of this study with other studies. This is because the definition of the number of contact attempts or the amount of time taken to respond to an invitation varies from study to study due to sampling and recruitment methods employed and different measures used. However we would say that our findings are broadly similar to those of Meiklejohn 2012 [4] and Zhao 2009 [18]. Our findings also corroborate those of work conducted in Scotland, which used record linkage to examine rates of alcohol-related harm in participants in the SHeS compared with the general population [9, 10]. In this record linkage study, the difference in mortality rates between survey participants and the general population was used to investigate the validity of survey estimates, whereas in the present study we have estimated the potential impact of non-response on the survey estimates of consumption.

Total number of contact attempts is not an ideal measure of willingness to take part in a survey. In some instances it can be beneficial if the interviewer leaves and returns on another occasion so as to avoid a refusal [8]. Often, a substantial proportion of the total number of calls may take place after initial contact has been established, where the interviewer is trying to find a suitable time to complete the survey. For example in Lynn and Clarke’s 2002 study, fewer than half of households that received 10 or more calls required 10 calls to make an initial contact, the remainder required the extra calls subsequent to the first contact being established [8].

Missing data is a perennial problem for the analysis of survey data. In the present study we used the HSE interview weight, and the assumptions under the continuum of resistance model, to correct for unit non-response: thereby assigning larger weight to the subgroups of the population with lower propensities to respond to social surveys. However, our use of complete-case analysis may mean that there may still be residual bias in our estimates of alcohol consumption due to item non-response; for example, it is possible that the subset of survey participants with missing values for the alcohol questions may have been heavier drinkers than the participants with non-missing alcohol data.

Finally, the use of 7+ contact attempts as a measure of ‘difficult’ to contact is an arbitrary cut point (although this was also used by Zhao and colleagues [18]). Almost 20% of our sample required 7+ calls to elicit a response. We believe this was an appropriate proportion for reasons of obtaining adequate statistical power, and we did identify important differences between the 1–6 and 7+ groups. In a 2014 US study by Messiah and colleagues it was suggested that five attempts to contact was sufficient to reduce selection bias. After five attempts, the return per visit was low, and the prevalence of seven health conditions studied changed only by a very small amount, suggesting additional attempts were of decreasing value [30]. Further research could identify the optimum number of contact attempts, however this was beyond the scope of the present study.

Future research

There are other variables held by NatCen in the HSE paradata which could also be of interest in relation to how drinking is measured in surveys. For example, there is a variable describing the interviewers’ assessments of overall data quality, and other measures of ‘cooperativeness’, such as consent to record linkage and consent to a subsequent nurse visit.

Future research could address the limitation of the complete-case analysis used in our study by examining the impact on the estimates of alcohol consumption by using the technique of multiple imputation to substitute values for a missing items on alcohol consumption, alongside the adjustment made in this study under the assumptions of the continuum of resistance model (utilising the HSE paradata to distinguish between early- and late-responding participants). The continuum of resistance model could also be used to explore other health behaviours such as physical activity, and there is potential for the number of contact attempts to be incorporated as a covariate in the models used to estimate the survey non-response weights (such as the HSE interview weight). It would also be valuable to explore the continuum of resistance model in the context of a different study design, for example in follow ups to a randomised controlled trial in order to see any similarities in the patterns observed with the HSE.