Need for individual participant data meta-analysis

Individual participant data (IPD) meta-analysis involves synthesising raw line-by-line participant data across trials. It is considered the “gold standard” for meta-analysis [32], due to its many benefits. For instance, combining IPD from trials enables a larger pooled sample size and greater flexibility to align outcome definitions, thereby increasing statistical power to detect effects for rarer outcomes (such as the emergence of eating disorders) and in smaller samples compared to individual trials alone [33]. It can also lead to improved quantity and quality of available data for analyses by providing access to unpublished outcomes for all randomised participants and by enabling extensive data checking [34]. Further, by leveraging data from existing, previously funded trials, IPD meta-analysis reduces research waste and enables the rapid generation of new knowledge with immediate clinical implications. Of particular importance, collection of data at the individual level provides the opportunity to conduct more in-depth subgroup analyses than using standard aggregate data meta-analyses [34]. Analyses of participants enrolled in various behavioural interventions across diverse subgroups will also enhance generalisability. Overall, this methodological approach will support the identification of individuals at risk of eating disorders in the context of weight management.

Study aim and objectives

This study aims to understand which participants experience a change in eating disorder risk, or related symptoms, during and after behavioural weight management interventions conducted in adolescents or adults with overweight or obesity. This aim will be addressed by the following objectives: 1) to identify baseline participant risk factors which predict an increase or decrease in eating disorder risk, or related symptoms, during and following a weight management intervention; and 2) to identify whether there are baseline participant risk factors which predict change in eating disorder risk, or related symptoms, if they receive any behavioural weight management intervention compared with no intervention (i.e., no-treatment controls). These two objectives will be addressed in two separate but related studies, outlined in this protocol.

Eligibility criteria

Information sources and search strategy

A systematic search of the electronic databases MEDLINE, Embase, PsycINFO and Scopus and clinical trials registries ClinicalTrials.gov and the World Health Organization’s International Clinical Trials Registry Platform (WHO ICTRP) Search Portal [38] were conducted from inception to March 2022 and May 2022 respectively. The complete search strategies for all databases and clinical trials registries are listed in S1 Table. Collaborators will be asked to notify us of any additional eligible trials. Authors will also be able to notify the study team of an eligible trial through the study website (https://www.editcollaboration.com/). Trials will be able to continue to join the Collaboration on an ongoing basis.

Records identified through the database searches were imported into Covidence software (Veritas Health Innovation, Melbourne, Australia) to remove duplicates and to screen against the inclusion/exclusion criteria. Two EDIT Collaboration study team members independently screened records by title and abstract and then by full text. Reasons for full text exclusion were recorded. Discrepancies in study eligibility were resolved through discussion with a third member of the study team consulted if necessary.

Inviting trialists to join the collaboration

A list of eligible studies identified up to August 2022 can be found in S2 Table. The corresponding author/s of eligible studies have been invited to join the EDIT Collaboration via email. If no response was received after two attempts, other authors on the paper or listed on a registration record were emailed and we attempted to contact trialists using our networks. If authors are unable to be contacted after multiple attempts and the required individual participant data are not publicly available, the trial will be excluded, since our analyses are not possible using published summary data alone. Trialists that have agreed to join the Collaboration, as of August 2022, have been shaded in S2 Table.

Data collection, management and confidentiality

Individual participant variables

The individual participant data variables to be requested from trialists have been informed by expert consultation (Table 1) [39]. An initial list of individual characteristics potentially relevant to eating disorder risk was drafted in consultation with the EDIT Collaboration Scientific and Stakeholder Advisory Panels. These items were then developed into a survey and circulated internationally to eligible trialists and among obesity and eating disorder professional societies and consumer advocacy organisations. Participants were asked to rate the relevance of each individual characteristic to eating disorder risk within the context of weight management and to identify any variables relevant to eating disorder risk not already included [39]. Variables included in the final analyses will be dependent on the availability of IPD, i.e., we will only be able to include variables for which sufficient data are available.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Individual participant variables.

https://doi.org/10.1371/journal.pone.0282401.t001

Defining change in eating disorder risk

There is a paucity of literature to guide the assessment of and change in eating disorder risk in individuals with overweight or obesity. Few studies have validated self-report eating disorder questionnaires against a structured clinical interview in people with overweight or obesity [40]. Thus, it is challenging to identify normative data in this population to ascertain appropriate cut-points indicating someone may be at risk of, or may meet diagnostic criteria for an eating disorder. Similarly, there are no data to indicate clinically relevant changes in eating disorder risk scores on these self-report questionnaires when used longitudinally.

To identify participants at risk of developing an eating disorder in this study, we will assess the change in eating disorder risk in three ways. A standardised mean difference in eating disorder scores between baseline and post-intervention will be used as the primary outcome to determine a statistically significant change in eating disorder risk as a continuous variable. The Reliable Change Index [41] and pre-determined clinical cut-points will be secondary outcomes for assessments with appropriate data and reliability to inform analyses. The Reliable Change Index provides a measure of statistical and clinical significance while considering scale reliability [41, 42]. The index shows how much, and in what direction, an outcome measure has changed at the individual level, and whether that change is both reliable and clinically significant. Table 2 summarises the eating disorder measures used by included trials and the pre-defined cut-points used for each measure to categorize individuals as being high or low risk.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Eating disorder assessments used by trials eligible for inclusion in the EDIT Collaboration and their pre-specified risk cut-point.

https://doi.org/10.1371/journal.pone.0282401.t002

Questionnaires examining global eating disorder risk, for example, the Eating Disorder Examination Questionnaire or the Eating Disorder Inventory, usually include a sub-scale on dietary restraint. While dietary restraint or dieting is considered an eating disorder risk factor in community samples, dietary change is often a core component of behavioural weight management. This presents a challenge with examining eating disorder risk in the context of weight management interventions. A systematic review examining the change in dietary restraint compared to other eating disorder risk factors found that these sub-scales did not align in their direction of effect [49]. Thus, current measures of dietary restraint do not appear to be appropriate measures of eating disorder risk in the context of weight management, and existing sub-scales may not adequately distinguish between flexible and rigid restraint. Removal of the dietary restraint subscale used within questionnaires will be examined as part of sensitivity analyses.

Once the IPD dataset has been collated, these will be used in two separate studies addressing the two objectives.

Eligibility criteria

Study 1 will include weight management intervention arms only. Waitlist, no-treatment control arms and weight-neutral arms will be excluded. Adults and adolescents will be included, but analysed separately, and separate models will be built for each group.

Primary and secondary outcome

The primary outcome will be a standardised mean difference in global eating disorder score from baseline to immediately post-intervention and at 6- and 12- months follow-up, adjusting for the standardised global eating disorder scores at baseline [50, 51]. A secondary outcome will be a standardised mean binge eating score and frequency immediately post-intervention and at 6- and 12- months follow-up, adjusting for the standardised binge eating scores at baseline [50, 51].

Predictors: Individual risk factors

Individual participant characteristics in Table 1 will be tested as predictors of eating disorder risk. These risk factors will be adjusted based on available data in individual studies. If risk factors are not available across studies, they may not be analysed. We will critically discuss any lack of evidence on these factors in the publication of our results.

A final set of risk factors to be analysed, will be detailed in the Statistical Analysis Plan. The Statistical Analysis Plan will be finalised and made publicly available on Open Science Framework, with a timestamp, once data collection is finalised but before any analyses are undertaken [34].

Covariates

Where possible, the impact of the COVID-19 pandemic and related lockdowns will be examined as a covariate in studies collecting data from 2020 onwards.

Data analysis

Multilevel models

Separate models will be built for adults and adolescents using the individual participant data, following the same steps. We will build multilevel linear models with random effects for each intervention arm (i.e., each included treatment arm in each study) to account for intervention-level differences. All analyses will be performed using the open-source software R [52], using the package lme4 [53]. Models will be built for the primary outcome (global eating disorder scores) and secondary outcome (binge eating scores). If prognostic variables are highly correlated (|r| >0.7), this may corrupt estimates and model stability, and one of the variables will be removed from the model [54]. The selection of the variable removed will be documented by the authors, and based on its utility, pragmatic qualities and theoretical importance (e.g., which variable is more easily collected, and which variable would be hypothesized to be a more likely causal influence on the outcome). Heterogeneity across interventions will be assessed by calculating the proportion of random variance explained by random intervention effects.

Model stability, internal validation, and internal-external cross-validation

Variable selection can often produce optimistic models that have been overfitted to the development dataset and compromise the validity of model predictions for new populations. Due to this possibility, we will probe our model with multiple stability, validation and sensitivity analyses. We will use internal validation to examine overfitting and model stability via the resampling technique of bootstrapping. Bootstrapping will be done within each intervention arm, will be representative of the size of the intervention arm, and will be repeated 100 times. Each resampled dataset allows for the internal validation of how robust the variable selection process is to small deviations in input [55, 56]. Likewise, the development of new model estimates using the new bootstrapped datasets allows for the evaluation of optimism. The performance of the bootstrap models will be evaluated against the bootstrapped and original data. Specifically, we will examine the difference in performance of the bootstrap model on both datasets to create an optimism estimate and repeat this process 100 times for each performance statistic. The difference between the initial performance statistic based on the original model and the average of all optimism estimates provides an optimism adjusted performance statistic [58–60]. Internal-external cross-validation will be examined by rerunning the model development process omitting one study, the predictive performance of the model is then examined against the omitted study to externally validate the model. If there is an appropriate sample size [61–63], this process is repeated for each study and meta-analysis, and is used to summarise estimates and heterogeneity to evaluate how reliable the model is when applied to different populations [60]. If any studies have notable case-mix differences (e.g., majority of the sample has high socio-economic status or has class 3 obesity), their omission will also be explored in this cross-validation process to improve the generalisability of the model. We will perform a sensitivity analysis to explore if basic conclusions hold when comparing the final model with those obtained after eliminating some variables from the final model or including additional ones. Finally, we will perform sensitivity analyses to explore model stability with the spread of collinear variables.

Eligibility criteria

Study 2 will include trials with one or more weight management intervention arms being compared to a waitlist or no-treatment control arm and trials comparing a weight management intervention with a weight-neutral health-focused intervention. Adults and adolescents will be included, but analysed separately, and separate models will be built for each.

Primary outcome

The primary outcome will be a standardised mean difference in global eating disorder scores between baseline and immediately post-intervention and at 6- and 12- months follow-up.

Secondary outcome

The secondary outcome will be a standardised mean difference in binge eating scores and frequency between baseline and immediately post-intervention and at 6- and 12- months follow-up.

Individual risk factors

Individual participant characteristics in Table 1 will be examined as risk factors for eating disorder development. Due to multiple comparisons inflating our type 1 error risk, significant results will be interpreted with caution, assessing the pattern of results instead of single significant findings in isolation, and taking into account the range of uncertainty using the 95% confidence interval [64, 65]. As with Study 1, these risk factors will be adjusted based on the data availability and be detailed further in the Statistical Analysis Plan.

Data analysis

We will compare individual participant characteristics (Table 1) descriptively at baseline within the intervention and control group for each study individually and across studies. We will describe the number of participants with a statistically significant increase or decrease in global eating disorder or binge eating score in both the intervention and the control group, and the number of participants moving from below to above, or from above to below, a pre-defined eating disorder risk cut-point (Table 2).

The main analysis will be performed using a random-effect one-stage approach, stratified by trial. To assess whether interventions compared to control affect primary and secondary outcomes overall, we will build multilevel linear models for the continuous primary outcome of global eating disorder score, and other continuous outcomes, and general linear models with a log link function for binary outcomes (e.g., above or below a cut-point). We will be adjusting for eating disorder score at baseline to improve precision, as recommended by Riley et al. [33]. The heterogeneity of treatment effects across trials will also be estimated using I² which captures the proportion of the total variance driven by heterogeneity, 95% prediction intervals which capture the likely range of effect sizes from an equivalent population of studies, and τ² the between study variance. If excessive statistical heterogeneity in treatment effects or inconsistency across trials is detected, then the rationale for combining trials will be questioned and the source of heterogeneity explored.

To analyse individual-level risk factors (i.e., analyse if there are differential treatment effects on eating disorder outcomes for subgroups of participants/ by participant-level predictor variables) differences in intervention effect by risk factor on eating disorder risk score will be analysed by testing an intervention by risk factor interaction term. To account for aggregation bias, this analysis will be stratified by trial, with random effects placed on the within trial interaction [66]. Results will be reported with appropriate estimates, 95% confidence intervals and two-sided p-values of intervention effects and interaction effects.

Risk of bias assessment

Two reviewers will assess the risk of bias independently with discrepancies resolved through discussion. Each intervention arm in Study 1 will be assessed using the US Academy of Nutrition and Dietetics Quality Criteria Checklist: Primary Research [67]. The checklist allows a rating (positive, neutral, or negative) of each intervention arm within the study. The Cochrane Risk of Bias Tool [68] will be used to assess risk of bias for trials included in Study 2.

Data quality and consistency

Data quality will be assessed by examining the IPD and any published materials following a checklist of items derived from the literature and expert consultations, which has been piloted in previous IPDs (manuscript in progress). The IPD checks will examine items such as randomisation methods and sequences, inconsistencies in the data or implausible values, retraction notices and ethical approval. Any concerns will be cross-checked with the trialists.

Publication bias

Potential selection and publication bias will be investigated by comparing IPD meta-analyses of prospectively versus retrospectively included trials in a sensitivity analysis. Where possible, we will include unpublished data, which may alleviate selective outcome reporting bias. Contour-enhanced funnel plots will examine possible differences in results to detect indications of publication bias.

Confidence in cumulative evidence

Certainty of evidence will be appraised using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach [69].

Missing data

Baseline characteristics of participants completing studies and those who have withdrawn will be compared to identify any differences relevant to eating disorder risk. The principles of Jakobsen et al. [70] will be used to determine whether we will apply complete case analysis (CCA) or multiple imputation. If multiple imputation is the most appropriate approach, we will assume missing at random (MAR) and we will apply a fully conditional (FCS) approach labelled multiple imputation by chained equations (MICE). One regression model will be applied to each trial and treatment group separately [71]. Fifteen imputed dataset copies will be created and meta-analysis will be applied to each dataset and combined using Rubin’s rules. Care will be taken to ensure congeniality between the analysis model and the imputation model where possible (e.g., clustering, non-linear effects etc). This process will be applied with the mice package in R [72]. If large proportions (>40%) of data are missing within a trial for an outcome, this trial will be excluded from the primary analysis and included in a sensitivity analysis with complete case analysis to probe whether trials with high missingness influence our estimates.

Planned sensitivity analyses

Sensitivity analyses will be conducted for Study 1 and 2 excluding interventions with high rates of missing data (>30%), high risk of bias in any domain, trials with a possible conflict of interest and excluding dietary restraint sub-scales within eating disorder scores.

Additional planned analyses

The following additional planned analyses will be conducted, dependant on additional funding, and a separate protocol will be drafted and made publicly available for each: 1. Change in weight/ BMI related outcomes between baseline and post-intervention and 6- and 12-months follow-up as a predictor of change in eating disorder risk. Where possible for adolescent data, height and weight will be used to standardise BMI across studies (such as using BMI z-score and/ or BMI as a percentage of the 95^th percentile); 2. Rate of weight loss between baseline and post-intervention and 6- and 12-months follow-up as a predictor of eating disorder risk; 3. Change in cardiometabolic health outcomes (e.g., lipids, insulin, blood pressure, liver function tests) between baseline and post-intervention and 6- and 12-months follow-up (dependant on additional funding to collate outcomes).; 4. Understanding the temporal change in global eating disorder scores and binge eating scores during weight management interventions and during the follow-up period.

Members of the EDIT Collaboration as at August 2022

Study team: Natalie Lister, Hiba Jebeile, Anna L. Seidler, Brittany J. Johnson, Rabia Khalid, Hannah Melville, Sol Libesman, Ruth Tredinnick, Samantha Pryde, Caitlin M. McMaster, Kylie E. Hunter.

Scientific and Stakeholder Advisory Panel: Amy L. Ahern, Lisa Askie, Louise A. Baur, Caroline Braet, Kelly Cooper, Genevieve Dammery, Sarah P. Garnett, Rebecca Golley, Alicia Grunseit, Andrew J. Hill, Rebecca A. Jones, Ted K. Kyle, Sarah Maguire, Faith A. Heeren, Dasha Nicholls, Susan J. Paxton, Fiona Quigley, Amanda Sainsbury-Salis, Kate Steinbeck, Denise E. Wilfley, Jacqlyn Yourell.

Trial representatives: Rachel Barnes, Melanie Bean, Kristine Beaulieu, Maxine Bonham, Kerri Boutelle, Caroline Braet, Hoi Lun Cheng, Simona Calugi, Michelle Cardel, Kelly Carpenter, Riccardo Dalle Grave, Yngvild Sørebø Danielsen, Marcelo Demarzo, Aimee Dordevic, Dawn Eichen, Andrea Goldschmidt, Anja Hilbert, Katrijn Houben, Natalie B. Lister, Mara Lofrano Prado, Braulio Henrique Magnani Branco, Corby Martin, Anne McTiernan, Janell Mensinger, Faith A. Heeren, Dasha Nicholls, Carly Pacanowski, Stephanie Partridge, Wagner Prado, Sofia Ramalho, Hollie Raynor, Kay Rhee, Elizabeth Rieger, Eric Robinson, Amanda Sainsbury-Salis, Vera Salvo, Brian M Shelley, Nancy Sherwood, Sharon Simpson, Hanna F Skjakodegard, Evelyn Smith, Marian Tanofsky-Kraff, Rachael W. Taylor, Annelies Van Eyck, Krista Varady, Alaina Vidmar, Victoria Whitelock, Denise E. Wilfley, Jack Yanovski.