Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

The effect of remuneration schedule on data completion and retention in the pregnancy eating attributes study (PEAS)

  • Ndeah Terry,

    Roles Funding acquisition, Writing – original draft

    Affiliation Division of Intramural Population Health Research, Social and Behavioral Sciences Branch, Eunice Kennedy Shriver National Institute of Child Health and Human Development, Bethesda, Maryland, United States of America

    ⨯
  • Leah M. Lipsky,

    Roles Conceptualization, Resources, Supervision, Writing – review & editing

    Affiliation Division of Intramural Population Health Research, Social and Behavioral Sciences Branch, Eunice Kennedy Shriver National Institute of Child Health and Human Development, Bethesda, Maryland, United States of America

    ⨯
  • Anna Maria Siega-Riz,

    Roles Writing – review & editing

    Affiliation Department of Nutrition and Biostatistics and Epidemiology, School of Public Health and Health Sciences, University of Massachusetts, Amherst, Massachusetts, United States of America

    ⨯
  • Aiyi Liu,

    Roles Conceptualization, Formal analysis, Methodology, Validation

    Affiliation Division of Intramural Population Health Research, Social and Behavioral Sciences Branch, Eunice Kennedy Shriver National Institute of Child Health and Human Development, Bethesda, Maryland, United States of America

    ⨯
  • Tonja R. Nansel

    Roles Conceptualization, Methodology, Supervision, Writing – review & editing

    nanselt@mail.nih.gov

    Affiliation Division of Intramural Population Health Research, Social and Behavioral Sciences Branch, Eunice Kennedy Shriver National Institute of Child Health and Human Development, Bethesda, Maryland, United States of America

    ⨯

Abstract

Maximizing data completion and study retention is essential in population research. This study examined the effect of remuneration schedule and data collection modality on data completion and retention in the Pregnancy Eating Attributes Study cohort. Participants (n = 458) completed online surveys and attended six in-person study visits. Initially, remuneration was a prespecified amount per visit, then was changed mid-study to be prorated based on the number of forms completed. Additionally, survey data collection modality was changed to in-person at the sixth study visit. In this secondary data analysis, there was no effect of remuneration schedule on withdrawal rates or time-to-withdrawal. Survey completion was significantly lower under prorated remuneration at the first visit but did not significantly differ at subsequent visits. The lump sum group had significantly greater odds of completely the first and second trimester dietary record (OR = 4.1, OR = 2.6, respectively) then the prorated group but were almost half as likely to complete the dietary record at the 6-month postpartum visit (OR = 0.5). Survey completion at sixth visit was significantly higher for in-person versus online completion (68.6% vs. 93.1%). Findings suggest that remuneration schedule and data collection modality can impact completion of self- reported assessments.

Citation: Terry N, Lipsky LM, Siega-Riz AM, Liu A, Nansel TR (2021) The effect of remuneration schedule on data completion and retention in the pregnancy eating attributes study (PEAS). PLoS ONE 16(5): e0251533. https://doi.org/10.1371/journal.pone.0251533

Editor: Corrie Whisner, Arizona State University, UNITED STATES

Received: November 9, 2020; Accepted: April 27, 2021; Published: May 13, 2021

This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: This research was supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development Intramural Research Program (contract #HHSN275201300015C and #HHSN275201300026I/HHSN27500002). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Participants in research studies are typically provided with monetary remuneration as compensation for their time and effort. Remuneration may influence participation, data completion, or study retention [1–4], thereby impacting internal and external validity. Furthermore, recent advances in technology have facilitated the expansion of off-site, participant-initiated data collection, but whether this data collection modality impacts data completion or participant retention is unknown. Understanding how remuneration and data collection modality influence recruitment, retention, and data completion is critical for informing the most efficient and cost-effective design of human subjects’ research.

Participant remuneration is typically distributed via a predetermined schedule based on the time and effort associated with participation (e.g. assessment time, cost of transportation, child care, etc.), and evidence suggests that adequate remuneration may be critical to incentivize study participation [5–8] and retention [9]. Remuneration schedule, which refers to the system of dispersal of funds to participants throughout a study, varies across studies [10]. Remuneration may be provided conditionally (i.e. only after the completion of certain study tasks) in either a lump sum at a single time point or piecemeal, or as a prespecified amount paid unconditionally (i.e. not study task dependent) throughout the study according to milestones (i.e. number of visits completed) as specified by an institution’s human subjects’ review board. Another approach is to enter participants into a lottery for a gift card or monetary reward, if allowed by the institutional review board. To our knowledge, no studies have investigated how different payment schedules affect data completion and retention.

Additionally, studies use different strategies to collect data that may impact participant retention and data completion. Self-report measures may be collected, for example, at a central study location, at in-home assessments conducted by research staff, by telephone, or via participant self-administered online assessments. In-person survey completion at a central location may improve efficiency from the investigator’s perspective but necessitates physical space and on-site staffing and requires participant effort in terms of scheduling, transportation and parking. Alternatively, off-site participant-initiated survey completion via secure website or mobile applications may be more flexible and reduce participant burden associated with attending study visits at a central location. However, this approach may increase the cognitive burden associated with initiating and completing assessments, may be impacted by distraction, competing priorities, and limits participants to those who have internet or mobile access. Differences in these data collection modalities may influence participant retention and data completion. However, this has not been empirically investigated.

The purpose of this secondary analysis was to investigate the effect of remuneration schedule and data collection modality on participant data completion and retention in the Pregnancy Eating Attributes Study (PEAS). The study enrolled a cohort of women ≤12 weeks gestation to study eating behaviors and weight change from pregnancy through one-year postpartum. All participants were drawn from the same source population; however, two changes to study procedures were made based on findings from ongoing data collection monitoring. Because initial data completion rates during pregnancy assessments were lower than anticipated, the remuneration schedule was changed mid-study. Those recruited early in the study received a prespecified remuneration amount at each study visit regardless of how many self-initiated online forms they completed. Participants recruited later in the study were paid conditionally for each online form they completed. Additionally, in response to poorer completion of self-initiated, off-site online surveys during postpartum, data collection procedures were changed, and the number of required surveys were reduced for the final study visit such that participants completed surveys at the in-person assessment. These changes in study procedures facilitate an investigation into whether differences in remuneration schedule impacted data completion or withdrawal, and whether in-person versus off-site participant-initiated data collection modality influenced data completion.

Methods

Study design and participants

PEAS was a prospective observational study of 458 healthy pregnant women recruited at ≤12 weeks gestation and followed through one-year postpartum. Details of study recruitment and methods have been published elsewhere [11]. Participants were recruited from women receiving prenatal care at two obstetrics clinics in the University of North Carolina at Chapel Hill Healthcare System. Inclusion criteria were: confirmed pregnant ≤12 weeks gestation at enrollment; uncomplicated singleton pregnancy anticipated; age 18–45 years at screening; willingness to undergo study procedures and provide informed consent for her participation and assent for the baby’s participation; BMI ≥18.5 kg/m2; able to complete self-report assessments in English; access to internet with email; plan to deliver at the University of North Carolina Women’s Hospital; and plan to remain in the geographical vicinity of the clinical site for one year following delivery. Exclusion criteria included pre-existing diabetes; multiple pregnancy; participant-reported eating disorder; any chronic illnesses or use of medication that could affect diet or weight; psychosocial condition hindering participation in the study. Recruitment occurred from November 2014 to December 2016. Data collection was completed by August 2018. Protocols including modifications to the mode and remuneration were approved by the University of North Carolina at Chapel Hill institutional review board.

Anthropometrics and biospecimens were collected at in-person study visits once per pregnancy trimester and three times between delivery up to one year postpartum. Participants were also asked to complete self- administered online surveys on eating- and health-related behaviors and a 24-hour dietary recall outside of study visits via a secure study website. Participants logged on to the website with their username and password within specified time windows around each visit: window were from 6–12 weeks, 16–27 weeks, and 28–36 weeks gestation; and 4–14 weeks, 23–31 weeks, and 50–58 weeks postpartum. The website listed all required surveys for that window, with a link to the online survey form, and participants could complete them all at once or across multiple logins. When the visit window closed, the surveys were no longer accessible. Research assistants monitored the online data system to determine whether subjects completed the surveys and 24-hour dietary recalls. For subjects with incomplete forms, email reminders were given three weeks prior and phone reminders one week prior to window closure.

Remuneration

Initially, participants received a prespecified remuneration at each study visit—$50 each for the first and third prenatal and first postpartum visit; $75 each for the second prenatal and second postpartum visit, and $100 for the final postpartum visit, for a total of $400 for completion of all visits. Due to low completion rates of online forms in the first several months of data collection, the remuneration schedule was changed in February 2016 for all subsequently recruited participants to be prorated based on the number of completed self-administered forms, under the hypothesis that a pro-rated remuneration schedule would increase survey completion. Under the prorated remuneration schedule, participants received $15 for each clinic visit, $15 for the dietary recall, and $3-$8 per online form (based on the form length), for a maximum total of $400. The first 284 participants received lump sum remuneration; the remaining 174 received prorated remuneration. Participants remained under the same remuneration schedule for the entire study.

Data collection procedures at final study visit

When ongoing study monitoring indicated particularly low rates of data completion at the one- year postpartum visit, a second change in study procedures occurred. To ensure that the most important self-report measures were obtained at the final postpartum study visit, several surveys were eliminated from the assessment schedule, reducing completion time by about half, and participants were asked to complete all surveys during the clinic visit if they had not already completed them at home. This change occurred in March 2017, after 122 participants had already completed the final visit; 199 participants completed the final visit under the revised data collection modality.

Statistical analysis

Differences between the two remuneration groups in socio-demographic characteristics were examined using Student t-tests for continuous variables and Pearson’s chi-squared test for categorical variables; education and income were included as covariates in all subsequent analyses by remuneration schedule. Remuneration schedule as a predictor of withdrawal by visit was modeled using Poisson regression. Group differences in study retention at one year postpartum were examined by Student t-test. Test for differences between remuneration groups in the percent of measures completed at each assessment period was determined by analysis of covariance. Differences in percent of participants who completed the 24-hour dietary recall at each visit by remuneration group were examined by logistic regression with prorated remuneration as the referent group. Data completion rates before and after the changes to the last postpartum visit were examined by Student t-test. All analyses were completed using SAS version 9.4.

Results

Participants were mostly white, highly educated, and working at least part time (Table 1). There were no significant differences between remuneration groups in age, marital status, employment, education, race, ethnicity, and receipt of government aid. Differences in education and household income approached statistical significance and were, therefore, used as covariates in subsequent analyses.

thumbnail
Download:
Table 1. Baseline sociodemographics of women in PEASa under lump sum and prorated remuneration.

https://doi.org/10.1371/journal.pone.0251533.t001

Of 458 participants enrolled, 367 remained in the study through delivery and 321 through one-year postpartum for an overall study retention rate of 70%. Among the participants that withdrew, 91 (20%) withdrew prior to delivery and 46 (10%) withdrew during postpartum. Reasons for withdrawal included 48 no longer willing to participate; 29 experienced miscarriage, stillbirth, or death of baby; 17 moved away or changed medical provider; 37 were noncompliant with study visits; and 6 developed conditions resulting in ineligibility. There was no significant difference in the time to withdrawal between the two remuneration schedules, and no interaction of visit with remuneration schedule on number of withdrawals (95% confidence interval [CI] = -0.11,0.52; p = 0.19). There were no significant differences in the number of withdrawals between the two remuneration groups (27% in the lump sum remuneration, 33% in the prorated remuneration, χ2 = 2.13, p = 0.14).

Survey completion was significantly lower under prorated remuneration than lump sum remuneration at the first trimester visit (Table 2). Completion rates did not significantly differ between remuneration schedules at the subsequent study visits. Similarly, participants in the lump sum remuneration group were more likely to complete the 24-hour dietary recalls at the first two visits, but less likely to complete dietary recalls at 6 months postpartum. No differences were observed at subsequent visits (Table 3).

thumbnail
Download:
Table 2. Survey completion by remuneration schedulea.

https://doi.org/10.1371/journal.pone.0251533.t002

thumbnail
Download:
Table 3. Dietary record completion (n, %) and OR (95% CI) of dietary record completion associated with remuneration schedulea.

https://doi.org/10.1371/journal.pone.0251533.t003

Data completion at the one-year postpartum visit was significantly higher when participants completed surveys at the study visit, with a mean±se of 68.6±3.9% survey completion prior to procedural change, versus 93.1±1.8% afterward, (p < .001).

Discussion

To increase data completion in this study of women assessed during pregnancy and postpartum, investigators changed the remuneration schedule approximately midway through data collection by linking remuneration amount to completion of each self-report measure rather than using a lump-sum remuneration schedule. Study findings indicate that the prorated remuneration schedule resulted in lower data completion than lump sum remuneration at initial study visits. No differences were observed in data collection at later study visits, and retention and time to withdrawal were unchanged. As such, the findings are contrary to the research team’s hypothesis and intention for changing the remuneration schedule mid-study. In contrast, changing data collection modality from off-site, participant-initiated to in-person assessment at the one- year postpartum visit had the most significant effect on data completion, with substantially higher data completion after the modality change.

The absence of an effect of remuneration schedule on retention suggests that other factors likely influenced withdrawals. Motivations for research participation previously reported include scientific interest or curiosity, the desire to further scientific knowledge, and humanitarian reasons [12]. Intrinsic motivators such as willingness to help medical research, improving the knowledge of science, and altruism are the most frequent reasons pregnant women report entering clinical studies [13]. Although payment is typically expected for study participation, participants in one study in a lower-income South African population reported they were willing to participate even if no compensation were provided [14]. Monetary compensation may be among the top motivating factors in populations of low-income or disproportional unemployment [15]. The PEAS sample was different in that on average women were highly educated and of relatively high income. While intrinsic motivators and monetary compensation may motivate research participation, retention may be impacted by unrelated issues such as changing family circumstances, health events, or job responsibilities [16, 17]. Multiple retention strategies that have been shown to increase retention rates [18–20] were used in PEAS including periodic newsletters, holiday cards, and provision of convenient times and locations for study visits. Therefore, retention in the PEAS sample may be more attributable to the various strategies used, rather than remuneration schedule.

While lower data completion under prorated remuneration was unexpected, results may be consistent with previous findings suggesting that monetary incentives may undermine intrinsic motivation [21, 22]. While task-noncontingent monetary rewards like lump sum remuneration have shown no impact on intrinsic motivation [23], task-contingent rewards like prorated remuneration have been found to decrease intrinsic motivation and reduce performance. As such, the prorated remuneration schedule used in PEAS could have decreased participants’ intrinsic motivation (i.e. a motivation shift from intrinsic to extrinsic), thus resulting in lower data completion rates. However, this explanation would not account for the lack of differences observed by remuneration schedule at later study visits. Additionally, the amount offered per survey under prorated remuneration may not have been adequate to motivate participants to complete surveys given the income levels of our participants. One study assessing performance quizzes and volunteer tasks at different payment levels suggests that effect of monetary incentives in small amounts can be detrimental to performance [24]. Thus, these findings taken together with previous work suggest that larger lump sum payments may lead to more favorable data completion and retention, and partitioning remuneration into a smaller series of payments may be a deterrent to data completion and retention.

Analysis of the one-year postpartum visit indicated that survey completion was dramatically improved by administering the surveys in-person during the study visit and reducing the number of surveys rather than relying on patient-initiated survey completion offsite. While logistical issues at the clinical site did not allow for in-person survey administration during pregnancy, and there is no literature directly comparing in-person survey administration versus off-site participant-initiated survey administration, these findings suggest that in-person administration of measures should be used whenever feasible and underscore the need to determine methods to improve self-administered survey completion.

Study findings should be interpreted in light of several limitations. Participants were not randomized into the remuneration groups; however, there were no significant differences in the socio-demographic characteristics between groups and no known historical changes across the study period (e.g. changes in study procedures, eligibility criteria, recruitment rates, or the population served by the clinics) that would impact comparability of the two groups. The study sample was largely well-educated with limited socioeconomic or racial diversity, and from a single geographic region; thus, findings may not be generalizable to participants with different demographic characteristics.

Conclusion

Findings from this study indicate that remuneration schedule and data collection modality can impact completion of self-reported assessments. Changing the remuneration from a lump sum, task-noncontingent approach to a task-contingent prorated system resulted in lower data completion rates at initial visits but did not result in differential data completion at later visits. In contrast, changing the data collection modality from off-site, self-initiated to in-person resulted in substantial improvement in data completion. Further research is needed to understand how remuneration practices and data collection modality intersect with the economic status and demographics of diverse populations to influence data completion and retention.

References

  1. 1. Brealey SD, Atwell C, Bryan S, Coulton S, Cox H, Cross B, et al. Improving response rates using a monetary incentive for patient completion of questionnaires: an observational study. BMC Medical Research Methodology. 2007;7(1). pmid:17326837
  2. 2. Simmons E, Wilmot A, Incentive payments on social surveys: a literature review. Social Survey Methodology Bulletin. 2004; 1–11.
  3. 3. Singer E, Groves RM, Corning A. Differential Incentives: Beliefs about practices, perceptions of equity, and effects on survey participation. Public Opinion Quarterly. 1999;63(2):251–60.
  4. 4. Edwards PJ, Roberts IG, Clarke MJ, Diguiseppi C, Wentz R, Kwan I, et al. Methods to increase response rates to postal questionnaires. Cochrane Database of Systematic Reviews. 2007;(2):MR000008. pmid:17443629
  5. 5. Bentley JP. The influence of risk and monetary payment on the research participation decision making process. Journal of Medical Ethics. 2004;30(3):293–8. pmid:15173366
  6. 6. Grady C, Bedarida G, Sinaii N, Gregorio MA, Emanuel EJ. Motivations, enrollment decisions, and socio-demographic characteristics of healthy volunteers in phase 1 research. Clinical Trials: Journal of the Society for Clinical Trials. 2017;14(5):526–36. pmid:28783972
  7. 7. Almeida L, Azevedo B, Nunes T, Vaz-Da-Silva M, Soares-Da-Silva P. Why healthy subjects volunteer for phase I studies and how they perceive their participation? European Journal of Clinical Pharmacology. 2007;63(11):1085–94. pmid:17891536
  8. 8. Bigorra J, Baños JE. Weight of financial reward in the decision by medical students and experienced healthy volunteers to participate in clinical trials. European Journal of Clinical Pharmacology. 1990;38(5):443–6. pmid:2379528
  9. 9. Janson SL, Alioto ME, Boushey HA. Attrition and retention of ethnically diverse subjects in a multicenter randomized controlled research trial. Controlled Clinical Trials. 2001;22(6).
  10. 10. Stunkel L, Grady C. More than the money: a review of the literature examining healthy volunteer motivations. Contemporary Clinical Trials. 2011;32(3):342–52. pmid:21146635
  11. 11. Nansel TR, Lipsky LM, Siega-Riz AM, Burger K, Faith M, Liu A. Pregnancy eating attributes study (PEAS): a cohort study examining behavioral and environmental influences on diet and weight change in pregnancy and postpartum. BMC Nutrition. 2016;2(1). pmid:28663822
  12. 12. Tishler CL, Bartholomae S. The Recruitment of normal healthy volunteers: a review of the literature on the use of financial incentives. The Journal of Clinical Pharmacology. 2002;42(4):365–75. pmid:11936560
  13. 13. Zande ISVD Graaf RVD, Hooft L Delden JJV. Facilitators and barriers to pregnant women’s participation in research: a systematic review. Women and Birth. 2018;31(5):350–61. pmid:29373261
  14. 14. Burgess LJ, Sulzer N, Emanuel S. Clinical trial remuneration: the patients’ perspective. SAMJ: South African Medical Journal. 2008 Feb;98(2):95–7. pmid:18350200
  15. 15. Grady C. Payment of clinical research subjects. Journal of Clinical Investigation. 2005;115(7):1681–7. pmid:16007244
  16. 16. Baker L, Lavender T, Tincello D. Factors that influence women’s decisions about whether to participate in research: an exploratory study. Birth. 2005;32(1):60–6. pmid:15725206
  17. 17. Brumatti LV, Montico M, Russian S, Tognin V, Bin M, Barbone F, et al. Analysis of motivations that lead women to participate (or not) in a newborn cohort study. BMC Pediatrics. 2013;13(1). pmid:23294523
  18. 18. Abshire M, Dinglas VD, Cajita MIA, Eakin MN, Needham DM, Himmelfarb CD. Participant retention practices in longitudinal clinical research studies with high retention rates. BMC Medical Research Methodology. 2017;17(1). pmid:28219336
  19. 19. Robinson KA, Dennison CR, Wayman DM, Pronovost PJ, Needham DM. Systematic review identifies number of strategies important for retaining study participants. Journal of Clinical Epidemiology. 2007;60(8). pmid:17606170
  20. 20. Robinson KA, Dinglas VD, Sukrithan V, Yalamanchilli R, Mendez-Tellez PA, Dennison-Himmelfarb C, et al. Updated systematic review identifies substantial number of retention strategies: using more strategies retains more study participants. Journal of Clinical Epidemiology. 2015;68(12):1481–7. pmid:26186981
  21. 21. Himmelstein DU, Ariely D, Woolhandler S. Pay-for-performance: toxic to quality? Insights from behavioral economics. International Journal of Health Services. 2014;44(2):203–14. pmid:24919299
  22. 22. Promberger M, Marteau TM. When do financial incentives reduce intrinsic motivation? Comparing behaviors studied in psychological and economic literatures. Health Psychology. 2013;32(9):950–7. pmid:24001245
  23. 23. Strang S, Park SQ, Strombach T, Kenning P. Applied economics: The use of monetary incentives to modulate behavior. In Progress in Brain Research 2016 Jan 1 (Vol. 229, pp. 285–301). Elsevier. https://doi.org/10.1016/bs.pbr.2016.06.010 pmid:27926443
    • 24. Gneezy U, Rustichini A. Pay enough or don’t pay at all. Quarterly Journal of Economics. 2000;115(3):791–810.