Defining PPIE for this review

Our working definition for PPIE with children and young people for this review builds on the previously mentioned INVOLVE definitions [3] which are the most widely referred to definitions within the UK. Our intentionally broad definition of PPIE for this review–‘health related research being carried out with or by children and young people rather than to, about or for them’; or, ‘where information and knowledge about research is provided and disseminated’ [3].

Stage 1: Identifying the research question

An overarching research question guided our systematic search strategy and reporting of results: What is known about public involvement and engagement activities with children and young people in health related research? This question enabled us to adequately capture a broad range of existing literature while providing the opportunity for further research objectives to be added and modified throughout the review. This iterative process was useful as we became increasingly familiar with the literature.

Stage 2: Identifying relevant studies

Search terms were developed based on consideration of the population (children and young people aged under 25 years); concept (public involvement defined as doing research ‘with’ or ‘by’ the public, rather than ‘to’, ‘about’ or ‘for’ the public [32] and public engagement defined as “where information and knowledge about research is provided and disseminated” [1]; and context (health research).

A comprehensive list of search terms was identified and refined through searches on Scopus and Medline by the review team. AR and JH led the development of the search strategy with the support of a health research librarian. Truncation and proximity operators were employed to increase the sensitivity of the search (see S1 Appendix for full Medline search). Searches were undertaken in Scopus, Medline, CINAHL, Cochrane and PsychInfo databases in December 2019.

The reference lists of included documents were reviewed for additional papers, and Scopus and Google Scholar were consulted to identify the citing literature. A search of the grey literature (pdf files and webpages), including a hand search, was also completed in December 2019. Grey literature was identified via Open Grey, Google and from the websites of the Royal College of Paediatrics and Child Health, Royal College of Nursing, Barnardo’s, Department of Health, National Institute of Health Research Portfolio, Childlink, WellChild, Shine, Generation R, UNICEF, Save the Children, INVOLVE and iCan for research and conference publications.

Stage 3: Study selection

Studies were included in the review if they met the criteria outlined in Table 1. The search parameters were papers published 1st January 2000 – 16th December 2019. The age range of 0-≤24 years was used to encompass children and young people and was determined based on the World Health Organization (2014) [33] definition of a ‘child’ as a person under the age of 18 years and a ‘young person as under 24 years of age’ (10–24 years).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Inclusion and exclusion criteria.

https://doi.org/10.1371/journal.pone.0252774.t001

Citations were imported into Covidence (an online review tool that simplifies the screening and extraction process) and duplicates were removed. Following this, study selection occurred using a two-stage screening process. In Stage 1, titles and abstracts of all papers were each blind reviewed by two members of the review team (AR, JH) and conflicts resolved by a third reviewer from the team (TB, LB, BC). Seven papers, unavailable in full-text format, were excluded. In Stage 2, all members of the review team (AR, JH, TB, LB, BC) were allocated papers with two reviewers each screening all full-text papers. Conflicts were resolved through discussion between the two reviewers or where further advice needed by a third reviewer. Where papers relating to the same study reported on different aspects of the PPIE, all relevant papers were included.

Stage 4: Charting the data

We iteratively developed and refined a data extraction table reporting on nine key sections (Table 2) based on the Guidance for Reporting Involvement of Patients and the Public 2 (GRIPP2 short and long-form) [27]. The Guidance for Reporting Involvement of Patients and the Public–short-form and long-form (GRIPP2-SF and GRIPP2-LF) [27] reporting checklists provide key items for the reporting of PPI. The short version (GRIPP2-SF) is for reporting PPI in any study and a long version (GRIPP2-LF) is for when the study is mainly about PPI in research.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Overview of data extraction items based on adapted Guidance for Reporting Involvement of Patients and the Public 2 (GRIPP2) form [27].

https://doi.org/10.1371/journal.pone.0252774.t002

Core data were extracted on the author(s), year of publication, country of origin, research study aim, methodology, method and sample. PPIE-specific data extraction (Table 2) focused on PPIE aim, PPIE terminology, definition and underpinning concepts/theory; PPIE population; PPIE design and stages of involvement; methods by which PPIE was evaluated; evidence of the impact of PPIE; conclusions and lessons learned from PPIE; limitations related to PPIE; and recommendations arising from PPIE. Five reviewers (AR, JH, TB, LB, BC) independently extracted data from their allocated articles with extractions checked for consistency and condensed by one reviewer (BC).

Quality assessment.

Although assessing the methodological and other qualities of studies within a scoping review can help contextualise findings and enable interpretation, Arksey and O’Malleyhttps://onlinelibrary.wiley.com/doi/full/10.1111/hex.13069 - hex13069-bib-0019 [30] suggest that quality assessment is not required and Levac [31] recognises the difficulties in assessing quality across a wide range of published and grey literature. Despite awareness of these issues, we initially sought to report on the quality of the studies using the Mixed Methods Appraisal Tool (MMAT) [34]. However, the MMAT directed our attention towards methodological issues when we were most interested in the quality of reporting of the PPIE.

No tool existed that enabled us to assess the quality of reporting from a broad range of papers, and which was not overly onerous. Having used elements of the GRIPP2 (LF- and SF) [27] to chart and extract the relevant information from the papers, we developed a short appraisal tool based on key areas of the GRIPP2-LF [27] using a similar format to the MMAT, to support our review of the quality of PPIE reporting. This tool focused on items necessary to assess the quality of the reporting for the focus of our review. One reviewer (AR) led the development of this appraisal tool. Through an iterative and condensing process involving the whole team, items from each section of the GRIPP2-LF were selected and turned into questions to develop a tool we named the QRIPPAT (Quality of Reporting Involvement of Patients and the Public Appraisal Tool). The QRIPPAT focuses on four broad themes each with several questions: PPIE aim and definition, PPIE methods, PPIE findings and discussion, and PPIE learning and reflections, all of which require only a Yes/No response (Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. QRIPPAT (Quality of Reporting Involvement and Engagement of Patients and the Public Appraisal Tool).

https://doi.org/10.1371/journal.pone.0252774.t003

Assigning scores to quality appraisal can be seen as contentious, especially if the scoring system is unweighted or unvalidated. However, scoring papers can reflect a measure of their worth and the score can be used, along with other considerations, as part of the critique of a paper. With this in mind, we have equally weighted each element of the QRIPPAT and assigned a score of 1 for each Yes and 0 for each No, giving a range of 0–10, where 0 is the lowest score for quality and 10 the highest quality score. The results of the QRIPPAT assessment (Table 4) reveals the overall quality in the reporting of the PPIE in the studies under review.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Quality of Reporting Involvement and Engagement of Patients and the Public Appraisal Tool (QRIPPAT).

https://doi.org/10.1371/journal.pone.0252774.t004

Stage 5: Collating, summarising, and reporting results

Collated and summarised information from the included papers was added to the data extraction table in S1 Table and it proved invaluable in helping to develop initial themes that were finalised through discussions involving all members of the research team. Due to the wide variation in PPIE approaches in the included studies and significant variability in the quality of reporting of PPIE, a narrative approach offered the best fit to summarise the results of the review. The NIHR’s research process model [68] which outlines eight stages in the research process in which research-oriented PPIE can take place was condensed to create a better fit for reporting our findings. The findings outlined were explicitly reported by the study authors and not inferred by the authors of this review.

Stage 6: Consultation

Although not commonly used within scoping reviews, we were determined to ensure our review modelled good PPIE practice. Before commencing the review, we engaged with a young man who is an experienced service user via a service user group within our Faculty. This discussion helped to refine the scope of the review and, through his involvement, we incorporated exploring the wider impacts of PPIE and a critical consideration of not only which children and young people are included in PPIE but also those who may be excluded. Furthermore, the review benefited from the expertise of a team member who is a skilled Family Engagement Officer (RM) who challenged and influenced the team’s thinking through her insights into the priorities and areas which might be most important for children and young people involved in PPIE activities. RM provided feedback on the QRIPPAT extraction and the overall data extraction, checked the credibility of the evidence synthesis, identified gaps in the data and made suggestions as to present the findings.

Building on our earlier consultation work, the lay summary of the key points of the evidence generated from the review and recommendations for researchers is being co-produced by a group of young service users.

Study characteristics

The 40 articles included 38 primary research studies (the PPIE from one study was reported in three articles) [39, 60, 67]. The PRISMA diagram below (Fig 1) illustrates the article search, screening, and review process. The PPIE was undertaken as part of research conducted in the United Kingdom (n = 32) [18–20, 22, 36–39, 42, 44, 45, 47–67], Australia (n = 3) [14, 43, 46], United States (n = 3) [21, 35, 41], Canada (n = 1) [13] and New Zealand (n = 1) [40]. The majority of the articles (n = 37) were published after 2010 [13, 18–22, 35–45, 47–52, 54–67] with three published between 2000–2010 [14, 46, 53].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. PRISMA diagram.

https://doi.org/10.1371/journal.pone.0252774.g001

Children and young people reported as being involved or excluded in PPIE

Most studies gave demographics of the children and young people involved in the PPIE in terms of age, with fewer (n = 22) specifying more detailed demographics, for example, gender or ethnicity, health conditions or health experiences of the children and young people [14, 18, 35, 36, 38, 40–44, 46, 48, 49, 51, 53, 56–59, 61, 62, 64]. Five studies did not report the age range of the children and young people [40, 44, 46, 53, 59]. One study specified they had engaged children younger than four years [37] and one had engaged with children and young people from 4–24 years old [22]. Seven studies had solely engaged those in young adulthood (18–25 years) [13, 18, 20, 35, 39, 60, 67] and twelve had solely engaged with adolescents (10–18 years) [21, 36, 42, 43, 45, 50, 51, 55, 58, 61–63]. No studies had engaged solely with children <11 years. Five studies had spanned childhood and adolescence (aged between 5 to 18 years) [19, 48, 54, 64, 65] and nine had spanned adolescence and young adulthood (10–25 years) [14, 38, 41, 47, 49, 52, 56, 57, 66].

Eleven studies referred to children and young people having specific conditions: arthritis [45], asthma [37], cancer [66], diabetes [21, 39, 60, 67], physical disability [22, 50], PICU experiences [19, 54], and depression or stroke [20]. Three studies addressed more general health experiences, such as the experience of accessing healthcare services [47] or a range of health experiences or conditions [55, 65]. Three studies had engaged with children and young people with other experiences that impacted health, for example, drug use [13], care leavers [52], asylum seekers [52] and experiences of therapy [63].

Typically, the numbers of children and young people involved in PPIE were small, although the range was wide from three [18] to 300 [43]. Of the 33 studies which reported the number of children and young people involved, seventeen had involved ten or fewer children and young people [13, 18, 20, 21, 35–37, 39, 41, 42, 48, 50, 59, 60, 63, 65, 67], eight had involved 11–20 children and young people [14, 19, 38, 45, 47, 51, 52, 55], six had involved 21–100 children and young people [22, 49, 54, 58, 61, 62] and two between 200–300 children and young people [43, 66].

In thirteen studies, the PPIE work involved a broader population than just children and young people and included parents [18–21, 45, 55, 64], adult stakeholders [14, 54], professionals [58] and combinations of families and professionals [22, 50, 65].

Methods reported as being used to involve children and young people in PPIE

A condensed version of the NIHR research process model [68] (Fig 2) was used to guide the findings of children and young people’s involvement in PPIE. PPIE activities were reported as occurring throughout various stages of a research study, and most commonly, PPIE with children and young people occurred across multiple stages. However, in twelve studies children and young people’s involvement was limited to one stage of the research [19, 36, 37, 40, 44, 46, 50, 54, 61, 63, 64, 66], this was frequently within the initial stages of prioritising, design and development the research [19, 37, 44, 50, 54, 63, 64, 66]. In eight studies PPIE with children and young people occurred across two stages of a research study: prioritising, design and development, and undertaking stages [14, 48, 58, 65]; undertaking and analysis stages [42, 47]; analysis and dissemination stages [20]; and prioritising, design and development, and dissemination stages [18]. In twelve studies PPIE with children and young people occurred across three stages: prioritising, design and development, undertaking research, and analysis stages [35, 41, 43]; prioritising, design and development, undertaking research, and dissemination stages [21, 56, 57]; prioritising, design and development, analysis, and dissemination stages [67]; and, undertaking research, analysis, and dissemination stages [13, 39, 49, 51, 52]. In eight studies [22, 38, 45, 53, 55, 59, 60, 62] children and young people were involved across all four stages.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. PPIE involvement in the stages of the research process.

https://doi.org/10.1371/journal.pone.0252774.g002

Why PPIE is taking place and what happens as a result

Of the studies which presented evaluation methods most reported relatively simple descriptions of the impact of PPIE based on informal evaluations and often insufficient detail was presented. However, one paper acknowledged that an embedded evaluation methodology was able to offer robustness and determination of real-world meaningfulness [48].

Limitations of involvement of children and young people in PPIE

Typically, the limitations of the involvement of children and young people in PPIE were reported as population issues such as recruitment and retention issues [19, 21, 37, 41, 45, 48, 50, 51] and non-representative samples/selection bias [21, 41, 45, 50, 51, 53, 54, 60, 61]. Constraints on time reduced the extent and depth that children and young people were able to be involved [13, 22, 39, 43, 51] as did resource and financial restrictions [22, 37, 43, 51, 60]. There was a need for more comprehensive training for researchers to support the involvement of children and young people [36, 52] and for the improvements to the design of activities to ensure accessibility and engagement with children and young people [55, 61]. Delays to studies can impact on young people’s on-going involvement in a study and diminish the positive impact of training and capacity building for children and young people [41]. Several studies acknowledged that the robust involvement and participation of children and young people must commence at the earliest stages of a study to ensure children and young people views are integrated into planning and design [39, 46, 54, 55].

Other limitations arose relating to managing group dynamics [48], ensuring meaningful engagement and accountability [41, 43, 46, 52] and either under or over-estimation by the researchers of the children and young people’s knowledge and skills [47, 52] and ability to commit to time-intensive activities [13, 52, 63]. In some studies, children and young people actively resisted the limited roles assigned to them and negotiated more active and involved roles [13, 42]. Other limitations reflect the need for researchers to be more flexible in response to young people’s personal circumstances and to ensure the voices of young people “less frequently heard” are able to play an active role in the process [38]. This closely aligns with ensuring that children and young people’s perspectives are not lost in translation during the research process [43, 61].

Which children and young people are reported as being involved in PPIE

Across the studies, children and young people aged 4–24 years were involved in health related PPIE. Only seven studies [19, 22, 37, 48, 54, 64, 65] had engaged with children under ten years old. A lack of clear reporting on additional demographic factors and limited reporting on health conditions make it difficult to comment on which children and young people were excluded from PPIE involvement. However, some studies did note the underrepresentation of ethnic minority groups [53, 54] and children and young people with severe or complex needs [45, 53, 54]. Wider literature on healthcare access and transition note the additional barriers faced by children and young people from ethnic minority groups [71–73] and with chronic conditions [74, 75], suggesting PPIE is currently falling short of the ambition of involving all children and young people in research to “improve health and wellbeing for themselves, their families and their communities” [1, 2].

While the majority of studies recruited children and young people from populations with direct experience of the research topic, a substantial minority [19, 38, 44, 48, 49, 51, 53, 56, 57, 59, 61, 62, 64, 65] recruited from existing PPIE youth groups. Members of these established PPIE groups were reported as having good engagement with [38], favourable views towards health services and high health and research literacy [61]. This aligns with previous research findings which report that children and young people who are involved in PPIE tend to be more outgoing, self-confident, critical and comfortable in formal education than their non-involved peers [76, 77]. The barriers to recruiting and retaining children and young people identified in our review included constrained research timeframes [22, 37, 43], lack of funding [22, 37, 43, 60], and limits to young people’s availability to attend [13, 21, 41, 51], may compound this lack of diverse representation. This suggests the limits set by research and budgetary timelines can lead to PPIE which does not fit in with the lives of children and young people, particularly if they are not involved in negotiating preferable and feasible activities and timelines in the early planning stages [77, 78].

The methods reported as being used to involve children and young people in PPIE

The reviewed studies involved children and young people across a variety of research designs including qualitative, mixed methods, RCTs and systematic reviews. PPIE involvement took place at all phases of the research project, including development (n = 29), undertaking and management (n = 66), analysis (n = 22) and dissemination (n = 19). This breadth of involvement resonates with findings from a previous scoping review [79] and confirms that is it is feasible to effectively involve children and young people in PPIE at every stage of the research process. However, only half of the studies (n = 20) had PPIE involvement across more than two of these phases suggesting that PPIE involvement is often either not sufficiently planned and/or sustained across the life cycle of research projects.

As noted in a previous scoping review of the involvement of children and young people in research design [80], much of the PPIE involvement reported exists on the lower levels of Hart’s ladder of participation [16], where young people are consulted and involved in adult initiated decisions but do not have opportunities for shared-decision making or initiating action [16]. Many of the reviewed studies noted that shorter-term, consultation based activities limited the opportunities for children and young people and researchers to build relationships [18, 39, 51] but that more meaningful and authentic engagement was achieved when young people were given or chose to adopt more active roles [13, 40, 41, 46, 48, 62, 63]. This concurs with previous reviews which found that more active PPIE involvement can lead to more positive impacts [79, 81]. However, we would argue as do others [69], that these hierarchical views of what constitutes ‘good’ PPIE can create fewer opportunities for flexible PPIE based on what young people would choose, want, and have the time and assets to participate in. This is not to say that expectations of what PPIE activities can be achieved should be lowered. One study in our review described children and young people negotiating a more active PPIE role by leading workshops, data collection and analysis [42], emphasising the importance of being guided by the desire of children and young people, if evident, for greater research involvement. However, as reported by several studies in our review, moving PPIE involvement towards more active roles for children and young people is complex and requires flexibility, planning [18, 41, 47, 54, 62] and training for both children and young people and researchers [45].

Why PPIE is taking place and what happens as a result

The impact of PPIE is a key aspect of the UK standards [4] and encourages researchers to identify and share the benefits, changes and learning gained from public insights. Despite increased commitment among researchers, funders and policymakers to demonstrate the positive impacts of PPIE involvement [1, 2, 82], the evidence base on the impacts of PPIE remains patchy and largely observational [80, 83]. It has been suggested that difficulties in assessing the impact of PPIE arise due to the complexity of PPIE [84] and that impact is highly contingent on context and the precise nature of involvement [85].

In agreement with a previous systematic review on the impacts of adult PPIE [81], our review suggests a wide range of positive PPIE impacts for children and young people, research, policy and the wider community across all phases of the research process. Evidence from the reviewed studies suggests that the involvement of children and young people help to refine research priorities, increase the accessibility and attractiveness of research methods and ensure children and young people’s perspectives are represented in analysis, outputs and dissemination. A range of positive individual (for example, new research skills or increased confidence) and interpersonal (for example, professional and community networks) impacts were reported for children and young people. Two studies suggested longer-term impacts including employment [35] and community advocacy roles [13]. However, in line with previous research [80, 83], the majority of this evidence on impacts was anecdotal, with only seven studies using formal methods to evaluate and evidence the impact of PPIE. While the GRIPP-2 reporting standards have the potential to improve the reporting of PPIE impact, this evidence base can only be developed through the inclusion of formal, planned evaluation activities during the early research development stages [83]. However, we would again argue against the hierarchical view of evaluation that prioritises statistical evidence of impact [86]. The diverse range of impacts noted in the reviewed studies suggests a context specific approach to impact evaluation should be taken [86] which consults with children and young people, to ensure that PPIE has positive impacts not only for researchers and research funders but also for the children and young people who have volunteered their time to participate [82].

Consistent with the search strategy which was focused on health research rather than health policy, only two studies had reported positive impacts of PPIE with children and young people on policy which is unsurprising as policy impacts tend to take longer to be realised and often fall outside of the timescales of a research project. The numerous life transitions experienced by children and young people such as moving schools, colleges and universities; transitioning from child to adult services; and age restrictions on organised groups can make it more difficult for researchers to sustain relationships and measure long term outcomes once their research has been concluded. Research funders should therefore consider how resources allocated for PPIE can be most effectively used, including allowing continued activity and follow-up beyond the time-restrictions of the research project.

Implications for practice

Our scoping review found that PPIE with children and young people is feasible across all phases of research for a range of ages, populations and contexts. Efforts should be made to prevent the exclusion of children and young people from minority ethnic groups, younger children, those with severe or complex need, lower socio-economic groups and other marginalised populations from PPIE. Smaller groups, varied opportunities for participation according to abilities and preferences, and community organisation partnership can allow more diverse groups of children and young people to be recruited and retained. Engagement with children and young people appears more meaningful and authentic when it moves beyond consultation to allow them opportunities for active involvement and shared decision making across the life course of the project.

PPIE is widely regarded as having positive impacts on children and young people and research, but evaluation of PPIE activity must be incorporated at the planning stage to ensure these impacts are evidenced.

Overall, our review found that reporting of PPIE activity with children and young people was variable and there was poor quality reporting of definitions, underpinning theory, and the evaluation of PPIE impact. Despite the existence of the GRIPP-2 quality reporting standards [80], uncertainty is evident around reporting of PPIE for children and young people, particularly as it is often not a requirement of research publications.

We would encourage researchers and practitioners to embrace a broader understanding of what effective, good quality PPIE can be in the context of their work with children and young people. Based on the findings of this scoping review we recommend a new, non-hierarchical perspective on PPIE for children and young people that encompasses, supports and values all elements of participation by acknowledging that a flexible, context specific approach to involvement and engagement, led by children and young people’s choice, is necessary.

PPIE with children and young people should be pragmatic, flexible, and suited to the context, setting and the focus of the study. It should be led by children and young people’s preferences, choices, abilities and interests and should respect their time, skills, and commitment. Consultation, engagement and participation can all offer children and young people worthwhile ways of contributing to research with the level, purpose and impact of involvement determined by the children and young people themselves.

Strengths and limitations of the scoping review

Although we used a wide range of search terms to capture the variation in PPIE terminology, inconsistencies in the keywords and terms used for PPIE means some studies may not have been included and this may have limited the findings. The lack of formal evaluation and the lack of standardised and clear reporting of PPIE led to difficulties in extracting relevant information from the papers and limits what can be reported in our review. Few journals require information about PPIE, which may lead to the under-reporting of PPIE activities. While the British Medical Journal (BMJ) has introduced the requirement for specific information about PPI, a recent study found that only 11% of studies published in the BMJ reported PPI activity [26].

Despite these limitations, our study has strengths such as our commitment to PPIE. Our search was strengthened by the involvement of a PPIE expert (RM) and a service user who helped us define our research questions, focus our data extraction and this input strengthened our findings. A further strength of our study was our commitment to overcome the lack of a suitable tool for appraising the quality of PPIE reporting. To address this, we have developed a simple tool for quality appraisal of PPIE research. This tool, the QRIPPAT (Quality of Reporting Involvement of Patients and the Public Appraisal Tool), was developed from a validated reporting checklist, the GRIPP2 [27], and has potential to be developed further and used to evaluate ca the quality of PPIE reporting across multiple studies.

Implications

PPIE with children and young people should be pragmatic, flexible and led by children and young people’s preferences, choices, abilities and interests and should respect their time, skills, and commitment.