Figures
Abstract
Background
The aim was to evaluate the readability of research information leaflets (RIL) for minors asked to participate in biomedical research studies and to assess the factors influencing this readability.
Methods and Findings
All the pediatric protocols from three French pediatric clinical research units were included (N = 104). Three criteria were used to evaluate readability: length of the text, Flesch's readability score and presence of illustrations. We compared the readability of RIL to texts specifically written for children (school textbooks, school exams or extracts from literary works). We assessed the effect of protocol characteristics on readability. The RIL had a median length of 608 words [350 words, 25th percentile; 1005 words, 75th percentile], corresponding to two pages. The readability of the RIL, with a median Flesch score of 40 [30; 47], was much poorer than that of pediatric reference texts, with a Flesch score of 67 [60; 73]. A small proportion of RIL (13/91; 14%) were illustrated. The RIL were longer (p<0.001), more readable (p<0.001) and more likely to be illustrated (p<0.009) for industrial than for institutional sponsors.
Citation: Ménoni V, Lucas N, Leforestier J-F, Doz F, Chatellier G, Jacqz-Aigain E, et al. (2011) Readability of the Written Study Information in Pediatric Research in France. PLoS ONE 6(4): e18484. https://doi.org/10.1371/journal.pone.0018484
Editor: Lisa Hartling, Alberta Research Centre for Health Evidence, University of Alberta, Canada
Received: November 2, 2010; Accepted: March 9, 2011; Published: April 6, 2011
Copyright: © 2011 Menoni et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was funded by the Assistance Publique des Hôpitaux de Paris as a Programme Hospitalier de Recherche Clinique. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
The participation of minors in clinical research protocols requires authorization from their legal guardians. However, this authorization cannot override the refusal of the child [1], [2]. The investigating pediatrician must therefore seek the child's voluntary cooperation in the research protocol, after providing the child with information appropriate for his or her level of development [3]. European regulations require pediatric patients to be provided with information about studies in which they are asked to participate, including their risks and benefits, in a language that the child is likely to understand [4]. The way in which information is delivered to a child for possible inclusion in a research protocol must be approved by the institutional review board.
Although there is no consensus regarding the use of a separate assent document for research, many institutional review boards require their use when presenting study information for children [5]. Therefore, if required, assent forms should be written and presented in a manner that optimizes understanding. Depending on the age of the child, the information supplied may be provided on an assent form written either exclusively for the child or for both parents and children [5], [6].
The information and consent forms for adults (patients or parents) asked to participate in clinical research studies have been evaluated by several researchers, using readability indices, such as that of Flesch [7]–[18]. Documents for adults are generally long (more than five pages) and of poor readability. Is this also the case for the research information leaflets (RIL) for children? There is no recent publication about the measure of the readability of children's written study information, only for children's health literacy [19]. Most studies in this domain have targeted parents. Only one article to date has reported the readability of written study information for minors [20]. Based on a single RIL, the authors showed that improvements in the readability of this document were accompanied by improvements in both the acceptance of the study and its understanding by children.
The aim of our study was to evaluate the readability of a large sample of pediatric RIL in clinical research and to assess the factors influencing this readability, to determine whether efforts are required to improve the readability of pediatric RIL in clinical research.
Methods
Collection of information documents
We collected all the pediatric research protocols from three public pediatric clinical research centers in France. All protocols had been authorized by the Comité de Protection des Personnes (institutional review board, IRB) between 2002 and 2009. For each protocol, we determined: the goal of the study (therapeutic or not), the type of sponsor (industrial or institutional), the year in which the IRB authorized the study, the field of medical research (oncology or other), whether a randomization procedure was used, whether the disease addressed by the protocol was potentially life-threatening, the phase level of the study (I, II, III or IV) and whether the protocol involved invasive tests (other than taking blood). Pediatric protocols including only children under the age of six years (corresponding to the age at which children learn to read in France) or unconscious children were excluded. RIL for children were collected and classified by age, if the ages of the children to be included were indicated in the inclusion criteria of the protocol or in the document itself. RIL were assigned to four categories on the basis of the age of the intended reader (figure 1): child (age between 6 and 11 years), adolescent (age between 12 and 17 years), child and adolescent (age unspecified, 6 to 17 years), and RIL written for parents, to allow them to communicate the necessary information to their children and including a specific space for the child to sign (common parents/child).
RIL: Research Information Leaflet.
Readability determination
We evaluated readability on the basis of three criteria: length of the text, Flesch readability score and the presence of illustrations [21]. Text length was determined by a word count. Flesch score [22] was calculated as follows: 206.835 - (1.015sl) – (0.846wl), where sl is sentence length (mean number of words per sentence) and wl is word length (mean number of syllables per word). The resulting score lies between 0 for texts that are not easily understood, and 100 for readily understandable texts. Scores between 60 and 70 are considered to be the standard reading range for the general population. This score can be calculated with Microsoft Word® software for texts written in English. For texts written in French, we have developed a Flesch score calculator, which is now freely available [21]. The presence of illustrations (pictures, diagrams or tables) was checked manually.
Comparison texts
Texts appropriate for readers of particular ages were used: extracts from textbooks written for children of 6 to 8 years of age or of 9 to 11 years of age, texts from French national examinations performed in 2000 to 2009 for the Brevet National des Collèges targeting 14-year-old adolescents, extracts from children's literature (Harry Potter, Alice in Wonderland, The Little Prince, Pinocchio, Snow White and the Seven Dwarfs and Babar).
RIL for both parents and children were used as comparison texts for RIL written for children, because these texts were mainly targeted at adults.
Data analysis
The RIL computer files were collected and analyzed with a PERL script that we had previously developed [21]. Statistical analyses were performed with NCSS® software. Continuous variables are presented as medians, with 25th and 75th percentiles. Fisher's exact test was used to compare categorical variables. Nonparametric Mann and Whitney or Kruskal–Wallis tests were used to compare continuous variables, and Tukey-Kramer tests were used for multiple comparisons. A p value <0.05 was considered significant.
Results
Protocol population
We included 104 pediatric protocols. Protocols including only patients who could not read, either because they were unconscious (N = 2) or because they were too young (under the age of six years, N = 10) were excluded. We also excluded protocols in which there was no written information for children (N = 20). For the remaining 72 protocols, we collected 91 RIL, which we then classified according to the age of the targeted reader (figure 1).
RIL characteristics
All the data concerning research information leaflet characteristics are included in table 1, other than the year in which IRB approval was obtained. Twenty-nine of the RIL (32%) were from oncological studies, whereas the other 62 RIL (68%) encompassed 19 different pediatric specialties: surgical specialties (n = 11, 18%) such as cardiac, orthopedic, ophthalmologic and dental surgery, for example and non surgical specialties (n = 51, 82%), such as infection biology, rheumatology, intensive care, nephrology, diabetology and nutrition for the most part.
Length of the text
The RIL analyzed comprised a median of 743 words [434; 1211], corresponding to three pages. RIL specifically written for children were significantly (p<0.001) shorter than those written for both parents and children: 608 words [353; 972] versus 1134 [913; 1423] (table 2).
An industrial sponsor was the only variable having a significant effect on the length of the RIL (word count). RIL coming from studies having an industrial sponsor were longer (P<0.001) than those from protocols having an institutional sponsor: 1257 [948; 2016] vs. 635 [355; 997]. The other variables studied (year of approval by the IRB, field of medical research, goal of study, phase level, presence of randomization, invasive tests and life-threatening condition) had no effect on RIL length.
Flesch readability score
The total Flesch score of the RIL analyzed was 35 [26; 45]. RIL written specifically for children were significantly (p<0.001) more readable than those written for both parents and children: 40 [30; 47] versus 25 [22; 28]. This readability was much lower than that of the texts usually read by children. Textbooks for children aged from six to eight years, books for children aged from 9 to 11 years and extracts from children's literature had Flesch scores of 68 [62; 77], 67 [61; 73] and 68 [55; 76], respectively. French national examination texts used to assess French children's performances in the Brevet National des Collèges (a national examination for children aged 14 to 15 years) had a Flesch score of 62 [55; 69]. Pediatric RIL were significantly (p<0.001) less readable than each of these categories of comparison texts (figure 2).
Two of the other variables studied were significantly associated with the Flesch score of RIL. RIL from protocols with an industrial sponsor were more readable (p<0.001) than RIL from protocols with an institutional sponsor: 46 [40; 52] vs 30 [25; 42]. RIL from phase I and phase II drug trial protocols were more readable (p<0.001) than RIL from phase III and phase IV drug trial protocols: 44 [32; 52] vs 31 [23; 43].
Presence of illustrations
Only a small proportion of the RIL analyzed (13/91; 14%) contained an illustration: a drawing to brighten up the document (either linked to the research or purely decorative), a diagram or a table explaining how the study would be carried out, a diagram or photograph of the procedure evaluated in the protocol (medical equipment, surgical procedure). With the exception of the examination papers, all the comparison texts were illustrated (54/64; 84%). RIL from protocols with industrial sponsors were more frequently illustrated than RIL from protocols with institutional sponsors (38% vs 9%; p = 0.009). None of the RIL from non therapeutic protocols was illustrated (p<0.001). Illustrations were present in 23% of the RIL from phase I and phase II trials, versus only 8% of RIL from phase III and IV trials (p = 0.25).
Discussion
Current regulations [1]–[6] require pediatric patients to be informed, but not necessarily with a written document. However, 78% of our sample of pediatric protocols included the provision of written study information specifically for children. This finding is consistent with those of Whittle et al. [6], who interviewed 188 chairpersons of IRBs and reported that 68% of them felt that pediatric information should also be delivered in writing. Similarly, Kimberly [5] showed, by analyzing the decisions of 55 IRBs concerning 69 pediatric protocols, that 83% of the protocols accepted included documentation destined for the child, often divided into two sequential age ranges, each with a different mode of documentation.
Our study is the first to evaluate the readability of RIL from various sponsors as a function of the type of sponsor, aim of the study and risks. The initial sample of 104 protocols was fairly representative of pediatric biomedical research in France. Indeed, on September 27th 2010, a search of the Clinicaltrials.gov site found 270 ongoing interventional pediatric protocols in France. Our study included 40% of these protocols.
Based on our three criteria — Flesch score, length of the text and presence of illustrations — the RIL readability was poorer than that of other French texts destined for children. Several publications have dealt with the readability score of health documents for English, but not with complete pediatric RIL. Most child health information was written at a level above that appropriate for tenth grade [19].
RIL for both parents and children and most of the RIL for children alone contained no illustrations. Only 13 illustrations were identified and they varied considerably in type, as indicated in the results section. We are not aware of any study demonstrating illustrations to be useful in themselves in information documents for pediatric clinical studies, but many texts for children include illustrations. Textbooks and children's literature, which contribute to teaching and education, contained many illustrations, highlighting their importance for a young readership. A good understanding of text often requires is the reader to be able to process elaboratively (i.e., to form vivid mental images of the events of the study). Houts et al. [23] showed that the use of pictographs significantly increases the understanding of medical information among patients with low literacy levels.
The widely used Flesch score evaluates the readability of a text as a function of the length of the words and sentences used and has been validated and extensively used for the evaluation of written information readability [8]–[18], [20]–[22]. It therefore facilitates the rapid, objective and quantitative analysis of the complexity of a text. Using it, we found that the texts we selected as comparison texts obtained much higher Flesch scores than RIL, demonstrating the sensitivity of this tool. This score does not reflect the level of patient understanding, because the understanding of any particular individual depends on intrinsic factors (for example, first language, culture, level of education, age etc.). RIL destined for both parents and children, as expected, were less readable than those destined solely for the child. The Flesch index values obtained were very low for information documents destined for both parents and children. The readability of these documents was equivalent to that of documents destined for adults only [21], and was very different from that for texts destined for minors, accounting for the large difference between the values obtained for this type of text and the other categories. There are various issues concerning the ethics of a form destined for both the adult and the child. In addition to the difficulties a child is likely to experience in reading a document written for adults, the information included and the manner of expressing that information should be different in a document intended for children, particularly for the youngest children.
The RIL from phase I and II trials (with an industrial or institutional sponsor) were more readable and more likely to be illustrated than the RIL from phase III and IV protocols. This better readability and presentation may be due to the particular context of phase I and II protocols, in which the evaluation of drug risks is often the main aim of the study and which often include children who have experienced treatment failure. A similar observation was recently made by Cheung et al. [24], during an analysis of the readability of informed consent forms for adults.
It is somewhat impractical in the clinical setting to provide multiple assent forms written to satisfy all ages and/or reading abilities. Tait et al. [20] demonstrated that a single modified assent form appeared to close the gap in understanding between the younger and older children. This suggests that use of a modified format will be important in providing younger children with developmentally appropriate information that can enhance their decision-making abilities.
The readability of written study information intended for children asked to participate in clinical research was uniformly poor, and much worse than that of the texts usually read by children. Study information is written and presented with little consideration for the literacy, cognitive abilities and preferences of children, and this may be an ethical issue. Researchers should routinely compute the reading ease of study information sheets and make greater efforts to improve the readability of written documents for potential participants. Both the investigators and the IRBs should check these documents to ensure that their readability is appropriate for the age of the children targeted. Flesch scores are thus a potentially useful tool for improving the readability of information documents. Additional work should focus on how best to present information to children so that they are able to choose how much information they require and investigators can learn how best to educate their young potential research subjects.
Acknowledgments
We thank Mrs. Sylvie Maral and Mrs. Christelle Conde for their assistance with the collection of the documents.
Author Contributions
Conceived and designed the experiments: VM NL GC J-MT HC. Performed the experiments: VM NL GC J-MT HC. Analyzed the data: VM NL GC J-MT HC. Contributed reagents/materials/analysis tools: VM NL GC J-MT HC J-FL. Wrote the paper: VM NL GC J-MT HC FD EJ-A CG.
References
- 1.
World Medical Association (2008) Declaration of Helsinki: ethical principles for research involving human subjects. http://www.wma.net/en/20activities/10ethics/10helsinki/index.html Accessed September 30, 2010.
- 2.
(1996) International Conference on Harmonisation of technical requirements for registration of pharmaceuticals for human use. Guideline for Good Clinical Practice. 1996. http://www.ich.org/cache/compo/276-254-1.html Accessed September 30, 2010.
- 3.
World Health Organization and CIOMS (1981) The WHO/CIOMS guidelines, proposed international guidelines for biomedical research involving human subjects. http://www.tbethics.org/documents.htm Accessed September 30, 2010.
- 4.
(2010) Directive 2001/20/EC of the European Parliament and of the Council of 4 April 2001 relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use. http://ec.europa.eu/enterprise/sectors/pharmaceuticals/documents/eudralex/index_en.htm Accessed September 30, 2010.
- 5. Kimberly MB, Hoehn KS, Feudtner C, Nelson RM, Schreiner M (2006) Variation in standards of research compensation and child assent practices: a comparison of 69 institutional review board approved informed permission and assent forms for 3 multicenter pediatric clinical trials. Pediatrics 117: 1706–11.
- 6. Whittle A, Shah S, Wilfond B, Gensler G, Wendler D (2004) Institutional review board practices regarding assent in pediatric research. Pediatrics 113: 1747–52.
- 7. Grundner TM (1980) On the readability of surgical consent forms. N Engl J Med 302: 900–2.
- 8. Morrow GR (1980) How readable are subject consent forms? JAMA 244: 56–58.
- 9. Tarnowski KJ, Allen DM, Mayhall C, Kelly PA (1990) Readability of pediatric biomedical research informed consent forms. Pediatrics 85: 58–62.
- 10. Hopper KD, TenHave TR, Hartzel J (1995) Informed consent forms for clinical and research imaging procedures: how much do patients understand? AJR Am J Roentgenol 164: 493–96.
- 11. Grossman SA, Piantadosi S, Covahey C (1994) Are informed consent forms that describe clinical oncology research protocols readable by most patients and their families? J Clin Oncol 12: 2211–15.
- 12. Philipson SJ, Doyle MA, Gabram SG, Nightingale C, Philipson EH (1995) Informed consent for research: a study to evaluate readability and processability to effect change. J Investig Med 43: 459–67.
- 13. White LJ, Jones JS, Felton CW, Pool LC (1996) Informed consent for medical research: common discrepancies and readability. Acad Emerg Med 3: 745–50.
- 14. Mader TJ, Playe SJ (1997) Emergency medicine research consent form readability assessment. Ann Emerg Med 29: 534–39.
- 15. Sharp SM (2004) Consent documents for oncology trials: does anybody read these things? Am J Clin Oncol 27: 570–75.
- 16. Paris A, Cracowski JL, Ravanel N, Cornu C, Gueyffier F, et al. (2005) [Readability of informed consent forms for subjects participating in biomedical research: updating is required]. Presse Med 34: 13–18.
- 17. Christopher PP, Foti ME, Roy-Bujnowski K, Appelbaum PS (2007) Consent form readability and educational levels of potential participants in mental health research. Psychiatr Serv 58: 227–32.
- 18. Paasche-Orlow MK, Taylor HA, Brancati FL (2003) Readability standards for informed-consent forms as compared with actual readability. N Engl J Med 348: 721–26.
- 19. Sanders LM, Federico S, Klass P, Abrams MA, Dreyer B (2009) Literacy and child health: a systematic review. Arch Pediatr Adolesc Med 163(2): 131–40.
- 20. Tait AR, Voepel-Lewis T, Malviya S (2007) Presenting research information to children: a tale of two methods. Anesth Analg 105: 358–64.
- 21. Ménoni V, Lucas N, Leforestier JF, Dimet J, Doz F, et al. (2010) The readability of information and consent forms in clinical research in France. PLoS One 11: 5(5).
- 22. Flesch R (1948) A new readability yardstick. J Appl Psychol 32: 221–33.
- 23. Houts PS, Witmer JT, Egeth HE, Loscalzo MJ, Zabora JR (2001) Using pictographs to enhance recall of spoken medical instructions II. Patient Educ Couns 43(3): 231–42.
- 24. Cheung WY, Pond GR, Heslegrave RJ, Enright K, Potanina L, et al. (2010) The contents and readability of informed consent Forms for oncology clinical trials. Am J Clin Oncol 33: 387–92.