http://orcid.org/0000-0003-2164-8802
Figures
Abstract
Background
By means of meta-analysis of information from all relevant epidemiologic studies, we examined the hypothesis that Schistosoma infection in school-aged children (SAC) is associated with educational loss and cognitive deficits.
Methodology/Principal findings
This review was prospectively registered in the PROSPERO database (CRD42016040052). Medline, Biosis, and Web of Science were searched for studies published before August 2016 that evaluated associations between Schistosoma infection and cognitive or educational outcomes. Cognitive function was defined in four domains—learning, memory, reaction time, and innate intelligence. Educational outcome measures were defined as attendance and scholastic achievement. Risk of bias (ROB) was evaluated using the Newcastle-Ottawa quality assessment scale. Standardized mean differences (SMD) and 95% confidence intervals (CI) were calculated to compare cognitive and educational measures for Schistosoma infected /not dewormed vs. uninfected/dewormed children. Sensitivity analyses by study design, ROB, and sequential exclusion of individual studies were implemented. Thirty studies from 14 countries, including 38,992 SAC between 5–19 years old, were identified. Compared to uninfected children and children dewormed with praziquantel, the presence of Schistosoma infection and/or non-dewormed status was associated with deficits in school attendance (SMD = -0.36, 95%CI: -0.60, -0.12), scholastic achievement (SMD = -0.58, 95%CI: -0.96, -0.20), learning (SMD = -0.39, 95%CI: -0.70, -0.09) and memory (SMD = -0.28, 95%CI: -0.52, -0.04) tests. By contrast, Schistosoma-infected/non-dewormed and uninfected/dewormed children were similar with respect to performance in tests of reaction time (SMD = -0.06, 95%CI: -0.42, 0.30) and intelligence (SMD = -0.25, 95%CI: -0.57, 0.06). Schistosoma infection-associated deficits in educational measures were robust among observational studies, but not among interventional studies. The significance of infection-associated deficits in scholastic achievement was sensitive to ROB. Schistosoma infection-related deficits in learning and memory tests were invariant by ROB and study design.
Author summary
Empirical evidence for cognitive or educational benefits of anti-Schistosoma treatment is currently uncertain, despite the recommended practice of wide-scale deworming with praziquantel. We addressed this knowledge gap by synthesizing information from 30 relevant epidemiologic studies reporting on 38,992 children between 5–19 years old from 14 countries. In those studies, Schistosoma infection or non-dewormed status was associated with educational loss and cognitive deficits. Specifically, there were small to moderate deficits in both school attendance and scholastic achievement. Similarly, Schistosoma infection or non-dewormed status was associated with deficits in learning and memory domains of psychometrically tested cognitive function. However, there was no evidence of Schistosoma infection- or non-deworming-associated deficits on tests of innate intelligence or reaction-time. Overall, compared to Schistosoma-uninfected or to dewormed children, the presence of Schistosoma infection or non-dewormed status was associated with educational, learning, and memory deficits in school-aged children. The combined evidence suggests that early treatment of children in Schistosoma-endemic regions could mitigate these deficits.
Citation: Ezeamama AE, Bustinduy AL, Nkwata AK, Martinez L, Pabalan N, Boivin MJ, et al. (2018) Cognitive deficits and educational loss in children with schistosome infection—A systematic review and meta-analysis. PLoS Negl Trop Dis 12(1): e0005524. https://doi.org/10.1371/journal.pntd.0005524
Editor: Amadou Garba, Ministère de la Santé Publique et de la Lutte contre les Endémies, NIGER
Received: October 15, 2016; Accepted: March 23, 2017; Published: January 12, 2018
Copyright: © 2018 Ezeamama et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This work was supported by Schistosomiasis Consortium for Operational Research and Evaluation (SCORE), University of Georgia, Athens, GA, USA. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared no competing interests exist.
Introduction
An estimated 800 million persons in tropical and sub-tropical countries are at risk of infection by one of three main human Schistosoma parasites–S. mansoni, S. haematobium, and S. japonicum [1]. As many as 240 million adults and children are actively infected [2–4] resulting in as much as 3.3 million disability-adjusted life years (DALYs) lost per annum due to overt and subclinical morbidities of Schistosoma infection [4, 5]. Sub-Saharan Africa is most affected; children from endemic areas are often infected by two years of age and many remain chronically infected throughout their school-age years [6–8]. Periodic mass drug administration (MDA) with praziquantel in school-aged children has been recommended for morbidity control by the World Health Organization [6]. However, Schistosoma-infected pre-school children are not routinely treated in such settings, and they constitute a potentially high risk group for accumulation of morbidity [9]. At present, there is no specific guidance for anti-Schistosoma drug treatment of preschool children, partly because of the lack of a child-friendly pediatric formulation [10].
Treatment with praziquantel has a demonstrated effectiveness in reducing infection intensity within individuals and in reducing the prevalence of infection within communities [11]. Such treatment results in clear-cut improvements with respect to advanced schistosomiasis-associated morbidities such as urinary tract fibrosis and hepatosplenic disease, including peri-portal fibrosis [12, 13]. Epidemiologic studies have associated Schistosoma infections with adverse impacts on anemia, growth [14], fitness [15], pediatric quality-of-life [16], and sub-optimal child development [17]. Definitively linking Schistosoma infections to these non-specific and sub-clinical morbidities is complicated in the context of poverty.[8, 12, 17, 18] However, plausible biologic mechanisms of these adverse impacts have been described [2, 19] and the likely underestimation of their morbidity-related health impact has been highlighted [3].
In helminth-endemic regions, the recommendation of periodic deworming of children is explained on the basis of its expected salutary impact on a range of child-health outcomes including anemia [12], nutritional status [14, 17, 18, 20], and overall well-being [16]. In addition, periodic deworming has been linked directly or indirectly to enhancement of school attendance and educational achievement among children enrolled in school [6, 14, 21, 22]. However, the empirical evidence-base for cognitive and educational benefits of deworming remains controversial [23–27]. Recent criticism of the supposed benefits of deworming for educational enhancement has emphasized the undue influence of a single study in evidence reviews [28], which may have led to over-optimistic appraisals regarding the potential health and poverty alleviation benefits of deworming programs [29, 30].
Recent reviews of MDA effects have, thus far, focused on the impact of soil-transmitted helminth infections (STH), but health policy discussions–including those at the World Health Organization [30, 31], have tended to generalize findings to all helminths. Because different parasites can have dramatically different effects in terms of organ-specific and systemic pathologies, it is important now to distinguish the impact and potential benefits of individual anti-helminthic therapies [32]. To date, the evidence base for cognitive or educational benefits of anti-Schistosoma treatment has not undergone systematic review. The present systematic review and meta-analysis addresses the following questions: a) among school-aged children examined in the context of cross-sectional or case-control studies, is Schistosoma infection associated with worse performance in neurocognitive tests or with educational loss? b) among school-aged children enrolled in prospective studies with specific treatment for Schistosoma infection, is lack of treatment with praziquantel associated with worse performance in neurocognitive tests or with educational loss? For our current meta-analysis, we hypothesize that non-treatment or infection with Schistosoma infection is associated with educational loss and cognitive deficits in school-aged children from schistosomiasis-endemic regions.
Methods
Search strategy
This review, with pertinent information regarding our review protocol, was prospectively registered with the PROSPERO database as follows: http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42016040052 (see Supporting Information file S1 Text). We searched Medline, Web of Science, and Biosis electronic databases for original research articles, conference abstracts, or dissertations available as of August 22, 2016. Databases were searched with pre-specified keywords including “bilharzia”, “schistosomiasis”, “Schistosoma”, “school attendance”, “attention”, “impairment”, “memory”, “cognition”, among others. The complete search strategy is detailed in S2 Text.
Population, inclusion and exclusion criteria, study design
This systematic review and meta-analysis is focused on school-aged children five years and older. We did not restrict studies according to language, design, or publication date. Both interventional and observational studies were included in this review if they evaluated cognitive function in school-aged children using any psychometric test, or measured school attendance or achievement in relation to infection by Schistosoma parasites of any species. We excluded studies exclusively focused on pre-school aged children because of the absence of educational measures and the use of neurocognitive tests (e.g., the Mullen test) that were difficult to classify in terms of neurocognitive domains. We excluded meta-analyses and primary studies of soil-transmitted helminth infections where Schistosoma coinfection was absent.
Comparisons: Schistosoma infection vs. no infection & treatment with praziquantel vs. placebo
Schistosoma infection status was determined by microscopic examination of stool or urine as appropriate for species. Praziquantel was the primary deworming agent in interventional studies. The primary exposure for this meta-analysis included presence of infection or, operationally, infection was categorically defined based on study design as follows: 1) untreated/placebo versus praziquantel-treated in a randomized controlled trial, 2) any, versus no Schistosoma infection in cross-sectional studies, or 3) pre-, versus post-praziquantel treatment, or infection-free versus persistent infection, among Schistosoma-infected individuals in a longitudinal design study. “Untreated” refers to children determined to be infected but not dewormed.
Outcomes: Psychometrically assessed cognitive function
In the reviewed studies, psychometrically-assessed cognitive function was measured by a range of instruments that, for the purposes of this meta-analysis, were categorized into four domains: memory, learning/executive function, attention/reaction time, and intelligence (see S1 Table for details). The memory domain instruments included tests of working (short-term) memory as well as those of long-term memory. Attention/reaction time tests were those that measured the ability of a child to sustain concentration on a particular object, action, or thought, including their capacity to manage competing demands in their environments. The learning/executive function domain included tests to evaluate children’s performance in goal-oriented behavior, particularly components that are important for scholastic advancement. These test a cluster of cognitive processes that enable children to connect past experience with present action, and by so doing, engage in planning, organizing, strategizing, paying of attention to details, and to emotionally self-regulate, make necessary efforts to remember important details required for attainment of future goals [29]. We included in the ‘intelligence’ domain psychometric tests of intelligence quotient (IQ) that likely measures largely biologically-determined cognitive abilities, in contrast to cognitive performance measures that are environmentally pliable [33].
It was common for studies to use a suite of psychometric instruments to assess a single or multiple cognitive domains in enrolled children. When multiple instruments were used to measure the same cognitive domain, a grand mean of scores and a grand mean of standard deviation (SD) across all instruments were calculated. Thus, for each publication, one overall mean and SD value was determined for each domain. A study could contribute data to different cognitive domains if it used tools spanning across several cognitive domains. However, each instrument only contributed to one single domain of function, as shown in S1 Table.
Outcomes: Educational loss
Two dimensions of educational loss were tabulated–school attendance and scholastic achievement for children enrolled in school.
School attendance: For children enrolled in cross-sectional and longitudinal studies, attendance rate was respectively defined as the number of days children attended school over the past month or over the study period. In case-control studies, the percentage of children enrolled vs. not enrolled in school was calculated for Schistosoma-infected and non-infected children.
School achievement was assessed across studies based on: i) children’s pass rate on standardized teacher-generated tests; ii) the percent of children who were in appropriate class for age; iii) their enrollment in elite vs. non-elite schools; iv) their scores in the school function domain of pediatric quality-of-life inventory; v) their change in class position after treatment for Schistosoma infection; vi) an above average vs. average/below average scholastic performance as rated by a teacher; or vii) their pass rate in any kind of educational test, whether teacher-administered or not.
Study selection, data extraction and management
Two researchers (LM and AK) independently screened individual articles by title and abstract, after which eighty-eight full text articles were assessed for eligibility for inclusion in this review. Studies were excluded on the following basis: no outcome measure reported (n = 39), non-primary literature or a review article (n = 7), absence of both infection and outcome measures (n = 6), no variation in exposure (n = 2), a limited meeting abstract duplicated by later full publication (n = 3) and a nonhuman study (n = 1) Disagreements between reviewers on inclusion of a given study were resolved by consensus between LM and AK. If no consensus was reached, the article was further evaluated by an additional reviewer (AEE). Thereafter, two researchers (AEE and NP) independently extracted relevant data for meta-analyses. Where differences in approach to standard error (SE) estimation were noted, discrepancy was resolved by consensus between AEE and NP. When a potentially relevant publication did not present needed information for meta-analysis, the authors were contacted to request additional data. If the dataset was publicly available, we obtained needed values directly [28].
Statistical analysis
The method for deriving SD from respective studies depended on how data were presented in the original reports. Some papers presented median (m) and range (a to b) instead of means and SD. These measures were converted into approximate mean and SD as follows: , see [34], where
and S2 refer to the values of mean and variance, respectively. Some studies reported mean of respective measures and 95% confidence intervals. For such studies, SDs were derived as follows: SD = sqrt(N)*(upper limit–lower limit)/3.92. Other studies presented data on means and their standard errors (SE), and SD was estimated as SD = SE*the square root of N, the study size. For studies presenting data on differences in mean scores between two time points for treated/infected vs. untreated/uninfected groups, appropriate SD for mean difference was calculated using the approach recommended by the Cochrane Collaboration [35].
For the meta-analysis, studies were grouped into six categories of outcomes used to measure cognitive or school-based function: school attendance, school achievement, memory, learning, IQ, or attention. Standardized mean difference (SMD) estimates and 95% confidence intervals (CI) were calculated for each test. SMD estimates were classified as robustly statistically different if their confidence intervals excluded zero. SMD estimates were interpreted based on thresholds described by Cohen [36], as follows: ‘trivial’ (< |0.20|), ‘small’ (≥| 0.20| to < |0.50|), ‘moderate’ (≥ |50| to < |0.80|) or ‘large’ (≥ |0.80|) effects, according to standard practice in social science research.
All analyses and plots were implemented in STATA, versions 11 or 12. Heterogeneity between studies was measured with Higgins’s and Thompson’s I2 statistic and chi-square p-values [37]. Where between-study heterogeneity was high, random effects modelling was used to estimate a pooled summary effect across studies [38, 39]. In the absence of heterogeneity, fixed effects modeling was performed [39]. Publication bias was assessed using the Egger test [40]. In sensitivity analysis, we evaluated potential heterogeneity in pooled impact estimates based on: i) observational vs. intervention study design; ii) the quality of original studies based on the Newcastle-Ottawa quality assessment scale; and iii) by Schistosoma species. For sensitivity analyses by species, we distinguished between urogenital schistosomiasis (S. haematobium), which is often obvious to affected children, and intestinal/hepatosplenic schistosomiasis (S. mansoni/S. japonicum). The latter two infections are similarly diagnosed by stool exam and infection is seldom obvious to most children. We examined the potential for overly influential publications using the ‘metabias’ function in STATA to evaluate robustness of our pooled estimate, based on sequential removal of individual publications from the calculation of summary estimates. Lastly, we evaluated the impact of year of publication on the stability of the pooled estimate by iteratively including studies based on year of publication–i.e. starting from the earliest to the most recent publication using the ‘metacum’ function in STATA.
Quality assessment
Our investigation was guided by recommendations of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) initiative and the Meta-analysis of Observational Studies in Epidemiology (MOOSE) guidelines for observational studies [41]. Quality ranking of each study was implemented using an adapted version of the Newcastle-Ottawa quality assessment scale (NOQAS) [42, 43] to derive a quality score for each investigation with respect to: i) representativeness of the infected population sample or the selection cases and controls (this yielded a score with range 0 to 3*); ii) comparability with respect to known correlates of cognitive function/educational attainment (score range 0 to 6*); iii) the absence of bias in relation to outcome assessment in prospective cohort studies (0 to 3*) or exposure assessment in cross-sectional and case-control studies (0 to 3*). We adapted the comparability segment of this scale to account for confounding effects of age, sex, nutrition, and socioeconomic status in the relation between Schistosoma infection and educational/cognitive outcomes. Comparability with respect to these factors was either achieved by design (i.e. age or sex restriction for observational studies, or randomization for RCTs), or analytically, via stratified analyses or multivariable adjustment in regression models. Scores were assigned for attainment of comparability with respect to these factors as follows: age (score = 1*), sex (score = 1*), nutritional status (score = 2*) and socioeconomic status (score = 2*). For each study, the initial raw quality score (max = 12*) was rescaled to match the scale of 9* and then classified as low, high, or very high risk of bias based on precedents in prior literature [42].
Results
Our database search yielded a total 2914 unique records. The screening of titles and abstracts resulted in the exclusion of 2846 records leaving 78 unique papers for full text review. We identified an additional set of ten relevant studies from the bibliographies of relevant articles. Of these 88 articles, 58 were excluded based on full text review for reasons pecified in Fig 1.
A total of 30 epidemiologic studies that assessed differences in cognitive test scores (based on psychometric tests) and/or educational status (measured as scholastic achievement or school attendance rate) in relation to Schistosoma infection or treatment were selected for inclusion (Table 1). Of these, 21 studies were cross-sectional [16, 44–56] or case-control [57–63] design where children were classified based on presence vs. absence of Schistosoma infection. Seven were longitudinal studies or pre-post intervention studies that featured screening and treatment for Schistosoma infections at the time of first assessment [28, 64–68]. In these studies, outcome contrasts were made according to: a) the duration of persistent infection vs. duration of an infection-free interval, and b) the number of children for whom intensity of infection at last follow-up remained lower vs. the number of children for whom there was no change from baseline infection status after treatment with praziquantel.
Two studies were randomized controlled studies. Only one included study was a classic placbo-controlled randomized-trial intervention in which children with Schistosoma infection were randomized to praziquantel vs. placebo/no treatment [69]. The other study randomized children to screening vs. non-screening for Schistosoma infections [70]. Children in the non-screened arm remained untreated (although that sample was subsequently tested for infection to distinguish infected from uninfected children). In that study, among children randomized to screening, those found to be Schistosoma-infected were given treatment, and we were thus able to derive differences in cognitive test scores between Schistosoma-infected/treated vs. Schistosoma-infected/not-treated children (Table 2). The median follow-up duration across the nine logitudinal studies was 12 months. Minimum follow-up duration was one month and maximum follow-up duration was 36 months. In four of the longitudinal studies, follow-up duration was 6 months or less. In another four studies, follow-up duration was more than 12 months. One study had a 12 month follow-up.
In all, a total of 38,992 children between the ages of 5 to 19 years from 14 countries in three continents–Africa, Asia and North America, were included in this review. The vast majority of studies were of children from Africa–Nigeria (n = 4), Egypt (n = 4), South Africa (n = 4), Tanzania (n = 3), Kenya (n = 4), Mali (n = 1), Côte D’Ivoire (n = 1), Zambia (n = 1); Ghana (n = 1) and Ethiopia (n = 1). Three studies were conducted in Southeast Asia or Asia [The Philippines (n = 2) and China (n = 1)], one study was conducted in St. Lucia (Caribbean), and another study was implemented in Madagascar.
A total of 36,626 children were studied in the context of cross-sectional or case control studies. Of these, cognitive test scores and/or indicators of educational loss were measured in 23,126 (59.3%) children infected with one of three schistosomiasis species (S. haematobium (20 studies), S. japonicum (3 studies), or S. mansoni (10 studies)). These infected children were compared to 13,835 (40.6%) children without Schistosoma infection. A total of 2,366 children were studied in the context of randomized intervention studies including praziquantel vs. placebo, or prospective treatment studies that included baseline and follow-up assessments of cognitive function (Table 2). Of the 30 studies included in this review, three (10%) and 16 (53.3%) were judged to be at very high or high risk of bias, respectively, per the NOQAS (Table 3).
Schistosoma infection and its associations with school attendance and educational attainment
Sixteen studies evaluated Schistosoma infection-related differences in school attendance (Table 2). Schistosoma infection-associated attendance deficits varied in magnitude and direction by study design. Specifically, we did not find any evidence of association between Schistosoma infection and school attendance among the two interventional studies (SMD = 0.03, 95%CI: -0.73, 0.78; Table 4); however, the observed infection-associated deficit in school-attendance was robust for the pooled estimate of 14 observational studies (SMD = -0.42, 95%CI: -0.70, -0.14). Compared to uninfected or praziquantel-treated children, the magnitude and direction of infection-associated deficit in school attendance was similar for children infected with S. haematobium or S. mansoni. Within strata of study quality, the association between infection and scholastic achievement was directionally consistent and statistically robust (Table 4). Across all studies, regardless of design or ROB, a deficit in school-attendance was evident for Schistosoma-infected or non-praziquantel-treated children compared to uninfected or praziquantel treated children (n = 15 studies; SMD = -0.36, 95% CI: -0.60, -0.12).
The overall finding of schistosomiasis-associated achievement deficit (n = 16 studies; SMD = -0.58, 95%CI: -0.96, -0.20) was directionally consistent but varied in magnitude by study design and study quality (Tables 3 and 4). Specifically, achievement deficit with Schistosoma infection was noted in the pooled estimate derived from four intervention studies, but the magnitude of this association was lower and not statistically robust (SMD = -0.35, 95%CI: -0.71, 0.01; Table 4). Compared to uninfected or PZQ-treated children, the infection or non-treatment associated deficit in scholastic attainment was directionally consistent across different Schistosoma species, although the magnitude of effect was higher for infection with S. haematobium (SMD = -0.62; 95% CI:-1.09, -0.14) than for infection with S. mansoni (SMD = -0.22; 95% CI:-0.40, -0.05). Among observational study designs, scholastic achievement deficit was statistically robust and of larger magnitude (n = 12 studies; SMD = -0.65, 95% CI: -1.12, -0.17, Table 4). Similarly, Schistosoma infection was not associated with deficits in scholastic achievement among studies identified as low risk of bias (n = 6 studies; SMD = -0.08, 95%CI: -0.21, 0.02; Table 4). The estimates of infection-related deficit in scholastic achievement increased when study quality was lower: for studies with high risk of bias (n = 7) the SMD = -0.84 (95% CI:-1.52, -0.16); for studies with very high risk of bias (n = 7) SMD = -0.92 95% CI: -2.1, 0.28) although the estimated difference was statistically imprecise (i.e., with a wider CI) among the studies with greatest risk of bias.
Impact of Schistosoma infection on psychometrically evaluated cognitive domains
We found Schistosoma infection-associated deficits in memory tests (n = 8 studies; SMD = -0.28, 95% CI: -0.52, -0.04; Table 2). Similarly, Schistosoma infection was associated with small-to-moderate deficits in learning tests (n = 6 studies; SMD = -0.39, 95%CI: -0.70, -0.09; Table 2). The Schistosoma infection-related deficits in memory tests were directionally consistent by study design; although separate pooled estimates for interventional studies (n = 4 studies; SMD = -0.36, 95% CI: -0.61, 0.09) and observational studies (n = 4 studies; SMD = -0.15, 95% CI: -0.34, 0.03) were not statistically robust. For Schistosoma infection-related association with learning, pooled estimates suggested the presence of deficits for infected/non-dewormed vs. uninfected/praziquantel-treated children (n = 6 studies; SMD = -0.39, 95% CI: -0.70, -0.09). However, the magnitude of this association differed according to study design. Here, larger pooled standardized differences were realized for interventional studies (n = 2 studies; SMD = -0.79, 95% CI:-1.19, -0.39) than for observational studies (n = 4 studies; SMD = -0.18, 95% CI: -0.35, -0.01). Schistosoma infection was not significantly associated with performance on tests of reaction time (n = 6 studies; SMD = -0.06, 95% CI: -0.42, 0.30) or performance in tests of innate intelligence (n = 4 studies; SMD = -0.25, 95% CI: -0.57, 0.06). For all psychometrically-assessed cognitive domains, the overall findings were not sensitive to study quality. Of note, the majority of publications with psychometrically-evaluated endpoints had low risk of bias and produced pooled estimates similar in magnitude and direction to the overall results (Table 4).
Sensitivity analyses
High levels of heterogeneity were observed across included studies (P ≤ 0.03, I2 ≥ 74.8%; Tables 2 and 4) for all outcome measures, but there was no evidence of undue influence by individual studies or of publication bias among the included studies (Egger’s test, all P-value ≥ 0.142, Table 2). Overall, inferences based on pooled estimates were generally insensitive to differences in study design (Table 4), to the exclusion of individual studies (S2 Table), or to the influence of publication year (S1 Fig).
Discussion
Principal findings and interpretations
This systematic review and meta-analysis of the cognitive and educational impact of Schistosoma infection in school-aged children supports the hypothesis that infection is associated with reduced school-attendance, with deficits in scholastic achievement and deficits in memory and learning domains of psychometrically evaluated cognitive function. It has previously been conjectured that Schistosoma infection may affect school attendance, scholastic achievement, and cognitive function, either directly through via deposition of Schistosoma eggs within the central nervous system, via physical discomfort and subsequent distraction due to the presence of the worms, or indirectly, via iron-deficiency and malnutrition [3, 71, 72]. However, Schistosoma infection or non-treatment was not associated with performance in tests of innate intelligence or reaction time. Inferences based on most pooled estimates for psychometrically assessed endpoints were generally insensitive to study design, Schistosoma species, and risk of study bias. However, associations between infection and educational outcomes were sensitive to study design and study quality–especially estimates for impact on scholastic achievement. The association between infection and scholastic achievement was directionally similar and statistically robust regardless of Schistosoma species; however, average effects were substantially larger for S. haematobium compared to S. mansoni infection. Cohen’s criteria for effect size suggest that the average Schistosoma infection-related deficits in education, learning, and memory performance range from ‘small’ to ‘moderate’. S. haematobium infection was associated with relatively larger deficits in scholastic achievement.
Prior meta-analysis of four randomized controlled trials that evaluated cognitive impacts of STH infections–which sometimes co-occur with schistosomiasis, have reached a different conclusion about the cognitive and scholastic effects of STH infection and the impact of interval treatments for STH [25, 26]. Those reviews concluded that there was substantial evidence that deworming for soil-transmitted helminth infections does not yield a cognitive or educational benefit. We note that our approach differed from these prior STH-based meta-analyses on several grounds: a) we included both interventional and observational studies to take advantage of all research data available on this question, b) we evaluated Schistosoma-associated impacts on two domains of educational loss (attendance and achievement), c) we further defined domains of cognitive function based on psychometrically-assessed testing to include the following: learning, memory, attention, and intelligence and d) the intervention, as defined in this meta-analysis, denoted treatment for Schistosoma infection, whether or not the study was randomized.
We intentionally included data from all available epidemiologic studies–whether interventional or observational in design, in this first systematic review and meta-analysis of Schistosoma infection-related differences in cognitive and educational outcomes. The inclusion of all available evidence reflects current standards for clinical evidence-gathering to inform health policy, in order to shape clinical practice based on the ‘best available’ relevant information [23, 24]. Future randomized-controlled trials to address this question are expected to be limited in scope and may be considered ethically objectionable given the current widespread adoption of deworming for schistosomiasis and STH. The current adoption of ‘preventive chemotherapy’ guidelines has been based on helminth-associated adverse effects on anemia and child growth. In consequence, meta-analysis of available evidence, as performed in the present study, remains the most practical strategy to inform current policy. By this approach, we have identified ‘small to moderate’ infection-related deficits in education, learning, and memory performance using the Cohen’s criteria of effect size. However, such numerically small deficits (per Cohen’s criteria) may significantly underestimate clinical significance of infection for childhood development, as Schistosoma infection is an exposure affecting millions of children in endemic regions. Hence, small to moderate deficits at the individual level may amount to large and important differences in disease burden at the population level [73, 74]. Our interpretation of SMD estimates is ultimately grounded in the importance of summary measures for clarifying existing knowledge gaps regarding the relationship of Schistosoma infections to respective outcomes, and the expected benefit of systematically lowering infection-related deficits in the millions of children at risk.
As a limitation of our approach, we acknowledge the possibility of residual confounding and bias in the primary literature, given that majority of included studies were observational or non-randomized intervention trials. Only two of the 30 included studies used an RCT design, thus a sensitivity analysis based on RCT vs. non-RCT study design was not possible. However, as part of sensitivity analysis we evaluated the potential for differences in pooled estimates based on our expanded definition of intervention as including longitudinal studies that included praziquantel treatment. Investigation of pooled estimate sensitivity by region of study, the year of publication, and by risk of bias did not result in materially different statistical inferences. Specific investigation of publication bias suggests that any possible greater likelihood of publishing positive studies did not unduly influence observed findings. Our approach of including both interventional and non-interventional studies is consistent with theoretical and empirical evidence that meta-analyses based on observational studies generally produce estimates of effect similar to those from meta-analyses based on randomized controlled trials, and that a priori exclusion of observational studies in systematic reviews is inappropriate and inconsistent with the evidence-based medical decision-making approach [75, 76]. In addition, the often restrictive inclusion criteria and short follow-up duration in RCTs could easily result in outcomes largely different from when the same interventions are applied to a general population.
Despite the intuitive appeal of our cognitive domain based evaluation, we acknowledge the critique that our classification of psychometric instruments by domain required some level of subjectivity, especially for tools that capture performance across multiple domains. We explicitly identified the instruments used in each study and, based on literature description of the major cognitive domain assessed by each tool, combined related tools into four separate domains. Ultimately, each instrument was assigned to one cognitive domain only. We have described the logic for our choices in the supplementary information (S1 Table) to provide a basis for further discussion and give sufficient context for critical evaluation of our approach in developing future studies.
Our decision to combine psychometric evaluations of cognitive functions in four domains (based on the primary capacity being tested) is a strength of our empirical approach. By so doing, we recognize that Schistosoma infection may not have equal impact on all cognitive domains. For example, to the extent that innate intelligence is strongly influenced by fixed or heritable factors, we did not anticipate infection related differences on tests of ‘intelligence quotient’. Unlike intelligence tests, we considered the other tests of memory, learning, and reaction time to be more sensitive to infection, and thus modifiable by presence/non-treatment vs. absence/treatment of Schistosoma infection. Unexpectedly, reaction time was not associated with Schistosoma infections. However, our findings of infection-related reductions in learning and memory tests are consistent with our hypotheses that these cognitive domains are sensitive to adverse environmental perturbations–including Schistosoma infection.
Remaining gaps, and recommendations for future research
The finding of infection-related cognitive deficits and educational loss reported here is clinically and health policy-relevant for mitigating the cognitive and functional morbidities of Schistosoma spp. infection in children. The ‘small-to-moderate’ effects demonstrated may, in reality, be an underestimate of the lifetime impact of Schistosoma infection on personal performance (as affected via ultimately irreversible educational and cognitive losses). Typical epidemiologic studies necessarily include only a constrained portion of the relevant etiologic period. Among school-age children, Schistosoma infection is often effectively already chronic and/or recurrent, with reinfection rates extremely high in the absence of meaningful environmental interventions to reduce re-infection following treatment. The cumulative cognitive and educational impact of persistent infection may not be adequately captured by the relatively short-term investigations of treatment impact included in this meta-analysis.
In schistosomiasis-endemic regions, many children are infected by age two and remain chronically infected through school-age and late adolescence [9]. Under current national Schistosoma control treatment guidelines, preschool-age children are not treated as part of routine deworming programs for STH or schistosomiasis [1, 6, 30]. These children may therefore suffer cumulative damage to their health and function that is currently not reflected in most short-term study outcomes (or this meta-analysis). Of note, recent investigations have demonstrated the safety and efficacy of praziquantel for treatment of Schistosoma infection in preschool children [9]. The existence of an adverse developmental impact of Schistosoma infection on cognitive/educational domains would be a major justification for expanding the age-bracket of children who should be treated with praziquantel. Currently, evidence suggests that the timing of infection across a life path is especially consequential in terms of the severity of cognitive and physiologic impairments experienced [77, 78]. Future investigations evaluating the relative differences in cognitive outcomes for pre-school children with and without Schistosoma infection will be important for understanding the magnitude of potential impact of better prevention of Schistosoma infection.
It is currently unknown whether the cognitive and educational loss associated with Schistosoma infection can be reversed with treatment alone. The fact that infection often occurs in the context of malnutrition, coincident parasitic infections, and extreme poverty suggests that cognitive remediation efforts will need to be multi-faceted, using an integrated disease management framework. We expect that educational and cognitive interventions will be most effective if initiated earlier in life and that the package of interventions may need to include remedial instruction, the prevention of reinfection for treated/cured children, management of comorbid health conditions, and interventions for improvement of nutritional status.
Policy implications
Our investigation suggests that Schistosoma infection/non-treatment is associated with educational and cognitive loss. Our findings further suggest a definite cognitive and educational benefit of anti-schistosomal deworming among school-age children. Future complex intervention studies of early childhood interventions, focused on improving child well-being and cognitive potential, are needed to determine to what extent these observed deficits are preventable or reversible. Interventions that employ an integrated disease management framework will likely identify cost-efficiencies for leveraging existing disease and nutrition treatment programs in helminth-affected regions.
Supporting information
S1 Text. PROSPERO protocol CRD42016040052 registered for this study.
https://doi.org/10.1371/journal.pntd.0005524.s001
(PDF)
S2 Text. Schematic of search strategy for the systematic review.
https://doi.org/10.1371/journal.pntd.0005524.s002
(DOCX)
S1 Table. Classification of individual study assessments into domains based on psychometric evaluation of cognitive function & education-related assessments.
https://doi.org/10.1371/journal.pntd.0005524.s004
(DOCX)
S2 Table. The impact of Schistosoma infection/non-treatment on educational loss, memory, and learning domains with serial exclusion of individual studies–a sensitivity analysis quantifying the influence of individual studies.
https://doi.org/10.1371/journal.pntd.0005524.s005
(DOCX)
S1 Fig. Sensitivity analysis of the impact of publication year on pooled estimates for educational and cognitive loss domains.
https://doi.org/10.1371/journal.pntd.0005524.s007
(DOCX)
Acknowledgments
The authors acknowledge the invaluable contribution of Dr. Daniel Colley from the University of Georgia, Department of Microbiology and the Center for Tropical and Emerging Global Diseases in reviewing early drafts of this work.
References
- 1.
WHO, Prevention and control of schistosomiasis and soil-transmitted helminthiasis. WHO Technical Report Series, 2002(912).
- 2. Colley D.G., et al., Human schistosomiasis. Lancet, 2014. 383(9936): p. 2253–64. pmid:24698483
- 3. King C.H., Dickman K., and Tisch D.J., Reassessment of the cost of chronic helmintic infection: a meta-analysis of disability-related outcomes in endemic schistosomiasis. Lancet, 2005. 365(9470): p. 1561–9. pmid:15866310
- 4. Steinmann P., et al., Schistosomiasis and water resources development: systematic review, meta-analysis, and estimates of people at risk. Lancet Infect Dis, 2006. 6(7): p. 411–25. pmid:16790382
- 5. Murray C.J., et al., Disability-adjusted life years (DALYs) for 291 diseases and injuries in 21 regions, 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet, 2012. 380(9859): p. 2197–223. pmid:23245608
- 6. Colley D.G., Morbidity control of schistosomiasis by mass drug administration: how can we do it best and what will it take to move on to elimination? Trop Med Health, 2014. 42(2 Suppl): p. 25–32. pmid:25425948
- 7. King C.H., Parasites and poverty: the case of schistosomiasis. Acta Trop, 2010. 113(2): p. 95–104. pmid:19962954
- 8. van der Werf M.J., et al., Quantification of clinical morbidity associated with schistosomiasis infection in sub-Saharan Africa. Acta Trop, 2003. 86(2–3): p. 125–39. pmid:12745133
- 9. Nalugwa A., et al., Single versus double dose praziquantel comparison on efficacy and schistosoma mansoni re-Infection in preschool-age children in Uganda: A randomized controlled trial. PLoS Negl Trop Dis, 2015. 9(5): p. e0003796. pmid:26011733
- 10. Stothard J.R., et al., Schistosomiasis in African infants and preschool children: let them now be treated! Trends Parasitol, 2013. 29(4): p. 197–205. pmid:23465781
- 11. Erko B., et al., Efficacy and side effects of praziquantel in the treatment of schistosomiasis mansoni in schoolchildren in Shesha Kekele Elementary School, Wondo Genet, Southern Ethiopia. Asian Pac J Trop Biomed, 2012. 2(3): p. 235–9. pmid:23569905
- 12. Friedman J.F., Kanzaria H.K., and McGarvey S.T., Human schistosomiasis and anemia: the relationship and potential mechanisms. Trends Parasitol, 2005. 21(8): p. 386–92. pmid:15967725
- 13. Andrade G., et al., Decline in infection-related morbidities following drug-mediated reductions in the intensity of Schistosoma infection: A systematic review and meta-analysis. PLoS Negl Trop Dis, 2017. 11(2): p. e0005372. pmid:28212414
- 14. Coutinho H.M., et al., Nutritional status improves after treatment of schistosoma japonicum-infected children and adolescents. J Nutr, 2006. 136(1): p. 183–8. pmid:16365080
- 15. Bustinduy A.L., et al., Measuring fitness of Kenyan children with polyparasitic infections using the 20-meter shuttle run test as a morbidity metric. Plos Neglected Tropical Diseases, 2011. 5(7): p. e1213. pmid:21750742
- 16. Terer C.C., et al., Evaluation of the Health-related Quality of Life of children in Schistosoma haematobium-endemic communities in Kenya: A cross-sectional study. Plos Neglected Tropical Diseases, 2013. 7(3): p. e2106. pmid:23505590
- 17. King C.H. and Dangerfield-Cha M., The unacknowledged impact of chronic schistosomiasis. Chronic Illn, 2008. 4(1): p. 65–79. pmid:18322031
- 18. Olveda D.U., et al., Bilharzia in the Philippines: past, present, and future. Int J Infect Dis, 2014. 18: p. 52–6. pmid:24211228
- 19. Asahi H. and Stadecker M.J., Analysis of egg antigens inducing hepatic lesions in schistosomiasis infection. Parasitol Int, 2003. 52(4): p. 361–7. pmid:14665394
- 20. Papier K., et al., Childhood malnutrition and parasitic helminth interactions. Clin Infect Dis, 2014. 59(2): p. 234–43. pmid:24704723
- 21. Brooker S., et al., Epidemiology of single and multiple species of helminth infections among school children in Busia District, Kenya. East Afr Med J, 2000. 77(3): p. 157–61. pmid:12858892
- 22. Bundy D.A., et al., Deworming and development: asking the right questions, asking the questions right. PLoS Negl Trop Dis, 2009. 3(1): p. e362. pmid:19172186
- 23. Davey C., et al., Re-analysis of health and educational impacts of a school-based deworming programme in western Kenya: a statistical replication of a cluster quasi-randomized stepped-wedge trial. Int J Epidemiol, 2015. 44(5): p. 1581–92. pmid:26203171
- 24. Hicks J.H., Kremer M., and Miguel E., Commentary: Deworming externalities and schooling impacts in Kenya: a comment on Aiken et al. (2015) and Davey et al. (2015). Int J Epidemiol, 2015. 44(5): p. 1593–6. pmid:26203170
- 25. Taylor-Robinson D.C., Jones A.P., and Garner P., Deworming drugs for treating soil-transmitted intestinal worms in children: effects on growth and school performance. Cochrane Database Syst Rev, 2007(4): p. CD000371. pmid:17943740
- 26. Taylor-Robinson D.C., et al., Deworming drugs for soil-transmitted intestinal worms in children: effects on nutritional indicators, haemoglobin and school performance. Cochrane Database Syst Rev, 2012. 7: p. CD000371.
- 27. Taylor-Robinson D.C., et al., Deworming drugs for soil-transmitted intestinal worms in children: effects on nutritional indicators, haemoglobin, and school performance. Cochrane Database Syst Rev, 2015(7): p. CD000371. pmid:26202783
- 28. Miguel E. and Kremer M., Worms: Identifying impacts on education and health in the presence of treatment externalities. Econometrica, 2004. 72(1): p. 159–217.
- 29. Blair C., School readiness. Integrating cognition and emotion in a neurobiological conceptualization of children's functioning at school entry. Am Psychol, 2002. 57(2): p. 111–27. pmid:11899554
- 30.
WHO, Strategy Development and Monitoring for Parasitic Diseases and Vector Control Team. Deworming: The Millennium Development Goals. The evidence is in: deworming helps meet the Millennium Development Goals. 2005, World Health Organization: Geneva.
- 31. Hotez P.J., New antipoverty drugs, vaccines, and diagnostics: a research agenda for the US President's Global Health Initiative (GHI). PLoS Negl Trop Dis, 2011. 5(5): p. e1133. pmid:21655348
- 32. Bustinduy A.L., et al., Population Pharmacokinetics and Pharmacodynamics of Praziquantel in Ugandan Children with Intestinal Schistosomiasis: Higher Dosages Are Required for Maximal Efficacy. MBio, 2016. 7(4).
- 33. Deary I.J., Differences in mental abilities. BMJ, 1998. 317(7174): p. 1701–3. pmid:9857133
- 34. Hozo S.P., Djulbegovic B., and Hozo I., Estimating the mean and variance from the median, range, and the size of a sample. BMC Med Res Methodol, 2005. 5: p. 13. pmid:15840177
- 35.
Imputing standard deviations for changes from baseline. [December 20, 2017]; Available from: http://handbook-5-1.cochrane.org/chapter_16/16_1_3_2_imputing_standard_deviations_for_changes_from_baseline.htm.
- 36.
Cohen J., Statistical power analysis for the behavioural sciences. rev. ed. 1977, New York: Academic Press.
- 37. Higgins J.P. and Thompson S.G., Quantifying heterogeneity in a meta-analysis. Stat Med, 2002. 21(11): p. 1539–58. pmid:12111919
- 38. Berkey C.S., et al., A random-effects regression model for meta-analysis. Stat Med, 1995. 14(4): p. 395–411. pmid:7746979
- 39. Schmidt F.L., Oh I.S., and Hayes T.L., Fixed- versus random-effects models in meta-analysis: model properties and an empirical comparison of differences in results. Br J Math Stat Psychol, 2009. 62(Pt 1): p. 97–128. pmid:18001516
- 40. Sterne J.A., Gavaghan D., and Egger M., Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol, 2000. 53(11): p. 1119–29. pmid:11106885
- 41. Stroup D.F., et al., Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA, 2000. 283(15): p. 2008–12. pmid:10789670
- 42. Lo C.K., Mertz D., and Loeb M., Newcastle-Ottawa Scale: comparing reviewers' to authors' assessments. BMC Med Res Methodol, 2014. 14: p. 45. pmid:24690082
- 43. Stang A., Critical evaluation of the Newcastle-Ottawa scale for the assessment of the quality of nonrandomized studies in meta-analyses. Eur J Epidemiol, 2010. 25(9): p. 603–5. pmid:20652370
- 44. Berhe N., Myrvang B., and Gundersen S.G., Gastro-intestinal symptoms associated with intense Schistosoma mansoni infection affect class-attentiveness of schoolchildren in Ethiopia. Acta Tropica, 2009. 110(1): p. 52–56. pmid:19283896
- 45. Castle W.M., Clarke V.D.V., and Hendrikz E., Effects of subclinical bilharziasis on mental-ability in schoolchildren. South African Medical Journal, 1974. 48(48): p. 2035–2038.
- 46. de Clercq D., et al., The relationship between Schistosoma haematobium infection and school performance and attendance in Bamako, Mali. Ann Trop Med Parasitol, 1998. 92(8): p. 851–8. pmid:10396345
- 47. Ejezie G.C. and Ade-Serrano M.A., Schistosoma haematobium in Ajara community of Badagry, Nigeria. A study on prevalence, intensity and morbidity from infection among primary school children. Trop Geogr Med, 1981. 33(2): p. 175–80. pmid:6974422
- 48. Ekanem E.E., et al., Effect of Schistosoma haematobium infection on the physical growth and school performance of Nigerian children. Cent Afr J Med, 1994. 40(2): p. 38–44. pmid:8082154
- 49. Epstein E.H. and Weisbrod B.A., Parasitic diseases and academic performance of schoolchildren. Social and Economic Studies, 1974. 23(4): p. 551–570.
- 50. Ezeamama A.E., et al., Helminth infection and cognitive impairment among Filipino children. American Journal of Tropical Medicine and Hygiene, 2005. 72(5): p. 540–548. pmid:15891127
- 51. Goldin D. and Barclay R., Schistosomiasis in Rural Zambia. Annals of Tropical Medicine and Parasitology, 1972. 66(2): p. 193–6. pmid:5038243
- 52. Haycock D.C. and Schutte C.H.J., Schistosoma-haematobium infection and scholastic attainment amongst black schoolchildren. South African Journal of Science, 1983. 79(9): p. 370–373.
- 53. Husein M.H., et al., Who misses out with school-based health programmes? A study of schistosomiasis control in Egypt. Transactions of the Royal Society of Tropical Medicine and Hygiene, 1996. 90(4): p. 362–365. pmid:8882176
- 54. Jukes M.C., et al., Heavy schistosomiasis associated with poor short-term memory and slower reaction times in Tanzanian schoolchildren. Trop Med Int Health, 2002. 7(2): p. 104–17. pmid:11841700
- 55. Rasoamanamihaja C.F., et al., Baseline prevalence and intensity of schistosomiasis at sentinel sites in Madagascar: Informing a national control strategy. Parasit Vectors, 2016. 9: p. 50. pmid:26822783
- 56. El-Hawey A.M., et al., Impacts of schistosomiasis on scholastic achievement of school children in two Egyptian rural communities. Egypt J Comm Med., 1990. 7: p. 27–39.
- 57. Beasley N.M., et al., The health of enrolled and non enrolled children of school age in Tanga, Tanzania. Acta Trop, 2000. 76(3): p. 223–9. pmid:10974162
- 58. Fentiman A., Hall A., and Bundy D., Health and cultural factors associated with enrolment in basic education: a study in rural Ghana. Soc Sci Med, 2001. 52(3): p. 429–39. pmid:11330777
- 59. Loveridge F.G., Ross W.F., and Blair D.M., Schistosomiasis; the effect of the disease on educational attainment. S Afr Med J, 1948. 22(7): p. 260–3. pmid:18862893
- 60. Mekheimar S.I. and Talaat M., School non-enrollment and its relation with health and schistosomiasis knowledge, attitudes and practices in rural Egypt. East Mediterr Health J, 2005. 11(3): p. 392–401. pmid:16602459
- 61. Nazel M.W., et al., Schistosoma mansoni infection and cognitive functions of primary school children, in Kafr El Sheikh, Egypt. J Egypt Public Health Assoc, 1999. 74(1–2): p. 97–119. pmid:17216955
- 62. Tiruneh M., et al., Schistosomiasis mansoni in school attenders and non-attenders in Northwest Ethiopia. Ethiop J. Health Dev., 2001. 15(2): p. 117–123.
- 63. Useh M.F. and Ejezie G.C., School-based schistosomiasis control programmes: a comparative study on the prevalence and intensity of urinary schistosomiasis among Nigerian school-age children in and out of school. Transactions of the Royal Society of Tropical Medicine and Hygiene, 1999. 93(4): p. 387–391. pmid:10674083
- 64. Ezeamama A.E., et al., Treatment for Schistosoma japonicum, reduction of intestinal parasite load, and cognitive test score improvements in school-aged children. Plos Neglected Tropical Diseases, 2012. 6(5): p. e1634. pmid:22563514
- 65. Hurlimann E., et al., Effect of deworming on school-aged children's physical fitness, cognition and clinical parameters in a malaria-helminth co-endemic area of Cote d'Ivoire. BMC Infect Dis, 2014. 14: p. 411. pmid:25060173
- 66. Jordan P. and Randall K., Bilharziasis in Tanganyika—Observations on Its effects and effects of treatment in schoolchildren. Journal of Tropical Medicine and Hygiene, 1962. 65(1): p. 1–6.
- 67. Kimura E., et al., Effects of Schistosoma haematobium infection on mental test scores of Kenyan school children. Trop Med Parasitol, 1992. 43(3): p. 155–8. pmid:1470832
- 68. Meremikwu M.M., et al., Treatment of schistosomiasis haematobium with praziquantel in children: Its effect on educational performance in rural Nigeria. Trop Med, 2000. 42(1): p. 39–45.
- 69. Nokes C., et al., Evidence for an improvement in cognitive function following treatment of Schistosoma japonicum infection in Chinese primary schoolchildren. Am J Trop Med Hyg, 1999. 60(4): p. 556–65. pmid:10348228
- 70. Grigorenko E.L., Sternberg R.J., Jukes M., Alcock K., Lambo J., Ngorosho D., Nokes C., and Bundy D.A., Effects of antiparasitic treatment on dynamically and statically tested cognitive skills over time. Journal of Applied Developmental Psychology, 2006. 27: p. 499–526.
- 71. Kvalsvig J.D., The effects of Schistosomiasis haematobium on the activity of school children. J Trop Med Hyg, 1986. 89(2): p. 85–90. pmid:3095562
- 72. Watkins W.E. and Pollitt E., "Stupidity or worms": do intestinal worms impair mental performance? Psychol Bull, 1997. 121(2): p. 171–91. pmid:9100486
- 73. Ellis P.D. Thresholds for interpreting effect sizes. 2009 [cited 2015 09/06]; Available from: http://www.polyu.edu.hk/mm/effectsizefaqs/thresholds_for_interpreting_effect_sizes2.html.
- 74.
Glass G.V., McGaw B., and Smith M.L., Meta-Analysis in Social Research. 1981, Beverly Hills: Sage.
- 75. Manchikanti L., et al., Evidence-based medicine, systematic reviews, and guidelines in interventional pain management: part 6. Systematic reviews and meta-analyses of observational studies. Pain Physician, 2009. 12(5): p. 819–50. pmid:19787009
- 76. Shrier I., et al., Should meta-analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles. Am J Epidemiol, 2007. 166(10): p. 1203–9. pmid:17712019
- 77. Hagberg H., Gressens P., and Mallard C., Inflammation during fetal and neonatal life: implications for neurologic and neuropsychiatric disease in children and adults. Ann Neurol, 2012. 71(4): p. 444–57. pmid:22334391
- 78. Hagberg H., Peebles D., and Mallard C., Models of white matter injury: comparison of infectious, hypoxic-ischemic, and excitotoxic insults. Ment Retard Dev Disabil Res Rev, 2002. 8(1): p. 30–8. pmid:11921384