Dataset

We obtained data for rat animal models of ischemic stroke from the Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies (CAMARADES) database [41], focusing our meta-analysis on animal models that reported outcomes in infarct volume (see Materials and methods for inclusion criteria of studies). We extracted data for infarct volume from 1,318 control group cohorts from 778 studies for our analyses, investigating the effects of methodology and variability. We extracted data for the effect of treatment on infarct volume from 1,803 treatment/control group cohort pairs from 791 studies for our analyses, investigating the effects of drug treatment on interindividual variability (see Data Availability Statement section for full data and code).

Methodology and variability

To identify methodological procedures that generated variability in disease state, we first meta-analyzed variability in infarct volume for control group animals. We quantify variability as the lnCV rather than the log of standard deviation because we found that our data showed a linear log mean–variance relationship (i.e., Taylor’s law, where the variance increases with an increase in the mean [42]; S1 Fig). Overall, the coefficient of variation (CV) in infarct volume across control groups was around 23.6% of the mean (lnCV = −1.444, CI = −1.546 to −1.342, τ² = 0.565; Fig 1). We found large differences in variability of infarct volume (I²_total = 93.7%), suggesting that sampling variance alone cannot account for differences in the reported variability across control groups (Table 1). The I² attributable to study was 49.6%, suggesting that methodological differences across studies explained some of this heterogeneity, although a moderate amount (42.9%) of I² remained unexplained (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The effects of methodological parameters on variability (CV) in infarct volume across control groups.

Mean estimates of unconditional (marginalized), group-specific coefficients of variation (%) are indicated as gray circles, while the overall estimate is indicated as a gray diamond. Moreover, 95% CIs are shown as gray lines and are asymmetric due to back-transformation of log coefficient of variation (lnCV) to the natural scale. Spontaneous occlusion generated the highest estimate of variability as indicated by the arrowhead. The overall and group-specific estimates were obtained from MLMA and MLMR models, respectively. The data underlying this figure can be found at https://doi.org/10.6084/m9.figshare.14527317.v4. CI, credible interval; CV, coefficient of variation; lnCV, log coefficient of variation; MLMA, multilevel meta-analysis; MLMR, multilevel meta-regression.

https://doi.org/10.1371/journal.pbio.3001009.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Heterogeneity (I²) estimates for analyses of methodology on variability (lnCV) and drug treatment on mean (lnRR) and variance (lnCVR) in rat infarct volume.

https://doi.org/10.1371/journal.pbio.3001009.t001

We detected statistically significant differences in variability of infarct volume between various methodological approaches (Fig 1; see S1 and S2 Tables for unconditional and conditional model coefficients, respectively). Among occlusion methods, models with spontaneous occlusion produced the greatest variability in infarct volume (CV = 52.5%; lnCV = −0.644, −1.633 to 0.345), while filamental occlusion had lowest variability (CV = 17.9%; lnCV = −1.720, −2.195 to −1.244). Studies using temporary models of ischemia had higher variability in infarct volume (CV = 25.2%; lnCV = −1.377, −1.500 to −1.255) compared with permanent models. Variability was slightly but significantly lower with longer time of damage assessment (lnCV = −1.404, −1.521 to −1.288) and greater median weight of the control group cohort (lnCV = −1.366, −1.486 to −1.245).

Drug treatment effects and interindividual variation

To quantify generalizability in drug treatment outcomes, we meta-analyzed the mean and the CV in infarct volume for the effects observed in control/experimental contrasts. We quantified the mean and interindividual variability as the lnRR and lnCVR, respectively. Overall, mean infarct volume in experimental groups was around 33.1%, smaller than in control groups (lnRR = −0.402, −0.461 to −0.343; Fig 2A), while the CV in experimental groups was around 32.4% higher than in control groups (lnCVR = 0.280, 0.210 to 0.351; Fig 2B). Overall, heterogeneity in lnRR was very high, while that for lnCVR was moderate, and moderate amounts of heterogeneity were partitioned into the study level for both (Table 1).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. The effects of drug treatments on the difference in (a) mean (lnRR); and (b) variability (lnCVR) in infarct volume across control and experimental rat groups.

Mean estimates of unconditional (marginalized), group-specific effects are shown as gray circles, while the overall estimate is indicated by the gray diamonds. Moreover, 95% CIs are shown as gray lines. Negative lnRR estimates indicate that mean infarct volume is smaller in experimental versus control rats. Negative lnCVR estimates show that interindividual variability in infarct volume is smaller in experimental versus control rats (e.g., HBOT indicated by left-pointing arrowhead), while positive lnCVR estimates show that variability in infarct volume is greater in experimental versus control rats (e.g., ARBs indicated by right-pointing arrowhead). The overall and group-specific estimates were obtained from MLMA and MLMR models, respectively. The data underlying this figure can be found at https://doi.org/10.6084/m9.figshare.14527317.v4. ARB, angiotensin receptor blocker; CI, credible interval; HBOT, hyperbaric oxygen therapy; HMG-CoA, β-Hydroxy β-methylglutaryl-CoA; lnCVR, log coefficient of variation ratio; lnRR, log response ratio; MLMA, multilevel meta-analysis; MLMR, multilevel meta-regression; NO, nitric oxide; NOS, nitric oxide synthase; PPAR, peroxisome proliferator–activated receptor.

https://doi.org/10.1371/journal.pbio.3001009.g002

Both the mean and variability in infarct volume differed significantly across drug treatment groups (Fig 2; see S3 and S4 Tables for unconditional and conditional model coefficients, respectively). Treatment with hypothermia resulted in the largest reduction of mean infarct volume in experimental groups relative to controls (around 49.7% lower in experimental groups than controls; lnRR = −0.687, −0.775 to −0.599). However, hypothermia also had the most variable and inconsistent effect (i.e., intersubject variation) in reducing infarct volume, with the largest ratio of CV between experimental and control groups (interindividual variability around 60.0% higher in experimental groups compared with controls; lnCVR = 0.470, 0.349 to 0.591). In contrast, environmental treatments were the least effective in reducing mean infarct volume (around 7.3% greater in experimental groups than controls; lnRR = 0.071, −0.166 to 0.308). Hyperbaric oxygen therapy (HBOT) has the least variable and most consistent effect on infarct volume (variability around 45.3% less in experimental groups relative to controls; lnCVR = −0.603, −1.483 to 0.277).

Thrombolytics, which include the only regulatory-approved treatment (i.e., tissue plasminogen activator; tPA [33]), reduced mean infarct volume by around 29.6% in experimental relative to control groups (lnRR = −0.351, −0.446 to −0.256). The CV across experimental groups for thrombolytics was around 17.4% higher than control groups (lnCVR = 0.160, 0.031 to 0.289), but it is notable that this increased intersubject variability is much less than that seen with hypothermia. Through quantifying variability in drug treatment outcomes, we propose that treatments be considered generalizable if they reduced mean infarct volume and concurrently show low interindividual variability (i.e., negative lnRR and lnCVR estimates; Fig 3). Drug treatments that on average reduced infarct volume but had variable and inconsistent effects (i.e., had negative lnRR and positive lnCVR estimates; Fig 3) are ungeneralizable but might be appropriate for clinical exploitation in selected patients [30,43]. Conversely, the least successful treatments can be identified as those that consistently do not reduce mean infarct volume (i.e., positive lnRR and lnCVR estimates; Fig 3). We explored whether the sex of groups used in experiments affected lnRR or lnCVR (see Materials and methods for multilevel meta-regression [MLMR] model parameters), but differences in mean or variability of infarct volume did not vary significantly between female and male cohorts (see S5 and S6 Tables for contrast model estimates for sex effects).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Categorization of treatment effects based on mean efficacy (lnRR) and interindividual variability in efficacy (lnCVR).

Estimates (circles) represent unconditional (marginalized), treatment-specific means (lnRR), variability (lnCVR), and their 95% CIs (solid lines) obtained from MLMR models. Treatments that significantly reduce infarct volume (negative lnRR) without significantly affecting the variation are highlighted green, with citicoline indicated by a diamond as the only treatment to significantly reduce infarct volume and also have a negative point estimate of lnCVR. Treatments that significantly reduce infarct volume and increase interindividual variability (positive lnCVR) are highlighted blue. The effects of hypothermia (most negative and positive mean and variability estimates, respectively) and thrombolytics (which include the only regulatory-approved treatment) are highlighted in pink. Histograms show the relationship of the mean and variance in infarct volume between control (orange) and treatment (blue) groups in each quadrant of the graph. The data underlying this figure can be found at https://doi.org/10.6084/m9.figshare.14527317.v4. CI, credible interval; lnCVR, log coefficient of variation ratio; lnRR, log response ratio; MLMR, multilevel meta-regression.

https://doi.org/10.1371/journal.pbio.3001009.g003

Generate variability through methodology

Among stroke animal models, studies may differ in the design of a number of parameters, including the genetic composition of animals (e.g., the sex and strain of rats used [32,44]) as well as laboratory and operational environments (e.g., methods for stroke induction, the duration of ischemia, and the type of anesthesia used [37,38,45]). However, an impediment to heterogenization is that we have not previously had reliable estimates for which methodological parameters may be most successful in generating variability in phenotypic outcomes [15]. Our results therefore quantify heterogeneity and rank the experimental factors that can generate variability in disease state into animal models so that we can most efficiently capture heterogeneity in experimental design.

Our analyses of operational factors reveal that heterogeneity in outcomes may be induced by incorporating spontaneous (CV = 52.5%), embolic (CV = 32.3%), and endothelin (CV = 27.8%) methods of occlusion. Temporary models of occlusion also generate significantly more variability in disease state than permanent models (CV = 25.2% and 20.5%, respectively). Where choices permit, we suggest that these operational design considerations are a valuable approach for introducing variability into animal models, in conjunction with more familiar proposals to diversify the laboratory environment (e.g., through differences in animal housing conditions and feeding regimens [16,19]). Depending on the type and purpose of study, such operational and laboratory design considerations that increase heterogeneity in outcomes through environmental effects may be especially valuable when variability cannot be introduced through the animal’s genetic composition (e.g., for studies that are interested in sex-specific [46,47] or strain-specific outcomes [44,48]).

Considering genetic factors, proposals to include more heterogeneous study samples recommend the inclusion of both sexes over just male or female animals [49–51], as well as the use of multiple strains of inbred mice and rats (or even, multiple species) [52,53]. Recent meta-analyses of variability in male and female rodents show that males may be as or more variable than females in their phenotypic response [54,55]. We also find that male (CV = 23.5%) and female (CV = 23.9%) rats generate quantitatively equal amounts of variability. Counterintuitively, however, we find that studies that used both sexes produce the most consistent outcomes (CV = 17.3%; see S1 Table for full, unconditional model coefficients). We caution that a moderate amount of the total heterogeneity remained unexplained (i.e., residual variation; Table 1). Thus, these outcomes of sex on estimates of variability may be due to confounding effects of unaccounted for differences in experimental design. We therefore emphasize the importance of considering both genetic and environmental parameters for effective heterogenization of studies [56,57].

An alternative approach to heterogenization of experimental designs within studies is to introduce variability by conducting experiments across multiple research laboratories (i.e., multi-laboratory approach) [20,24,58]. Importantly, such an approach inherently captures “unaccounted” sources of variability in experimental conditions that are difficult to systematically manipulate within a single-center study [16,19]. We argue that, especially where logistical constraints may hinder multi-laboratory approaches (e.g., for earlier, basic, and exploratory studies), introducing heterogeneity within studies may provide the most practical alternative [23]. Indeed, by meta-analyzing the variability introduced by differences in experimental methodology across studies, we can begin to find ways in which to heterogenize single studies in order to best capture the variation that exist across laboratories and studies [16,20].

Systematically introducing variability into a system comes at the cost of reduced statistical sensitivity [8,9] and necessitates larger studies [8,26,29]. While in the long-term increased replicability may reduce waste and outweigh the initial costs, these economic and ethical costs must, of course, be minimized. This can be done by identifying from a spectrum of all available methodological choices the most efficient means of introducing heterogeneity within experiments (Fig 1, S1 Table). For some methodological aspects such as operational factors, this will mean replacing current methodologies with choices that induce greater variability in baseline and control group outcomes (e.g., by changing methods of occlusion). For other design parameters that may traditionally be standardized such as genetic or lab environmental factors, this will mean deliberately incorporating these types of heterogeneity in a systematic manner, by including levels of these categorical (e.g., strain) or continuous (e.g., time of assessment) variables using a randomized block or fully factorial design (i.e., “controlled heterogenization” [59]). Regardless of the manner in which heterogeneity is incorporated, however, it is necessary to quantify the amount of variability that different experimental designs introduce, with the aim that researchers can then make informed decisions about how to most efficiently incorporate heterogeneity into study design [14–16,20]. Identifying sources of variability through meta-analysis of variance in existing animal data as we have done here is the most practical and economic way of establishing this much needed knowledge base.

Our analysis is not the first to assess the effects of experimental methodology on variation in disease state in rodent models of stroke [37 38]. Ström and colleagues [37] investigated similar components of experimental design on variation in infarct volume in rats. There are a number of methodological differences between their analyses and ours (e.g., differences in size of dataset and use of formal meta-analytic models). Despite these differences, our quantitative results are largely concordant. Where we differ substantially is in the interpretation of what is a desirable outcome. For example, Ström and colleagues [37] concluded that intraluminal filament procedures provide optimal occlusion methods as they generate minimal variation in disease outcome and maximize statistical power. Our analyses also identify that filament methods have low variation (CV = 17.9%); however, we argue that these gains in statistical power come at the cost of reduced replicability.

We attempted to provide formal statistical support for the hypothesis that heterogeneous methods result in more repeatable treatment effects. We used a second-order meta-regression to assess whether the amount of variation (lnCV) in disease states induced by occlusion methodology predicts heterogeneity in effect sizes for drug treatments using those methods (quantified as lnH [60]). As predicted, there is a negative relationship (slope = −0.876, −2.047 to 0.295; P = 0.142; S2 Fig), suggesting that methodologies that induce greater variability in baseline disease states are associated with more consistent treatment outcomes. We note, however, that our slope estimate is statistically nonsignificant and that our analysis was based on a small number of methodological groups (N = 7) with an unbalanced distribution of drugs/rat strains across those groups (see S7 Table for analysis details and full model results). Nonetheless, our results are encouraging, and we are excited to see further studies formally quantify the relationship between variability induced by methodological procedures and replicability in reported outcomes. Meta-analyses that quantify both variability in control and treatment outcomes, as done here, provide a useful approach for quantifying the relationship between methodological heterogenization and replicable outcomes.

Quantify variability to improve drug translation

Our second approach of simultaneously assessing both the mean and variation in treatment outcomes allows us to place potentially useful treatments into 2 distinct categories for further exploration: (1) beneficial and generalizable interventions, which are those that consistently reduce infarct volume across individuals; and (2) beneficial but non-generalizable interventions, which on average reduce infarct volume but result in large interindividual heterogeneity in outcomes. This latter group could even include treatments that do not necessarily reduce mean state, but have a large enough variance response to be beneficial to some [30,43,61].

Overall, we find that the stroke treatments in our dataset are usually effective, reducing infarct volume on average by 33.1% compared with controls. Out of these effective treatments, we identify 4 treatments that significantly reduced infarct volume but did not induce significant differences in the CV across experimental and control groups (green highlights in Fig 3). Nootropic treatments reduced infarct volume on average by 40.8%, while citicoline, antibiotic, and exercise treatments reduced infarct volume by around 27.5% to 28.8% compared with control groups. None of these treatments were estimated to significantly affect the CV, although estimated effects ranged from 5.7% smaller in experimental relative to controls for citicoline (highlighted with a triangle symbol in Fig 3) to 21.3% to 31.9% greater for the other treatments. We emphasize that these treatments may potentially be more generalizable in that the outcomes of these treatments are on average favorable and are relatively consistent at the individual level [33,34].

Second, we identify a handful of effective treatments that on average reduce infarct volume, but also generate significant amounts of variability in experimental groups (blue highlights in Fig 3; see S3 Table for rank order of unconditional estimates in mean and CV across treatments). Of particular interest to note is that while thrombolytics significantly increase variability in experimental groups relative to controls, they are still relatively consistent in reducing mean infarct volume (on average reducing infarct volume by 29.6%, while the CV in experimental groups is only 17.4% greater than controls). Out of treatments that significantly reduce mean infarct volume, thrombolytics rank second in terms of its consistency in effect, with overlapping confidence intervals in their effects on the CV with those of citicoline (Fig 3).

On the other hand, hypothermia is much more effective in reducing infarct volume (on average reducing infarct volume by 49.7%) but is the least consistent in doing so, estimating the greatest CV (60.0% greater in hypothermia treated groups than concurrent controls). Interestingly, efforts to exploit hypothermia for stroke in clinical trials have so far failed to identify a patient group who might reliably benefit [62]. Other treatments that greatly reduce average infarct volume while increasing the variation include, for example, omega-3, rho GTPase inhibitors, and estrogen treatments. As such, while these treatments confer a mean beneficial effect, this effect may not be generalizable across animals. Any future translation into clinical trials would require tailoring with effort put in to predicting response at the individual level [30]. To our knowledge, such tailoring has not been attempted because a treatment with high variability (inconsistency) is less likely to be statistically significant and pass the preclinical stage (even if it does improve a disease state) [30,43,61,63]. Our study represents the first meta-analyses to quantify both the efficacy and consistency of treatment effects in animal models. We believe that this approach will forge new opportunities for improving the generalizability and translation of preclinical trials by embracing both the mean and variability in outcomes.

Data collection and imputation

We identified studies of rat animal models for stroke from the CAMARADES electronic database (see S3 Fig for database query and selection). For our analysis, we only included experimental studies that reported mean infarct volume (and their associated sample size and standard deviation or standard error) in both control and experimental groups. Where necessary, we calculated the standard deviation from the standard error multiplied by the square root of (n– 1), where n is the sample size of the control or experimental group. Furthermore, when a study used multiple treatment groups for a control group (28% of identified studies), we divided the sample size of the control group equally among the treatment groups, which dealt with correlated errors and prevented sampling (error) variances being overly small [64]. Before calculating the effect sizes, we excluded data where (i) the standard error was reported as 0; or (ii) the sample size of the control group when divided was equal to or less than 1. We also excluded categorical predictors that were represented by fewer than 5 data points. Overall, 2.9% and 1.2% of all identified studies were excluded for methodological and drug treatment analyses, respectively (S3 Fig).

For meta-analysis of variance across methodological parameters, we focused on control groups with sufficient group-level information on the methodology of the experiment. Specifically, we collected and coded methodological predictors as closely as possible to the predictors used by Ström and colleagues [37] to produce a comparable meta-analysis (see full model parameters in S1 Table). For meta-analysis of variance across drug treatment, we included data from studies with sufficient group-level information on the drug group, rat strain, and sex of experimental/control groups (see full model parameters in S3 Table). For all analyses, we dealt with missing data via multiple imputation [65,66] using the package mice [67] as follows: We first generated multiple, simulated datasets (m = 20) by replacing missing values with possible values under the assumption that data are missing at random (MAR) [68,69]. After imputation, meta-analyses were performed on each imputed dataset (as described in the Statistical analysis section), and model estimates were then pooled across analyses into a single set of estimates and errors.

Calculating effect sizes

For meta-analyzing variance across methodological predictors, we calculated the lnCV and its associated sampling variance (s²_lnCV) for each control group. Since many biological systems appear to exhibit a relationship between the mean and the variance on the natural scale (i.e., Taylor’s law; [42,70]), an increase in the mean may correspond to an increase in variance. Our data indeed appears to exhibit a positive and linear relationship between log standard variation (lnSD) and log mean infarct volume (S1 Fig). When such a relationship holds in data, it may be most preferable to use an effect size such as lnCV, which estimates variance accounting for the mean, and this is the approach we have taken.

For meta-analyzing variance across drug treatments, we calculated the log coefficient of variance (lnCVR) and its associated sampling variance (s²_lnCVR) as given in equations (11) and (12) in Nakagawa and colleagues [69] (S8 Table). When meta-analyzing variance in the presence of Taylor’s law as it appears in our dataset, it may be most preferable to use lnCVR (over the log variance ratio, lnVR), which gives the variance of a contrast group accounting for differences in the mean. We therefore report all results using lnCVR in the manuscript. We note that both lnCV and lnCVR assume a linear relationship between log mean and log variance with the slope coefficient of 1 on the log scale. When slope estimates are closer to 0 or nonlinearities are present in the mean–variance relationship, other metrics of variability such as log variability ratio (lnVR) or an approach that directly estimates the strength of association between log mean and log variance (i.e., an arm-based meta-analysis [69]) based on log SD may be more appropriate (for an example of an arm-based approach, see S9 Table and S4 Fig for galaxy plot of lnRR on lnSD). We advise that future analyses of heterogeneity pick the most appropriate statistic and model of variability based on the mean–variance relationships present in their dataset [71]. In addition to assessing the effects of treatments on variance, we further quantified differences in mean infarct volume by calculating the lnRR of the mean for each control/experimental group within a study (lnRR) and its associated sampling variance (s²_lnRR). For both lnRR and lnCVR, we calculated effect sizes so that positive values corresponded to a larger mean or variance in the experimental group.

Statistical analysis

We implemented multilevel meta-analytic models in a likelihood-based package using the function “rma.mv” in the metafor package [72] as described in Eq 1: (1) where y_ij (the ith effect size of variability or mean infarct volume from a set of n effect sizes (i = 1,2,…,n) in the jth study from a set of k studies j = 1,2,…,k) is given by the grand mean (μ), the effects of fixed predictors (βx_ij), and random effects due to study (s_j), strain (t_j), residual (e_ij), and measurement error (m_ij) for the ith effect size in the jth study. Since variability in observed effects may be explained by measurement error (m_ij in Eq 1), we present total I² (the percentage of variance that cannot be explained by measurement error) and study I² (the percentage of variance explained by study-effects) to estimate the true variance in observed effects (i.e., meta-analytic heterogeneity) [60]. We interpreted I² of 25%, 50%, and 75% as small, medium, and large variance, respectively [60].

To estimate variance (lnCV) in outcome as a function of methodology in control groups, we constructed 2 meta-analytic models. First, we fitted a multilevel meta-analysis (MLMA) with the objective of estimating the overall average variability in infarct volume across studies. MLMA included a fixed intercept and random effects described in Eq 1. Second, we fitted a MLMR with the objective of estimating effects of methodological predictors on variability in infarct volume, by fitting the following fixed predictors: (i) method of occlusion; (ii) sex of animal cohort; (iii) type of ischemic model; (iv) type of anesthetic; (v) whether experiments were temperature controlled; (vi) whether rats were physiologically monitored; (vii) mean cohort weight; and (viii) time for evaluation of damage after focal ischemia (S1 Table). Mean cohort weight and time for evaluation were z-transformed prior to model fitting. We similarly constructed MLMA and MLMR models for lnRR and lnCVR (fitting each effect size as the response in separate models) to estimate the mean and variance in outcome as a function of drug treatment in our control/experimental groups, respectively. For these MLMR models, we included (i) drug treatment group, and (ii) sex of animal cohort as fixed predictors (S3 Table). Fixed effects were deemed statistically significant where their 95% credible intervals (CIs) did not span zero. For interpretation of results, we back-transformed model estimates from the log to the natural scale.

Since reported outcomes may be prone to within-study biases particularly with regard to mean estimates, we conducted a sensitivity analysis including publication quality as a random effect in our MLMR model. Publication quality was determined according to guidelines set out by the Stroke Academic Industry Roundtable (STAIR), which scored studies based on whether they implemented strategies to mitigate against both selection and detection bias [73]. Our sensitivity analysis did not lead to any qualitative changes in our main reported outcomes, and publication quality accounted for little in terms of differences in mean infarct volume (I²_lnRR = 0.7%; for full sensitivity model estimates, see S5 and S6 Figs and S10–S12 Tables). Finally, we tested for signs of publication bias (systematic bias in the published data due to the preferential publication of more significant results) in our data by visual inspection of funnel plots (S7 Fig) and conducting a type of Egger regression (precision-effect test and precision-effect estimate with standard errors, PET-PEESE) on lnRR [74]. Egger regression on lnRR suggested a small effect of publication bias in our mean estimate (1.6% difference between bias-corrected and uncorrected estimates of our meta-analytic mean; see S13 Table for publication bias test results). Egger regression cannot be used for lnCVR, and further, it is unlikely that publication bias occurs for lnCVR because such biases are not driven by the difference in standard deviations between the experimental and control groups [75]. All meta-analyses were conducted on the statistical programming environment R (v 3.2.2 [76]).