Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

A Systematic Review of Methods for Handling Missing Variance Data in Meta-Analyses of Interventions in Type 2 Diabetes Mellitus

Abstract

Aims

Meta-analysis is of critical importance to decision makers to assess the comparative efficacy and safety of interventions and is integral to health technology assessment. A major problem for the meta-analysis of continuous outcomes is that associated variance data are not consistently reported in trial publications. The omission of studies from a meta-analysis due to incomplete reporting may introduce bias. The objectives of this study are to summarise and describe the methods used for handling missing variance data in meta-analyses in populations with type 2 diabetes mellitus (T2DM).

Methods

Electronic databases, Embase, MEDLINE, and the Cochrane Library (accessed June 2015), were systematically searched to identify meta-analyses of interventions in patients with T2DM. Eligible studies included those which analysed the change in HbA1c from baseline.

Results

Sixty-seven publications reporting on meta-analyses of change in HbA1c from baseline in T2DM were identified. Approaches for dealing with missing variance data were reported in 41% of publications and included algebraic calculation, trial-level imputation, and no imputation.

Conclusions

Meta-analysis publications typically fail to report standardised approaches for dealing with missing variance data. While no particular imputation method is favoured, authors are discouraged from using a no-imputation approach. Instead, authors are encouraged to explore different approaches using sensitivity analyses and to improve the quality of reporting by documenting the methods used to deal with missing variance data.

Citation: Batson S, Burton H (2016) A Systematic Review of Methods for Handling Missing Variance Data in Meta-Analyses of Interventions in Type 2 Diabetes Mellitus. PLoS ONE 11(10): e0164827. https://doi.org/10.1371/journal.pone.0164827

Editor: James M. Wright, University of British Columbia, CANADA

Received: July 17, 2016; Accepted: September 30, 2016; Published: October 17, 2016

Copyright: © 2016 Batson, Burton. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: This study was funded by DRG Abacus. DRG Abacus provided support in the form of salary for authors SB and HB, but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.

Competing interests: We have the following interests. This study was funded by DRG Abacus. Batson and Burton are employed by DRG Abacus. There are no patents, products in development or marketed products to declare. This does not alter our adherence to all the PLOS ONE policies on sharing data and materials, as detailed online in the guide for authors.

Introduction

Meta-analysis is of critical importance for decision makers to assess the comparative efficacy and safety of interventions and is becoming an increasingly important component of health technology assessment. Indirect comparison and network meta-analyses are used to assess the comparative efficacy and safety of interventions in the absence of randomised controlled trials (RCTs) comparing these interventions. The most common method for analysing continuous outcomes in meta-analysis is to base the analysis on the mean change from baseline and an appropriate level of uncertainty such as the standard deviation (SD) [1]. A major problem for the meta-analysis of continuous outcomes is that trial publications do not consistently report the SDs for change in outcomes from baseline [2]. There is no empirical evidence exploring the reasons for missing SDs in RCTs but previous studies have reported that greater than 90% of studies report incomplete efficacy outcomes [3, 4]. In addition, these studies concluded that statistically significant trial efficacy results have statistically significantly higher odds of being fully reported in trial publications compared with non-significant results [3, 4]. The association of incomplete reporting of outcomes with statistical significance is of concern for meta-analysis. The omission of studies from a meta-analysis due to incomplete reporting may introduce bias and reduce statistical power.

The Cochrane handbook outlines methods of algebraic calculations for estimating the SD using alternative parametric summary statistics (p-values, confidence intervals [CIs], standard errors [SEs]) which may be reported in trial publications instead [5]. Algebraic calculation methods rely on reported parametric summary statistics with the assumption that they have approximate distributions. The statistical significance of hypothesis tests may not be reported as an exact value but instead in various ways, such as an upper bound or critical p-value. In practice a critical or upper-bound p-value (i.e. p<0.01) may be used as an approximation to compute the SD. This represents a conservative approach to quantifying uncertainty [5]. In practice many primary study publications fail to report any adequate measures of uncertainty around the change from baseline of continuous outcomes [3, 4] and, therefore, alternative methods for handling missing variance data are required.

Although it is recommended that authors are contacted for missing data, in practice they rarely respond. In the event that no measures of uncertainty are reported there are a number of methods for variance imputations reported in the literature [5–10]. A previous systematic review by Wiebe et al. highlighted that missing SDs is a widespread problem and that there is a lack of standardisation in the methods for handling missing variance data in meta-analysis [10]. This is likely a result of the lack of clear guidance in the Cochrane handbook or NICE technical support documents regarding the ‘gold standard’ approach to imputation.

Meta-analyses for type 2 diabetes mellitus (T2DM) are of particular interest because of the rising prevalence of the disease and the increasing burden on global healthcare resources. The demand for meta-analyses of diabetes therapies has grown over recent years due to the emergence of novel agents. The primary measure of control in T2DM is the change in HbA1c levels from baseline, which represents a relevant continuous outcome. This study aims to present a current review of the methods used to address missing SDs in meta-analysis of continuous outcomes in T2DM.

Methods

Search strategy

Electronic databases were searched on 4 June 2015 (Embase; MEDLINE® In-Process & Other Non-Indexed Citations; Ovid MEDLINE; Cochrane Library [Cochrane Central Register of Controlled Trials; Cochrane Database of Systematic Reviews; Database of Abstracts of Reviews of Effects; Cochrane Library Health Technology Assessment]) [S1 Table, supporting information]. Two reviewers independently screened the titles and abstracts of identified citations using pre-specified eligibility criteria; potentially relevant citations were then screened based on the full publication to identify pertinent studies for inclusion. Disagreements were resolved through discussion until a consensus was reached or via the involvement of a third reviewer where necessary.

Inclusion and exclusion criteria are summarised in Table 1.

thumbnail
Download:
Table 1. Inclusion and exclusion criteria.

https://doi.org/10.1371/journal.pone.0164827.t001

Inclusion criteria

A publication was included if it detailed a standard pair-wise meta-analysis, indirect comparison or network meta-analysis of change in HbA1c in patients with T2DM, and it also assessed the use of a pharmacological intervention for treatment or management of T2DM. In order to obtain a reasonable study sample size of only the most recent studies to reflect the current practices, only English language publications published from 2013 onwards were considered.

Exclusion criteria

Publications were excluded if the patient population did not have T2DM or the interventions assessed in the meta-analysis were dietary supplements or non-pharmacological management strategies, such as diet and exercise therapy.

Data extraction

Data extraction was performed into an extraction template and verified by a second extractor. Disagreements were discussed with a third party.

Results

A total of sixty-seven publications reporting a meta-analysis of change in HbA1c from baseline in T2DM patient populations were identified. The study flow diagram is shown in Fig 1.

thumbnail
Download:
Fig 1. Study flow diagram.

https://doi.org/10.1371/journal.pone.0164827.g001

A completed PRISMA checklist can be found in S2 Table. Thirty-nine (58%) of these publications did not report any approaches for dealing with missing variance data. Methods for handling missing variance data were reported in twenty-eight studies (42%) and are classified as algebraic recalculation, trial-level imputation, and no-imputation (Table 2).

thumbnail
Download:
Table 2. Summary of approaches to dealing with missing variance data across the included studies.

https://doi.org/10.1371/journal.pone.0164827.t002

Algebraic calculations

Twelve publications (18%) reported computation of SD from parametric summary statistics [11–22]. Six of these studies (9%) described the use of algebraic calculations but did not detail any further strategies for handling missing variance data [11–16]. A further six publications (9%) described the use of algebraic calculations and further strategies for imputation of missing SDs [17–22]. The following statistics were suggested for use across the publications: SEs, 95% CIs and p-values. None of the publications reported sufficient detail to assess which summary statistics were used in the calculations or to validate the calculations.

Trial-level imputation

It is possible to calculate missing SDs with an imputed value of a correlation coefficient if baseline and final SDs are known [5]. Five studies (7%) reported imputing SDs using a correlation coefficient [18, 23–26]. The value used for the correlation coefficient may be imputed from another study, or alternatively a conservative estimate of 0.5 is recommended [1, 5]. Four of the studies (6%) reported that a correlation coefficient of 0.5 was used [18, 23, 25, 26]. A single study reported that a correlation coefficient of 0.4 was employed, which appeared to be obtained using data from previous meta-analyses, though this was not explicitly stated [24].

A single study imputed the SD from a comparable trial using the same treatment interventions [22]. Two studies (3%) reported imputing SDs by averaging across trials, although different approaches regarding study selection were implemented: i) averaging across all other ‘main’ studies [27] or ii) averaging across other trials of the same treatment comparisons [28].

Two studies (3%) [19, 21] reported the use of imputation methods outlined in the Cochrane handbook [5]. The potential imputation methods provided by Cochrane include using a correlation coefficient, borrowing values from one or more studies in the same meta-analysis or from another meta-analysis, or a more sophisticated method (such as linear regression) [8]. A single study [29] used a prognostic method which predicts missing SEs from known SEs using a previously published equation [6]. A single study which used a trial-level imputation approach to handle missing variance data reported a sensitivity analysis of the approximations by performing a no-imputation approach [28].

No-imputation

Four studies (6%) excluded trials from their meta-analysis if there was insufficient information for algebraic calculation of SD [30–33]. None of the studies discussed the limitations and potential bias introduced by this approach in the publications.

Three studies reported attempts to contact authors for missing data but it was unclear if these attempts were successful and, if not, if other approaches (such as imputation) were explored [17, 34, 35].

Sensitivity analyses

Three publications (4%) [23, 28, 36] detailed sensitivity analyses examining the impact of trial-level imputation approaches on analysis results; of these, one study did not clearly describe the sensitivity analyses performed or the results [36]. Another study fully reported that two sensitivity analyses were performed by assuming correlation coefficients of either 0.25 or 0.75, concluding that the different correlation coefficients did not change the results of the analysis [23]. A further study performed a sensitivity analysis which excluded the trials that did not report SD and concluded that the results of this analysis were broadly similar with the base case analysis [28].

Discussion

A high proportion (58%) of recently reported meta-analysis publications in T2DM fail to report approaches to dealing with missing variance data for continuous outcomes. The majority of publications (42%) which do report a strategy for dealing with missing SDs are inconsistent or not comprehensive regarding the methods used for meta-analysis. The lack of transparency in approaches to handle missing variance data may in part be due to journal restrictions and a historical paucity regarding the standardisation and reporting of imputation methods.

Algebraic recalculation was stated in 18% of the identified meta-analysis publications, although it is expected that the large majority of the included studies will have used this approach given that the methods are outlined within the Cochrane handbook [5]. It was unclear if actual p-values were used in the algebraic calculations or if upper bound, critical, or non-significant p-values were employed in approximate algebraic recalculations. It is worth noting that such approximations are conservative and will result in larger SDs and a reduced weighting of that particular study in the meta-analysis [10].

The trial-level imputation methods identified in this review can be considered as relatively basic and included direct substitution, arithmetic mean substitution, and the imputation of correlation. There are other methods of imputation in the literature which were not applied in the identified studies. These methods include, but are not limited to, study level imputations from parametric summaries, regression with covariates, and coefficient of variation [10]. In addition, none of the methods identified in this review incorporated uncertainty associated with imputation. Multiple imputation is the only method which accounts for the implicit uncertainty from the missing SDs [9, 10, 37]. The findings of this study suggest that the choice of approach is based upon the simplicity and accessibility of methods with authors avoiding the use of other complex methods reported in the literature [9, 10, 37].

When missing SDs are imputed, some form of sensitivity analysis should be conducted to assess the robustness of the results of the analysis in relation to the assumptions made and as verification for the use of imputation. Few studies (4%) reported conducting sensitivity analyses surrounding the methods of imputation; those which did concluded that the results of the analyses were robust to the assumptions made. None of the studies explored sensitivity of the analyses to different methods of imputing the estimates of variance. However, three previous studies suggest that alternative methods for imputing estimates of variance do not alter the conclusions of a meta-analysis, although the generalisation of these findings to other meta-analyses is unknown [7, 38, 39].

As with all reviews, the current analysis was subject to limitations. A review is only as robust as the data supporting it; therefore, a main limitation of this research was poor reporting regarding the methods for handling missing variance data in the identified publications. Only English language publications were considered and there was no hand searching of grey literature. Only meta-analyses in T2DM were considered and the applicability of the results across other disease areas is unknown. However, we have no reason to believe the findings of this review wouldn’t be generalisable to meta-analyses of continuous outcomes, regardless of therapeutic area. The prevalence statistics obtained may be limited due to the relatively small sample of included studies (n = 67). Empirical work to compare methods to address missing variance data is beyond the scope of the review but further research to test the impact of the different approaches is recommended.

This study presented a review of the current methods used to address missing SDs in the meta-analysis of change in HbA1c in T2DM. The date restrictions of the literature searches have ensured that this review is representative of the most current practices used by authors to address missing variance data in meta-analysis. It is expected that this research represents the only published review of methods to deal with missing variance data focusing on a single therapeutic area. The use of a specific research question in a focused population facilitated data acquisition. The presented results are broadly consistent with the previous Wiebe et al. review, conducted over a decade ago, with searches performed in 2002 and no restriction regarding dates or the therapeutic area. The lack of standardisation of methods to address missing variance data appears to remain, although the current review identified a narrower range of methods used for variance imputation compared with those outlined in Wiebe at al. [10]. The brief guidelines on selecting a method for handling missing data set out by Wiebe at al. still represent a logical and systematic approach [10]. Further to this, authors are discouraged from using the no-imputation approach and are instead encouraged to improve the quality of reporting of their research. Sensitivity analyses are also encouraged and, as no method of imputation is known to be absolutely superior to another, exploration of different methods of imputation are desirable.

Supporting Information

S1 Table. Electronic literature searches.

https://doi.org/10.1371/journal.pone.0164827.s001

(DOCX)

S2 Table. PRISMA checklist.

https://doi.org/10.1371/journal.pone.0164827.s002

(DOCX)

Author Contributions

  1. Conceptualization: SB.
  2. Data curation: SB HB.
  3. Formal analysis: SB HB.
  4. Funding acquisition: SB HB.
  5. Investigation: SB HB.
  6. Methodology: SB HB.
  7. Project administration: SB HB.
  8. Resources: SB HB.
  9. Software: SB HB.
  10. Supervision: SB HB.
  11. Validation: SB HB.
  12. Visualization: SB HB.
  13. Writing – original draft: SB HB.
  14. Writing – review & editing: SB HB.

References

  1. 1. Dias S, Welton N.J., Sutton A.J. & Ades A.E. NICE DSU Technical Support Document 2: A Generalised Linear Modelling Framework for Pairwise and Network Meta-Analysis of Randomised Controlled Trials. www.nicedsuorg.uk. 2011.
  2. 2. Streiner DL, Joffe R. The adequacy of reporting randomized, controlled trials in the evaluation of antidepressants. Canadian journal of psychiatry Revue canadienne de psychiatrie. 1998;43(10):1026–30. pmid:9868568.
  3. 3. Chan AW, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ: Canadian Medical Association journal = journal de l'Association medicale canadienne. 2004;171(7):735–40. pmid:15451835; PubMed Central PMCID: PMC517858.
  4. 4. Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. Jama. 2004;291(20):2457–65. pmid:15161896.
  5. 5. Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Available from: Available from www.cochrane-handbook.org. Vol. 5.0.1 Sept 2008.
    • 6. Ma J, Liu W, Hunter A, Zhang W. Performing meta-analysis with incomplete statistical information in clinical trials. BMC medical research methodology. 2008;8:56. pmid:18706124; PubMed Central PMCID: PMC2571096.
    • 7. Furukawa TA, Barbui C, Cipriani A, Brambilla P, Watanabe N. Imputing missing standard deviations in meta-analyses can provide accurate results. Journal of clinical epidemiology. 2006;59(1):7–10. pmid:16360555.
    • 8. Marinho VC, Higgins JP, Sheiham A, Logan S. Fluoride toothpastes for preventing dental caries in children and adolescents. The Cochrane database of systematic reviews. 2003;(1):CD002278. pmid:12535435.
    • 9. Stevens JW. A note on dealing with missing standard errors in meta-analyses of continuous outcome measures in WinBUGS. Pharmaceutical statistics. 2011;10(4):374–8. pmid:21394888.
    • 10. Wiebe N, Vandermeer B, Platt RW, Klassen TP, Moher D, Barrowman NJ. A systematic review identifies a lack of standardization in methods for handling missing variance data. Journal of clinical epidemiology. 2006;59(4):342–53. pmid:16549255.
    • 11. Berhan A, Barker A. Sodium glucose co-transport 2 inhibitors in the treatment of type 2 diabetes mellitus: a meta-analysis of randomized double-blind controlled trials. BMC endocrine disorders. 2013;13:58. pmid:24341330; PubMed Central PMCID: PMC3883465.
    • 12. Berhan A, Berhan Y. Efficacy of alogliptin in type 2 diabetes treatment: a meta-analysis of randomized double-blind controlled studies. BMC endocrine disorders. 2013;13:9. pmid:23452780; PubMed Central PMCID: PMC3639816.
    • 13. Craddy P, Palin HJ, Johnson KI. Comparative effectiveness of dipeptidylpeptidase-4 inhibitors in type 2 diabetes: a systematic review and mixed treatment comparison. Diabetes therapy: research, treatment and education of diabetes and related disorders. 2014;5(1):1–41. pmid:24664619; PubMed Central PMCID: PMC4065303.
    • 14. Landman GW, de Bock GH, van Hateren KJ, van Dijk PR, Groenier KH, Gans RO, et al. Safety and efficacy of gliclazide as treatment for type 2 diabetes: a systematic review and meta-analysis of randomized trials. PloS one. 2014;9(2):e82880. pmid:24533045; PubMed Central PMCID: PMC3922704.
    • 15. Rys P, Wojciechowski P, Siejka S, Malecki P, Hak L, Malecki MT. A comparison of biphasic insulin aspart and insulin glargine administered with oral antidiabetic drugs in type 2 diabetes mellitus—a systematic review and meta-analysis. International journal of clinical practice. 2014;68(3):304–13. pmid:24471814.
    • 16. Yang XP, Lai D, Zhong XY, Shen HP, Huang YL. Efficacy and safety of canagliflozin in subjects with type 2 diabetes: systematic review and meta-analysis. European journal of clinical pharmacology. 2014;70(10):1149–58. pmid:25124541.
    • 17. Hemmingsen B, Schroll JB, Wetterslev J, Gluud C, Vaag A, Sonne DP, et al. Sulfonylurea versus metformin monotherapy in patients with type 2 diabetes: a Cochrane systematic review and meta-analysis of randomized clinical trials and trial sequential analysis. CMAJ open. 2014;2(3):E162–75. pmid:25295236; PubMed Central PMCID: PMC4185978.
    • 18. Mearns ES, Sobieraj DM, White CM, Saulsberry WJ, Kohn CG, Doleh Y, et al. Comparative efficacy and safety of antidiabetic drug regimens added to metformin monotherapy in patients with type 2 diabetes: a network meta-analysis. PloS one. 2015;10(4):e0125879. pmid:25919293; PubMed Central PMCID: PMC4412636.
    • 19. Orme M, Fenici P, Lomon ID, Wygant G, Townsend R, Roudaut M. A systematic review and mixed-treatment comparison of dapagliflozin with existing anti-diabetes treatments for those with type 2 diabetes mellitus inadequately controlled by sulfonylurea monotherapy. Diabetology & metabolic syndrome. 2014;6:73. pmid:25006351; PubMed Central PMCID: PMC4085736.
    • 20. Scott DA, Boye KS, Timlin L, Clark JF, Best JH. A network meta-analysis to compare glycaemic control in patients with type 2 diabetes treated with exenatide once weekly or liraglutide once daily in comparison with insulin glargine, exenatide twice daily or placebo. Diabetes, obesity & metabolism. 2013;15(3):213–23. pmid:22958381.
    • 21. Wang T, Gou Z, Wang F, Ma M, Zhai SD. Comparison of GLP-1 analogues versus sitagliptin in the management of type 2 diabetes: systematic review and meta-analysis of head-to-head studies. PloS one. 2014;9(8):e103798. pmid:25089625; PubMed Central PMCID: PMC4121242.
    • 22. Fournier M, Germe M, Theobald K, Scholz GH, Lehmacher W. Indirect comparison of lixisenatide versus neutral protamine Hagedorn insulin as add-on to metformin and sulphonylurea in patients with type 2 diabetes mellitus. German medical science: GMS e-journal. 2014;12:Doc14. pmid:25332702; PubMed Central PMCID: PMC4202665.
    • 23. Esposito K, Chiodini P, Maiorino MI, Bellastella G, Capuano A, Giugliano D. Glycaemic durability with dipeptidyl peptidase-4 inhibitors in type 2 diabetes: a systematic review and meta-analysis of long-term randomised controlled trials. BMJ open. 2014;4(6):e005442. pmid:24916090; PubMed Central PMCID: PMC4067816.
    • 24. Ren Z, Zhu K, Kang H, Lu M, Qu Z, Lu L, et al. Randomized controlled trial of the prophylactic effect of urea-based cream on sorafenib-associated hand-foot skin reactions in patients with advanced hepatocellular carcinoma. J Clin Oncol. 2015;33(8):894–900. http://dx.doi.org/10.1200/JCO.2013.52.9651. pmid:25667293.
    • 25. Zhu Q, Tong Y, Wu T, Li J, Tong N. Comparison of the hypoglycemic effect of acarbose monotherapy in patients with type 2 diabetes mellitus consuming an Eastern or Western diet: a systematic meta-analysis. Clinical therapeutics. 2013;35(6):880–99. pmid:23602502.
    • 26. Wang C, Mamza J, Idris I. Biphasic vs basal bolus insulin regimen in Type 2 diabetes: a systematic review and meta-analysis of randomized controlled trials. Diabetic medicine: a journal of the British Diabetic Association. 2015;32(5):585–94. pmid:25594251.
    • 27. Schmidt LJ, Habacher W, Augustin T, Krahulec E, Semlitsch T. A systematic review and meta-analysis of the efficacy of lixisenatide in the treatment of patients with type 2 diabetes. Diabetes, obesity & metabolism. 2014;16(9):769–79. pmid:24476092.
    • 28. Hirst JA, Farmer AJ, Dyar A, Lung TW, Stevens RJ. Estimating the effect of sulfonylurea on HbA1c in diabetes: a systematic review and meta-analysis. Diabetologia. 2013;56(5):973–84. pmid:23494446; PubMed Central PMCID: PMC3622755.
    • 29. Goring S, Hawkins N, Wygant G, Roudaut M, Townsend R, Wood I, et al. Dapagliflozin compared with other oral anti-diabetes treatments when added to metformin monotherapy: a systematic review and network meta-analysis. Diabetes, obesity & metabolism. 2014;16(5):433–42. pmid:24237939.
    • 30. Zhang Y, Hong J, Chi J, Gu W, Ning G, Wang W. Head-to-head comparison of dipeptidyl peptidase-IV inhibitors and sulfonylureas—a meta-analysis from randomized clinical trials. Diabetes/metabolism research and reviews. 2014;30(3):241–56. pmid:24123720.
    • 31. Katout M, Zhu H, Rutsky J, Shah P, Brook RD, Zhong J, et al. Effect of GLP-1 mimetics on blood pressure and relationship to weight loss and glycemia lowering: results of a systematic meta-analysis and meta-regression. American journal of hypertension. 2014;27(1):130–9. pmid:24263424.
    • 32. Kim YG, Hahn S, Oh TJ, Park KS, Cho YM. Differences in the HbA1c-lowering efficacy of glucagon-like peptide-1 analogues between Asians and non-Asians: a systematic review and meta-analysis. Diabetes, obesity & metabolism. 2014;16(10):900–9. pmid:24655583.
    • 33. Hou L, Zhao T, Liu Y, Zhang Y. Efficacy and safety of sitagliptin compared with sulfonylurea therapy in patients with type 2 diabetes showing inadequately controlled glycosylated hemoglobin with metformin monotherapy: A meta-analysis. Experimental and therapeutic medicine. 2015;9(4):1528–36. pmid:25780464; PubMed Central PMCID: PMC4353757.
    • 34. Cai X, Han X, Luo Y, Ji L. Comparisons of the efficacy of alpha glucosidase inhibitors on type 2 diabetes patients between Asian and Caucasian. PloS one. 2013;8(11):e79421. pmid:24236131; PubMed Central PMCID: PMC3827349.
    • 35. Vasilakou D, Karagiannis T, Athanasiadou E, Mainou M, Liakos A, Bekiari E, et al. Sodium-glucose cotransporter 2 inhibitors for type 2 diabetes: a systematic review and meta-analysis. Annals of internal medicine. 2013;159(4):262–74. pmid:24026259.
    • 36. Tricco AC, Antony J, Khan PA, Ghassemi M, Hamid JS, Ashoor H, et al. Safety and effectiveness of dipeptidyl peptidase-4 inhibitors versus intermediate-acting insulin or placebo for patients with type 2 diabetes failing two oral antihyperglycaemic agents: a systematic review and network meta-analysis. BMJ open. 2014;4(12):e005752. pmid:25537781; PubMed Central PMCID: PMC4275675.
    • 37. Rubin DB, Schenker N. Multiple imputation in health-care databases: an overview and some applications. Statistics in medicine. 1991;10(4):585–98. pmid:2057657.
    • 38. Thiessen Philbrook H, Barrowman N, Garg AX. Imputing variance estimates do not alter the conclusions of a meta-analysis with continuous outcomes: a case study of changes in renal function after living kidney donation. Journal of clinical epidemiology. 2007;60(3):228–40. pmid:17292016.
    • 39. Follmann D, Elliott P, Suh I, Cutler J. Variance imputation for overviews of clinical trials with continuous response. Journal of clinical epidemiology. 1992;45(7):769–73. pmid:1619456.