http://orcid.org/0000-0001-7851-6603
Figures
Abstract
The President’s Malaria Initiative (PMI) launched in 2005 as a key player in malaria prevention and treatment in sub-Saharan Africa (SSA). Several country-specific evaluations have demonstrated great progress in reducing under-five mortality associated with scaling up malaria interventions in PMI priority countries. Documentation of PMI’s specific contributions was limited, until the publication of Jakubowski, et al., which used difference-in-difference analysis to show a higher reduction of under-five mortality in PMI-supported countries than in others. To generate more evidence, this study used rigorous statistical analyses to assess the reduction in mortality attributable to PMI support. The study used generalized estimating equations and a series of matching procedures to evaluate the impact of PMI on under-five mortality and on population coverage of insecticide-treated nets (ITNs), indoor residual spraying (IRS), and artemisinin-based combination therapy (ACT) in SSA. The analyses used country-level secondary data and controlled for several country-level characteristics assumed to influence outcome measures of interest, PMI program participation, or both. The Mahalanobis distance metric, with 1:1 nearest neighbor matching adjusting for bias in population size in the particular country, showed a reduction in under-five mortality by approximately 12 per 1,000 live births (95% Confidence Interval [CI]: 20.6–3.1; p = 0.012). There were statistically significant increases in the population coverage of ITNs, IRS, and ACTs in PMI countries over the implementation period. ITN use in the population was 0.23% higher (95% CI average treatment effect on the treated: 0.17–0.30; p<0.001) in PMI-recipient countries than in non-PMI countries. The findings show that PMI contributed significantly to increasing the coverage of malaria control interventions and reducing under-five mortality in SSA.
Citation: Ye Y, Duah D (2019) The President’s Malaria Initiative contributed to reducing malaria burden in sub-Saharan Africa between 2004 and 2014: Evidence from generalized estimating equation analysis. PLoS ONE 14(5): e0217103. https://doi.org/10.1371/journal.pone.0217103
Editor: Luzia Helena Carvalho, Instituto Rene Rachou, BRAZIL
Received: January 30, 2019; Accepted: May 3, 2019; Published: May 24, 2019
Copyright: © 2019 Ye, Duah. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the manuscript and its Supporting Information files.
Funding: YY is working for ICF on a project funded by USAID. However, this study was perform as an independent scientist outside the normal working hours. The study was based on secondary data publicly available. The funder did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.
Competing interests: YY confirms that his commercial affiliation does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Many government and international organizations, including the United States, have established reducing mortality among children under the age of five to the barest minimum as their primary goal. To that end, significant investments have been made to improve health and nutrition programs. Over the past decade, the U.S. Government, through the President’s Malaria Initiative (PMI), has invested significant funds in the health sector, particularly in sub-Saharan Africa (SSA), contributing to an expansion of key malaria prevention and treatment interventions with the potential to significantly reduce malaria burden [1–3].
Despite the increase in spending from PMI, little is known about the impact attributable to this initiative. In recent years, several country-specific impact evaluations have been conducted in PMI-supported countries, using the plausibility argument, to assess the change in all-cause child mortality associated with expansion of malaria interventions over the past decade. These evaluations documented significant progress in most of the countries [4–7]. However, the design of the evaluation could not measure the specific contribution of PMI to changes observed.
To bridge this gap, Jakubowski, et al.[1] used modified Poisson regression analysis with the difference-in-differences (DID) model to demonstrate that PMI support in countries was associated with the reduction in the annual risk of children under five between 2004 and 2014. The use of DID with fixed effect allows time-invariant country-specific characteristics to be correlated with the other measured independent variables, accounting for potential endogeneity. The authors used robust statistical analytic techniques; however, there are some challenges with their methods. The authors relied on the assumption that in standard DID estimation, the group composition does not change systematically over time. The population structure in the control (non-PMI) and intervention (PMI) countries could change from one survey to another within the same country between 2004 and 2014, creating a selection bias. Current methods to account for these selection biases in DID models are inadequate [8].
To address this problem, matching methods highlight areas of the distribution of covariates, in which the treatment and control groups lack sufficient overlap, such that the resulting treatment- effect estimates would rely on extrapolation. Matching with regression-based model adjustments on the matched samples reduces bias from covariate differences. Furthermore, regression analysis on the matched samples could adjust for the small remaining differences and increase the efficiency of estimating impact. Using a series of nationally representative data from 32 countries in SSA between 2004 and 2014, this study used generalized estimating equations (GEE) regression and matching inferential methods to estimate the impact of PMI support on mortality among children under five, comparing countries receiving PMI support to those not receiving PMI support. The study hypothesized that both GEE and the matching technique would also result in a positive impact of PMI support, as found by Jakubowski, et al.[1]
Methods
Considerations in the choice of the statistic approach
To quantify the impact of PMI support on the reduction of under-five mortality (U5M), one must account for several other factors that could contribute to change in U5M in SSA. Controlling and comparing factors at one time point before the rollout of PMI support and assuming that all other factors would remain unchanged between PMI-recipient and non-PMI recipient countries could lead to biased impact estimates. These country-level characteristics vary over time, and their effect should be examined accordingly. This systematic difference between PMI- and non-PMI- recipient countries makes the two groups different, and changes in the outcome variable may not be attributable to the program. This study takes into account time-variant, country-level characteristics that could be associated with U5M. These factors could influence either the possibility of receiving PMI intervention, U5M, or both (confounders). These factors include gross national income (GNI) per capita based on purchasing power parity (PPP), health expenditure, government effectiveness index, political stability index, corruption index, rule of law, voice and accountability, and other large health funding sources.
Countries included in the analysis
This study involved 32 countries from SSA. Nineteen countries received the intervention (PMI), and 13 countries served as the non-PMI-recipient (control) countries in this study. Of the 19 countries receiving the intervention, three (Angola, Tanzania, and Uganda) have received the initiative since its full implementation in 2006. The Democratic Republic of the Congo, Guinea, Nigeria, and Zimbabwe were the last four countries to have received the intervention in 2011 (Fig 1).
PMI recipient/intervention countries (Year start): Angola (2006), Uganda (2006), Republic of Tanzania (2006),Malawi (2007), Mozambique (2007), Rwanda (2007), Senegal (2007), Benin (2008), Ethiopia (2008), Ghana (2008), Kenya (2008), Liberia (2008), Madagascar (2008), Mali (2008), Zambia (2008), Democratic Republic of the Congo (2011), Guinea (2011), Nigeria (2011), Zimbabwe (2011).
Intervention and program variables of interest
The President Malaria Initiative, launched in 2005, began operation in 2006, with the goal of reducing malaria-related morbidity and mortality by 50% across 15 high-burden countries in SSA through a rapid scale-up of four proven and highly effective malaria prevention and treatment measures. These effective interventions were insecticide-treated nets (ITNs), indoor residual spraying (IRS), accurate diagnosis and prompt treatment with artemisinin-based combination therapies (ACTs), and intermittent preventive treatment of pregnant women. Data on beneficiary countries were obtained from the Eleventh Annual Report to Congress, submitted in April 2017. For details on PMI’s intervention, see PMI website [9].
The selection of PMI countries is based on several criteria. In this study, the authors assumed that country selection was based on country-year characteristics, such as being a recipient of other development assistance for health, neonatal mortality rate, government effectiveness index, voice and accountability, regulatory quality, rule of law, political stability, control of corruption index, average Plasmodium falciparum rate in children ages 2–10, GNI per capita based on PPP, total population, total health expenditure, and rural population between 2004 and 2014. The study also assumed that the propensity of a country to receive the intervention is independent of the education level of the child’s mother, mother’s age, child’s age, socioeconomic status, and other household-level characteristics.
PMI support was measured as a binary indicator, coded as “0” if the PMI program did not provide funding to a given country in a given year and coded as “1” if funding was provided. PMI was also measured as a continuous variable to evaluate the impact of the program intensity (per capita disbursements in a given country and year).
Outcome measures
Primary outcome measure.
All-cause U5M rates (U5MRs) were obtained from the United Nations Inter-agency Group for Child Mortality Estimation (United Nations Children’s Foundation, World Health Organization, World Bank, United Nations Department of Economic and Social Affairs Population Division) at www.childmortality.org for 32 countries in SSA between 2004 and 2014 for PMI- and non-PMI-recipient countries. The U5MR was defined as the probability per 1,000 that a newborn will die before reaching age five, if subject to age-specific mortality rates of the specified year.
Secondary outcome measures.
Other outcome measures of interest in this study were population coverage of malaria prevention and treatment. These estimates were obtained from the Malaria Atlas Project [10] between 2004 and 2014 for both PMI- and non-PMI-recipient countries. These secondary outcome measures, which also serve as explanatory variables in the U5MR models, are as follows:
- Population coverage of ITNs: Proportion of people who slept under an ITN on any given night each year
- Population coverage of IRS: Proportion of the population protected by IRS
- Population coverage of ACTs: Proportion of fever cases in children under five receiving ACTs
Other competing health interventions.
The study used a continuous measure variable called “Other development assistants for health minus PMI” (ODAH-PMI) and controlled for it in the analysis. The ODAH-PMI is the amount of financial and in-kind support tracked from the source to the recipient country, region, or health focus area. This variable sums up all other health interventions, such as the U.S. President’s Emergency Fund for AIDS Relief, the Global Fund to Fight AIDS, Tuberculosis and Malaria, domestic interventions, and all other health disbursements. Data were obtained from the Institute for Health Metrics and Evaluation [11].
Other country-level explanatory variables.
Other country-level explanatory variables are the following: average Plasmodium falciparum rate in children ages 2–10, GNI per capita based on purchasing power parity (PPP), total population, health expenditure, rural population, government effectiveness index, voice and accountability, regulatory quality, rule of law, political stability—no violence, and control of corruption index between 2004 and 2014. Data were obtained from the World Bank website [12].
Statistical analysis
The authors used graphical tools for longitudinal/panel data analysis to explore the trend of U5MR and other parameters of interest between PMI- and non-PMI-recipient countries. Two types of analysis were performed. To assess the association of PMI, ITNs, ACTs, IRS, and U5MR, the authors first fitted a series of GEE. GEE from the Gaussian distribution were used to account for the correlated structure of the data set, [13,14] because observations within a country are more correlated than those between countries. The authors then performed a series of different matching techniques, including nearest neighbor matching based on propensity score and Mahalanobis distance metric measures, and Mahalanobis with propensity score matching (PSM) caliper and subclassification on the propensity score, to establish cause-effect relationships among PMI, ITNs, ACTs, IRS, and U5MR. Statistical significance was considered at alpha = 0.05 using two-tailed tests. All statistical analysis were performed with Stata version 15 (StataCorp, College Station, Texas, USA).
Estimating PMI participation (propensity score model accounting for clustering).
The authors first estimated the propensity for country i to participate in or receive PMI support in year j, which is unknown. They considered certain time-varying factors that could influence the probability of a country receiving PMI support, U5MR, or both. The authors incorporated clustered structure (country-level) in PSM to account for bias from unmeasured cluster-level confounders. The propensity score estimation based on the probit regression model considered the following factors: effective governance, rule of law, regulatory quality, voice and accountability, corruption index, political stability, GNI, recipient of other health intervention, domestic health expenditure, and proportion of rural population. These factors were assumed to have a direct effect on U5MR, the propensity of a country to receive the PMI intervention, or both [15–18].
Population coverage of ITNs, ACTs, IRS, and average Plasmodium falciparum rate in children ages 2–10 were not included in the propensity score model, because these variables may have been affected by the treatment variable of interest (PMI).
Coarsened exact matching.
To achieve consistency and evaluate the robustness of our impact estimate, Coarsened Exact Matching (CEM) [19] was used. This is a relatively new method for improving the estimation of causal effects by reducing imbalance in covariates—in this case, between PMI-recipient countries and nonrecipient countries. Countries were not selected randomly to receive PMI support, so the authors controlled for a set of pretreatment variables using the CEM algorithm. CEM bounds the degree of model dependence and causal effect estimation error by ex-ante user choice. It is monotonic imbalance bounding, which means that reducing the maximum imbalance on one variable—between PMI- and non-PMI-recipient countries—has no effect on other variables.
Assessing common support and diagnosing matches from PSM.
Numerical and graphical diagnoses were used to observe the common support of the distribution of propensity score between PMI- and non-PMI-recipient countries. The authors compared the multidimensional histograms and Kernel density plots of the covariates in the matched PMI and non-PMI countries. In this study, the authors assumed that the standardized differences greater than 10 percent in absolute value indicate a serious imbalance in the covariate of interest between PMI- recipient and nonrecipient countries [20]. The standardized difference d is given as follows:
In using CEM, the authors assessed the overall imbalance of covariates given by the statistic between PMI- and non-PMI-recipient countries. This imbalance measure was introduced [21] as a comprehensive measure of global imbalance.
Sensitivity analysis of GEE.
GEE:This study tested the robustness of the impact estimates of PMI support on U5MR with a series of sensitivity analyses. The authors assessed whether the results from the GEE analysis were robust to the type of model chosen, by estimating the effect of PMI on U5MR using Poisson, negative binomial, and gamma distribution. Sensitivity analyses of different specifications of the GEE intra-cluster correlation matrix were also performed based on autoregressive with lag 1 wave.
Matching procedures: Analyses of sensitivity to the ignorability assumption under PSM were performed. The authors calculated the bounds [22] for average PMI effects on the treated in the presence of unobserved heterogeneity (hidden bias) between PMI-recipient and -nonrecipient countries. Because different matching techniques produce slightly different results (impact estimates), Mahalanobis distance metric matching, Mahalanobis matching within propensity score calipers, subclassification, k:1, and nearest neighbor matching were performed to confirm the robustness of the impact estimate. In addition, because the intensity of aid disbursement varies among recipient countries over time, the authors estimated the amount received U5MR function in which the PMI-recipient countries might take on a continuum of values to ascertain whether similar results could be replicated.
Results
Comparing covariates between PMI- and non-PMI-recipient countries
The estimated U5MR per 1,000 live births between 2004 and 2014 was approximately 94 for PMI countries and approximately 108 for non-PMI-recipient countries, and the difference was statistically significant (p<0.001). PMI-recipient countries had relatively higher population coverage of ITNs, ACTs, and IRS. With the exceptions of corruption index, government effectiveness, and political stability (p>0.05), there was a statistically significant difference in all other covariates between PMI-recipient and -nonrecipient countries over time. Neonatal mortality rate, Plasmodium falciparum transmission rate for children ages 2–10, proportion of rural population, and domestic health expenditure were found to be higher in the PMI-recipient countries than in non-PMI-recipient countries (2004–2014). Governance indices, such as voice and accountability, regulatory quality, and rule of law, were found to be better over the years in PMI-recipient countries. Other health development assistants (million US$) were found to be significantly higher in PMI-recipient countries over the years, as shown in Table 1.
Results on matching procedure
The propensity score histogram by PMI status of countries in Fig 2 showed moderate overlap of the propensity scores, with a good control match for each PMI-recipient country. This indicates that few PMI countries did not find a good match (lack of common support). The diagnostic graph assessment of the covariate balance between PMI- and non-PMI-recipient countries showed that standardized percentage bias between the two groups reduced drastically after matching for all covariates used in the propensity score model (Fig 2). For instance, bias in GNI between the two groups reduced from -23.1% to -11.1%, resulting in an absolute percentage reduction in bias of 52%. The bias in voice and accountability index between PMI- and non-PMI-recipient countries reduced to 84.7% after matching. Between the two groups after matching, bias in other health development assistance was reduced by 83.5%, bias in domestic health expenditure was reduced by 45.1%, bias in effective governance was reduced by 87.3%, and bias in corruption was reduced by 95.6%. The overall imbalance in corruption before matching given by the L1 statistic, introduced by Blackwell, et al., [19] was 27.2% and was reduced to 11.1% after matching (Table 2).
Abbreviation: hexpend = Domestic health expenditure, gni = gross national income, otherdahpmi = Other development health assistant minus PMI, govn: Effective governance, voice: Voice and accountability, regulatory: Regulatory quality; index number 2 and 3 represent 2nd and 3rd order polynomial terms, respectively.
Impact estimates of PMI in reducing U5MR and increasing population coverage of ITNs, IRS, and ACTs
Adjusting for key covariates (ITNs, IRS, and ACTs), the Mahalanobis distance metric matching within propensity score calipers showed that U5MR reduced by 11 per 1,000 live births (95% CI ATET: 18.7–2.3; p = 0.012) in the PMI countries compared to the non-PMI countries. Mahalanobis distance metric with 1:1 nearest neighbor matching adjusting further for bias in population size and the particular country resulted in the reduction of U5MR by approximately 12 per 1,000 live births (95% CI ATET: 20.6–3.1; p = 0.012). There was a statistically significant increase in the population coverage of ITNs, IRS, and ACTs for PMI countries over the implementation period. Population coverage of ITN use increased by 0.23% (95% CI ATET: 0.17–0.30; p<0.001) for PMI-recipient countries compared to non-PMI countries. The impact of PMI became weightier with the use of 2:1 nearest neighbor matching based on Mahalanobis distance metric as the number of under-five deaths reduced to approximately 13 per 1,000 live births (95% CI ATET: 0.17–0.30; p = 0.002) in the PMI countries. Similar patterns were observed for population coverage of ITNs, IRS, and ACTs (Table 3).
Discussion
This study evaluated the impact of PMI on reducing U5MR and increasing population coverage of ITNs, IRS, and ACTs using population-averaged regression models for panel data analysis and matching procedures, which serve as one of several methods of estimating the causal effect of interventions using observational data. The use of matching procedures replicated a randomized experiment as closely as possible, by obtaining treatment (PMI-recipient countries) and control groups (non-PMI-recipient countries) with similar country-level covariate distributions to reduce bias from these measured covariates.
PMI was found to have statistically significant impact on the reductions in all-cause mortality rates among children under five and in the increase in population coverage of ITNs, IRS, and ACTs. These findings were consistent with the results obtained by Jakubowski, et al., [1] who used a DID analytic technique to determine significant statistical associations between PMI support and U5MR, ITNs, ACTs, and IRS. It is worth emphasizing that ITN use, ACTs, and IRS are all in the causal pathway of U5MR in SSA. This shows that increasing population coverage of these interventions through PMI support reduces U5MR. Several other studies have shown the effectiveness of these intermediate outcomes in increasing population coverage of IRS, ITNs, and ACTs, as was the case in this study. Eisele, et al., [23] based on two systematic literature reviews in Plasmodium falciparum endemic settings, have demonstrated the effect of the coverage of ITNs and IRS on preventing malaria-attributable mortality in children ages 1–59 months. Furthermore, Gimnig, et al. [24] also used an analytic technique similar to this study (propensity scores with regression models) to evaluate the effect of IRS programs on malaria-related outcomes and arrived at findings that were similar to our results.
Although sensitivity analysis was conducted to critically evaluate the robustness of our results, there may be several unmeasured variables that could systematically affect U5MR and the use of ITNs, IRS, and ACTs. Country-specific interventions that were not measured and controlled for in the analysis could affect the results of our findings, especially when these unknown factors affect PMI and comparison countries in different ways. Apart from the usual government interventions and funding partners’ support in terms of health spending that are usually captured in a national budget, several other nongovernmental organizations have supported initiatives to control malaria. These initiatives may not be known, and the data required to measure how they influence health outcomes are not readily available, particularly considering the need for cross-country comparisons. Furthermore, differences in country-specific malaria interventions and the proportion of children under five who are covered by these interventions could bias our results, because the lack of reliable data prevented this study from controlling for them. This study did not examine the spill-over effect of PMI, and future research should explore whether the investments made by PMI in recipient countries improved other health and non-health outcomes, such as improved immunization coverage, health infrastructure, educational attainment, socioeconomic status, access to and use of health care services, maternal and newborn care, and general quality of life. Unintended consequences, such as corruption and sale of other bed nets by private individuals in the health sector, could also be investigated.
That notwithstanding, this study contributes to the expanse of literature detailing the efficacy of health interventions. The findings provide additional evidence of significant decreases in child mortality rates after the introduction of PMI. There is an inverse relationship between PMI funding and reduction of all-cause U5M. In addition, PMI has resulted in higher population coverage of ITNs, IRS, and ACTs, which is in the causal pathway of all-cause U5M. Global players need to make a concerted effort to sustain these gains and strive for malaria elimination, in line with the Global Technical Strategy 2016–2030. PMI’s role will continue to be critical, and the recent expansion of PMI’s support to four additional countries will accelerate progress toward eliminating malaria in SSA.
Supporting information
S1 Table. Data on coverage of artimisinin-based combination theraphies.
https://doi.org/10.1371/journal.pone.0217103.s001
(XLS)
S2 Table. Data on coverage of indoor residual spray.
https://doi.org/10.1371/journal.pone.0217103.s002
(XLS)
S3 Table. Data on coverage of insecticide treated nets.
https://doi.org/10.1371/journal.pone.0217103.s003
(XLS)
S5 Table. Data on mortality among under five children.
https://doi.org/10.1371/journal.pone.0217103.s005
(XLS)
S6 Table. Data on additional country level co-variates.
https://doi.org/10.1371/journal.pone.0217103.s006
(XLS)
References
- 1. Jakubowski A, Stearns SC, Kruk ME, Angeles G, Thirumurthy H. The US President’s Malaria Initiative and under-5 child mortality in sub-Saharan Africa: A difference-in-differences analysis. PLoS medicine. 2017;14(6):e1002319. pmid:28609442
- 2. Dieleman JL, Schneider MT, Haakenstad A, Singh L, Sadat N, Birger M, et al. Development assistance for health: past trends, associations, and the future of international financial flows for health. The Lancet. 2016;387(10037):2536–44.
- 3.
World Health Organization. World malaria report 2015. World Health Organization; 2016 Jan 30. [available on internet at http://www.who.int/malaria/publications/world-malaria-report-2015/report/en/]: 27 July 2018.
- 4. Yé Y, Eisele TP, Eckert E, Korenromp E, Shah JA, Hershey CL et al. Framework for evaluating the health impact of the scale-up of malaria control interventions on all-cause child mortality in sub-Saharan Africa. Am J Trop Med Hyg. 2017;97(3_Suppl):9–19. pmid:28990923
- 5. Hershey C, Florey LS, Ali D, Bennett A, Luhanga M, Mathanga DP, et al. Malaria control interventions protect against malaria parasitemia, severe anemia and all-cause mortality in children less than five years of age in Malawi, 2000–2010. Am J Trop Med Hyg. 2017;97(3_Suppl):76–88 pmid:28990920
- 6. Florey LS, Bennett A, Hershey C, Bhattarai A, Nielsen CF, Ali D, et al. Impact of Insecticide-Treated Net Ownership on All-Cause Under-Five Mortality in Malawi, 2006–2010. Am J Trop Med Hyg. 2017;97(3_Suppl):65–75 pmid:28990922
- 7. Eckert E, Florey LS, Tongren JE, Salgado SR, Rukundo A, Habimana JP, et al. Impact Evaluation of Malaria Control Interventions on Morbidity and All-Cause Child Mortality in Rwanda, 2000–2010. Am J Trop Med Hyg. 2017;97(3_Suppl):99–110. pmid:28990918
- 8. DuGoff EH, Schuler M, Stuart EA. Generalizing observational study results: applying propensity score methods to complex surveys. Health Serv Res. 2014 Feb;49(1):284–303. pmid:23855598
- 9.
The President’s Malaria Initiative: https://www.pmi.gov/about. Accessed 27 March 2019.
- 10.
Malaria Atlas Project: https://map.ox.ac.uk/ Accessed 27 March 2019.
- 11.
The Institute for Health Metrics and Evaluation (IHME). http://www.healthdata.org/ Accessed 27 March 2019.
- 12.
The World Bank: Governance Indicators data: http://info.worldbank.org/governance/wgi/index.aspx#home and World Development Indicators data can be obtained at: http://databank.worldbank.org/data/reports.aspx. Accessed 27 March 2019.
- 13. Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;73(1):13–22.
- 14. Harrison DA, Hulin CL. Investigations of absenteeism: Using event history models to study the absence-taking process. J Appl Psychol. 1989;74(2):300.
- 15. Sachs JD, McArthur JW. The millennium project: A plan for meeting the millennium development goals. The Lancet. 2005;365(9456):347
- 16. Houweling TA, Kunst AE, Looman CW, Mackenbach JP. Determinants of under-5 mortality among the poor and the rich: A cross-national analysis of 43 developing countries. Int J Epidemiol. 2005;34(6), 1257–1265. pmid:16159940
- 17. Hewko J. Foreign direct investment in transitional economies: Does the rule of law matter. E Eur Const Rev. 2002;11:71
- 18.
Lambsdorff JG. Causes and consequences of corruption: What do we know from a cross-section of countries? In Rose-Ackerman S. (editor), International handbook on the economics of corruption. Cheltenham, UK: Edward Elgar; 2006. pp. 3–51.
- 19. Blackwell M, Iacus S, King G, & Porro G. (2009). cem: Coarsened exact matching in Stata. Stata J. 9(4), 524–546.
- 20. Normand SLT, Landrum MB, Guadagnoli E, Ayanian JZ, Ryan TJ, Cleary PD, et al. Validating recommendations for coronary angiography following acute myocardial infarction in the elderly: A matched analysis using propensity scores. J Clin Epidemiol. 2001; 54(4):387–398. pmid:11297888
- 21. Iacus S, King G, Porro G. CEM: software for coarsened exact matching. J Stat Softw. 2009;30(13):1–27
- 22. Rosenbaum PR, Rubin DB. Assessing sensitivity to an unobserved binary covariate in an observational study with binary outcome. J R Stat Soc Series B Stat Methodol. 1983; ():212–218.
- 23. Eisele TP, Larsen D, Steketee RW. Protective efficacy of interventions for preventing malaria mortality in children in Plasmodium falciparum endemic areas. Int J Epidemiol. 2010;39(suppl_1):i88–i101.
- 24. Gimnig JE, Otieno P, Were V, Marwanga D, Abong’o D, Wiegand R, et al. The effect of indoor residual spraying on the prevalence of malaria parasite infection, clinical malaria and anemia in an area of perennial transmission and moderate coverage of insecticide treated nets in western Kenya. PLoS One. 2016 Jan 5;11(1): e0145282. pmid:26731524