1 Introduction

Forests provide valuable ecosystem goods and services of local and global significance. According to the latest forest resource assessment of the United Nations Food and Agriculture Organization, our global stock of natural forests continues to shrink, albeit at a slower annual rate than in the past [1]. Reduced deforestation rates may be the result of slower economic growth, decreasing demand for cleared land in urbanizing economies, or a sign that conservation policies are succeeding [2]. However, the global drop in rates of tropical tree cover loss is mostly driven by a few countries, such as Brazil. This inter-regional variation represents a major challenge for efforts towards achieving Aichi Target 5 and Sustainable Development Goal 15 on forests [3]. In the long term, our planet’s forests remain vulnerable to land use changes from increasing demand for agricultural and forest products [4–6].

Multiple policies and programs are being deployed to reduce tropical deforestation, mitigating climate change, and curbing biodiversity loss. Besides actions on forests already included in a number of intended nationally determined contributions to climate change mitigation (INDC), the Paris Agreement, in its Article 5, encourages Parties to the United Framework Convention on Climate Change to implement policy approaches and positive incentives to reduce emissions from deforestation and forest degradation. And yet, our knowledge about how to achieve forest conservation and related development goals is fragmented at best [7–10]. This PLOS ONE Collection contributes to building such a knowledge base and adds to the emerging literature on the effectiveness of conservation policies and measures with a focus on tropical and subtropical biomes.

Section 2 describes the geographic and methodological scope of this Collection, as well as the policy instruments covered in the Collection’s articles. As an open collection, we hope that additional articles will be added in the future. Section 3 synthesizes the main findings from the articles included in the Collection to date and Section 4 identifies potential future research directions.

2 Geographic Scope, Methodological Approaches, and Policy Instruments Covered in the Collection

The Collection as of March 2016 brings together 13 empirical studies covering eight countries across four continents (Fig 1 and Table 1). Four studies evaluate forest conservation policies in Brazil and each presents new insights that help explain the remarkable drop in Amazon deforestation over the past decade. Policies in Costa Rica and Indonesia are addressed by two contributions each, whereas Chile, Colombia, Mexico, Namibia, and Tanzania are covered by one study each.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Geographic scope of the Collection at publication date.

https://doi.org/10.1371/journal.pone.0159152.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Collection overview.

https://doi.org/10.1371/journal.pone.0159152.t001

In addition, two studies address methodological issues in the evaluation of conservation policies, one with a focus on payments for environmental services (PES) and one with a focus on defining appropriate spatial scales of analysis.

Table 1 summarizes the methodological approaches used in each contribution as well as the policies or interventions examined. Most studies use some form of matching analysis in their empirical strategies. All authors rely on quasi-experimental evaluation designs when evaluating the effectiveness of forest conservation interventions, either because these interventions do not lend themselves well to experimental evaluation (e.g. protected areas) or because data were obtained only after the policies were rolled out. In both cases, matching procedures have helped researchers identify more realistic control units upon which to develop a possible counterfactual scenario. Matching was also used as a preprocessing step to reduce model dependence in post-matching regression analysis by various studies [13, 14, 21, 25]. Miteva et al. [17] employ a matching-based triple difference estimator to exploit the three-period panel structure of their data.

In addition to estimating average treatment effects, post-matching regression analysis (including non-parametric regression techniques) served the purpose of robustness checks, as in Costedoat et al. [15], or of identifying heterogeneity in treatment effects, as in Shah and Baylis [23]. Pailler et al. [19] employ difference-in-difference regression directly. Cisneros et al. [14] study causal mechanisms behind the average treatment effect of a public disclosure initiative in Brazil, using panel data in a regression and matching-based empirical strategy [26]. Finally, Sills et al. [24] use a synthetic control approach [27] not previously applied to evaluate conservation initiatives.

Most studies in the Collection rely on remote sensing-based indicators of forest cover change to measure conservation effectiveness. Especially in humid tropical climates, such indicators are subject to measurement errors, for example as a result of persistent cloud cover. However, as multi-year remote sensing products measuring land cover change at global scale become increasingly available, new opportunities arise to assess the reliability of quasi-experimental evaluation techniques. Cisneros et al. [26], for example, use several years of pre-treatment observations to formally test for the parallel time trend assumption in their empirical strategy. Börner et al. [9] and Costedoat et al. [15] assess the sensitivity of their results to varying spatial resolutions and Börner et al. [13] find that treatment effects become insignificant at high spatial resolutions.

The policies and programs evaluated in the Collection range from regulatory disincentives and related enforcement mechanisms (e.g., protected areas, public disclosure, and field inspections) to incentive-based measures (e.g. PES and certification), and enabling institutional arrangements, such as jurisdictional support measures and community-based natural resource management [28]. Of these interventions, protected areas represent the most frequently studied forest conservation tool in the evaluation literature [29], whereas counterfactual-based evaluations of incentive-based conservation programs are only slowly emerging [30]. While a considerable amount of literature exists on community-based natural resource management, few study designs allow for statistically rigorous assessments of effectiveness [31]. The Collection contributes to filling such gaps in the evidence on the effectiveness of conservation measures.

3 Synthesis of Findings

Here we synthesize the key findings of the Collection papers in terms of broad instrument categories (see also Table 1 for effect sizes and related evaluation periods).

Forest conservation effectiveness

To compare the forest conservation effects across the eight studies that explicitly measure changes in forest cover, we compute effect sizes in terms of average annual change in forest cover (Fig 2), following the approach proposed by Puyravaud [34] and used by Samii et al. [30] to systematically compare effect sizes across a number of PES schemes. Effects on annual average percentage forest cover and the respective standard errors are calculated as: Eq 1 and Eq 2 where FC_T is mean forest cover in treated observation units, Δ is the estimated effect, and t₂-t₁ the number of years elapsed over the evaluation period. When studies do not report mean forest cover, it is imputed based on descriptive statistics or obtained directly from the authors.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Effects on average annual forest cover change compared.

Horizontal bars and values in brackets represent standard errors. Three letter abbreviations are UN country codes.

https://doi.org/10.1371/journal.pone.0159152.g002

Most studies report effects between 0 and 0.5 percentage points (Fig 2). This effect range corresponds well to that found by Samii et al. for selected PES programs in the tropics, i.e., 0.21 percentage points for studies that measured deforestation and 0.5–1.6% for studies looking at forest cover. Small effects are thus not necessarily a unique feature of PES programs, but instead seem to be a more general characteristic of tropical forest conservation programs. This can be partly explained by the intervention context in which such programs typically occur (see Persson and Alpizar [35] for a formal treatment of this issue). Since many forest conservation initiatives have a remote location bias, they tend to target a large amount of forest land that is not immediately threatened by deforestation. For many programs, it is thus not surprising to find that large shares of forest would have been conserved even in the absence of the intervention. To judge whether the intervention was worthwhile, we have to assess whether the value of the additional forest cover achieved by the program, whatever the amount, justifies the costs of the intervention. As of yet, few evaluations of forest conservation programs include cost-effectiveness assessments.

Two Collection studies report annual effects on forest cover change that are about one order of magnitude higher than the 0–0.5% effect range, i.e. Costedoat et al. [15] and Miteva et al. [17]. While these studies may indeed have evaluated genuinely more effective programs, they also differ from the other six studies in terms of study design and intervention context. Both studies evaluate forest cover change in spatial locations that represent actual decision units, i.e. communities (ejidos) in Mexico and villages in Indonesia. In the Mexican case, a large amount of forest remnants exhibited a relatively high risk of deforestation and in the Indonesian case all villages held forests under logging concessions, and thus, are predestined to some form of land cover change.

4 Future Research Directions

Our Collection overview is only a snapshot of the emerging literature using counterfactual-based evaluation to measure the effectiveness of forest conservation initiatives. This literature has a strong focus on protected areas [29], but also increasingly covers incentive-based conservation measures, such as PES, and enabling community support measures [30]. By allowing for the construction of observed rather than stated outcome measures, the increased availability of and improved access to remote sensing-based forest cover estimates over the past decade has clearly advanced this line of research.

Vis-à-vis the existing literature on the effectiveness of conservation policies, the new studies in our Collection point to some incipient lessons for future research:

1. Beware of location bias: Most conservation policy interventions are implemented in contexts that are not representative and thus suffer from selection bias. However, the direction of bias can change depending on the underlying intervention strategy. For example, several Collection papers show that protected areas tend to be located in remote locations, reflecting lower opportunity costs of land and reduced potential for conflicting land use interests [20]. In some cases, however, protected areas are also intentionally established in high pressure areas [36], leading to a bias in the opposite direction. If a forest conservation policy is being systematically implemented in above or below-average pressure contexts, securing internal validity of evaluations is not enough for us to learn about its potential effectiveness.
2. Carefully document intervention context: A host of factors including pre-program levels of compliance with intervention goals, policy design, and quality of implementation co-determine outcomes—potentially as strongly as the proper policy instrument choice (see also [35]). High environmental threats increase the scope for effective counteraction. Careful documentation of context factors and intervention design elements is thus paramount to making sense of comparative analyses within and across policy categories.
3. Cautiously interpret early systematic reviews: It is probably too early to derive general lessons on individual policy instruments such as attempted in recent systematic reviews, for example, on PES [30]. As the studies in this Collection show, the effectiveness of forest conservation instruments in the same category can vary by factor six in terms of effects on annual forest cover change (see Fig 2), with high levels of variation particularly between, but even within countries. Until the sources of this variability are better understood, and studies are available from a variety of contexts (see 2.), it is premature to draw generalizable, externally valid conclusions on the effectiveness of individual instruments.
4. Push methodological boundaries in quasi-experimental evaluation: Some Collection papers apply heterogeneous treatment effect analysis, or formally measure the contribution of individual causal mechanisms behind average treatment effects. Such analytical extensions require additional assumptions and more careful interpretation, but help us understand where, when, and why interventions work. Moreover, many papers in this Collection show that spatial factors play an important role in affecting the results of empirical analyses. As methods in spatial analysis and statistics are rapidly developing, new and more sophisticated empirical strategies will increasingly become available as ready-to-use software packages for conservation impact evaluation.
5. Explore options for randomization: Randomized control trials have been conducted to evaluate conservation management practices, but are virtually absent from the literature on conservation policy effectiveness at the time this Collection was conceptualized [10]. Not all conservation policy measures lend themselves to randomization, but oversubscription and randomized phase-in clearly represent feasible strategies to evaluate PES and community-based conservation initiatives. Randomization may seem especially appropriate when programs are to be rolled out on a larger scale. Moreover, even if the intervention cannot be entirely randomized, one may still be able to experimentally vary certain contextual conditions or design features of the program in order to evaluate the effectiveness of key mechanisms of the conservation policy according to its theory of change.
6. Do not forget intervention costs: Few studies evaluating conservation policy effectiveness, including in this Collection, factor in policy implementation costs as additional performance criterion. Ultimately, however, decision-makers will have to balance policy effectiveness against costs. Especially if conservation policy instruments are part of a much broader environmental policy strategy, quantification of instrument-specific opportunity and implementation cost (including initial investment needs as well as recurrent annual expenditure) can be a daunting task.

It is not enough to ask: “what works and what doesn’t?”. We also need to know where, when, and why forest conservation initiatives failed or worked, and at what cost. While impact evaluation is an important piece of this puzzle, it clearly has shortcomings that require other qualitative and quantitative research approaches to complete the picture [37]. However, learning from practice for the design of better interventions for conservation, with more cost-effective and equitable outcomes, requires impact evaluation to become an integral part of the policy research cycle, so as to inform theory development and ex-ante impact assessment [38].

Acknowledgments

This research was financially supported by the European Commission–Grant n° DCI-ENV/2011/269520 and the Robert Bosch Foundation; the CGIAR Research Program on Forests, Trees and Agroforestry (FTA); and European Association of Environmental and Resource Economists. We thank the participants of the International Workshop “Evaluating Forest Conservation Initiatives: New Tools and Policy Needs”, 10–12th December 2013 in Barcelona, Spain, for discussions and comments that have helped putting together this Collection.