http://orcid.org/0000-0001-7331-4201
Figures
Abstract
Multi-stakeholder platforms have become mainstream in projects, programmes and policy interventions aiming to improve innovation and livelihoods systems, i.e. research for development interventions in low- and middle-income contexts. However, the evidence for multi-stakeholder platforms’ contribution to the performance of research for development interventions and their added value is not compelling. This paper focuses on stakeholder participation as one of the channels for multi-stakeholder platforms’ contribution to the performance of research for development interventions, i.e. stakeholder participation. It uses a quantitative approach and utilizes descriptive statistics and ARIMA models. It shows that, in three Ugandan multi-stakeholder platform cases studied, participation increased both in nominal and in unique terms. Moreover, participation was rather cyclical and fluctuated during the implementation of the research for development interventions. The study also shows that, in addition to locational and intervention factors such as type of the area along a rural–urban gradient targeted by the intervention and human resources provided for multi-stakeholder platform implementation, temporal elements such as phases of research for development intervention objectives and the innovation development process play significant roles in influencing participation. The study concludes that contribution of multi-stakeholder platforms to the performance of research for development projects, programs, policies and other initiatives is constrained by locational and temporal context and conditional on the participation requirements of the objectives pursued by research for development intervention.
Citation: Sartas M, van Asten P, Schut M, McCampbell M, Awori M, Muchunguzi P, et al. (2019) Factors influencing participation dynamics in research for development interventions with multi-stakeholder platforms: A metric approach to studying stakeholder participation. PLoS ONE 14(11): e0223044. https://doi.org/10.1371/journal.pone.0223044
Editor: Khin Thet Wai, Ministry of Health and Sports, MYANMAR
Received: April 3, 2018; Accepted: September 12, 2019; Published: November 14, 2019
Copyright: © 2019 Sartas et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data files are available from Figshare: https://figshare.com/articles/Factors_Influencing_Stakeholder_Participation_Dynamics_in_Research_for_Development_Interventions_with_Multi-Stakeholder_Platforms_A_metric_approach_to_studying_stakeholder_participation/9808127 or the Researchgate database with DOI: 10.13140/RG.2.2.35252.14725 and url: https://www.researchgate.net/publication/324170487_Factors_Influencing_Stakeholder_Participation_Dynamics_in_Research_for_Development_Interventions_with_Multi-Stakeholder_Platforms_A_metric_approach_to_studying_stakeholder_participation.
Funding: This research was undertaken as part of, and funded by, the CGIAR Research Program on Integrated Systems for the Humid Tropics (Humidtropics – between 2013 and 2016) and the CGIAR Research Program on Roots, Tubers and Bananas (RTB – from 2017 onwards) and supported by CGIAR Fund Donors (http://www.cgiar.org/about-us/our-funders/). Additional funding support was provided by the Belgian Directorate General for Development Cooperation and Humanitarian Aid (DGD) through the Consortium for Improving Agricultural Livelihoods in Central Africa (CIALCA). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
1. Introduction
Multi-stakeholder platforms (MSPs) have been attracting increasing attention from different agencies aiming to improve innovation and livelihood systems (hereafter systems). MSPs have been utilized to facilitate different research for development interventions [R4Ds] such as government policies [1] and international research and development programmes and projects [2]. The literature on MSPs in livelihoods research covers a broad range of dimensions, including agriculture [3], natural resource management [4,5], environment management [6] and health [7]. Specific forms of MSPs include, among others, innovation platforms [3], public private partnerships [8,9], sustainability platforms [10] and learning alliances [11].
R4Ds aim to improve complex systems and require approaches and tools that enable holistic identification and analyses of both constraints and opportunities [12,13]. This implies that R4Ds need to respond to the needs and challenges faced by different stakeholder groups [14–16], making collective action and multidisciplinary, multi-stakeholder partnerships between research and development actors key for their performance [7,13,17]. MSPs have gained momentum in the R4D world, notably because of their ability to foster collective action across multi-disciplinary actors [6,7,18]. Their popularity is especially high in low- and middle-income countries where partially or non-participatory approaches have frequently been reported as insufficient to improve systems [17].
Although MSPs are popular, available evidence on their specific contribution to the performance and impact of R4Ds [hereafter MSPs’ contribution] has been scarce. MSPs have been reported to contribute to achieving objectives of R4Ds [7,19,20], but also to ‘failures’ [1,4,21,22]. Consequently, several studies have tried to identify the factors that influence MSPs’ contribution. Many of these studies converge around the role of several processes that influence MSPs’ contribution i.e. process drivers [23].
Among these process drivers, stakeholder participation [hereafter participation] has often been identified as a key indicator [4,24,25]. Stakeholders’ interest and participation in R4D events are considered to be related to the success of R4Ds [26,27]. In addition, MSPs are assumed to align participation to current activities of R4Ds [24,25]. In other words, in their ability to enhance higher and more compatible participation in R4Ds [28], MSPs are considered to increase the performance of R4Ds. Therefore, better understanding of participation in R4Ds when it is organized through MSPs has the potential to contribute to increasing the performance of R4Ds and scaling R4D innovations in low- and middle-income intervention contexts.
This study aims to contribute the literature by providing scarce quantitative evidence about MSPs’ contribution using descriptive statistics and time series approach. Existing studies on participation that uses a time series approach do not explicitly refer to MSPs and either have very limited in scope [29,30] or uses participation as an explanatory or independent factor rather than a dependent variable [31–36]. This study is one of the first studies that focuses on understanding participation dynamics and various factors contributing to these dynamics within R4Ds and investigates participation as the dependent variable.
2. The conceptual framework and methodology
2.1 R4D interventions and MSPs
The R4D literature has various definitions for R4Ds and MSPs. Based on a review we made for this study, we differentiate R4Ds with three major characteristics. First, systematic research plays a vital role, which is reflected on the allocation of an important share of R4D resources on systematic research [37,38]. Second, R4Ds utilize a systems approach in which multiple objectives and innovations are pursued and the interactions among and between objectives and innovations are explicitly articulated [37,39,40]. Third, R4Ds include participatory approaches [3,38,39,41] and multi-disciplinary teams [38,40,42,43].
Multi-stakeholder platforms [MSP] are defined as decision-making bodies [44] or round-tables where a diversity of stakeholders [5] get together to get things done [45]. This description is comprehensive and different approaches such as public private partnerships [8,9,46–49], innovation platforms [3,50–53] and sustainability platforms [10,54,55] are considered as MSPs. We characterize MSPs with several traits. First, they are not a single event but a series of interconnected events. Second, both research and non-research stakeholders are involved in and influence intervention decisions on different activities such as analysis of problems, design and implementation of research plan.
Therefore, R4D interventions organized though MSPs present a specific approach to implementation of system interventions and MSPs is a specific form of stakeholder involvement. Although, every R4D carries participatory elements, not all R4Ds have an MSP. In the R4D literature, there are various examples of interventions which are both R4Ds and MSPs, either R4D or MSP, or none [Table 1]. In this study, we focus on the intervention group which are both R4Ds and MSPs.
2.2 Definition of participation as a driver of R4D intervention performance
The relation between participation and using an MSP approach in R4Ds has been the subject of many studies [4,5,24,56]. However, the understanding of what participation means varies significantly [87]. For instance, participation has been defined as [1] simple attendance at R4Ds events [88], [2] continuous involvement in these events [89,90] or active influence on R4Ds’ agendas [60,88] etc. Different types of participation include passive participation, participation by consultation, functional participation, empowering participation and interactive participation [28,88].
In this paper, we use a simple definition of participation as referring to stakeholders’ physical attendance in diverse R4D events since it is the most common denominator of all the participation definitions we come across in the literature. Moreover, physical attendance is a prerequisite for more comprehensive definitions of participation. We are aware that studying participation in this definition might not be sufficient to cover diversity of definitions available in the literature, however such a definition is necessary part for all participation of definitions and inform studies on participation regardless of the specific definition they use. In this study, we further unpack participation and investigate it both in nominal terms–number of stakeholders–and in unique terms–number of distinct stakeholders. In addition, we investigate both non-cumulative and cumulative participation.
2.3 How does participation change?
Many studies have argued that participation changes when it is organized through MSPs [24,27,56,91], and the change is mostly in the form of an increase [26,27,92]. In addition, numerous studies have argued that, in R4Ds, MSPs go through several phases, whose details are discussed in section 2.4.3 [1,91,93–95]. These studies argue that participation increases as R4Ds advance through these phases.
2.4 Factors affecting participation
In the R4D literature, participation has been argued to depend on diverse factors. We classify them as [1] locational [96–98], [2] intervention-related [3,28,98] and [3] temporal [4,99–101] factors. In this paper, we study these three types of factors.
Among various factors in these three categories, we study seven specific factors in total, which are frequently discussed in the literature and that can be empirically investigated within the limitations of the resources provided for the study. We provide a list of other factors in supporting information [S1 File] to facilitate further research on the participation. One of the specific factors is locational [geographical location in which the R4D operates], three are intervention-related [funding, human resources provided by the R4Ds and type of event] and three are temporal factors [periods based on different R4D objectives, innovation development process and other time aspects].
2.4.1 Location-related factors.
The R4D literature includes many studies focusing on the location-related factors that influence stakeholder participation in R4Ds [93,97,102]. Comby et al. [93] argued that local communities’ interests were different from those of national and regional actors, and that consequently their participation in R4Ds is higher as these have a more direct impact on their livelihoods. Moreover, local media more frequently refer to specific local problems, attracting more widespread attention and interest in such issues. Raford [102] added that local actors’ deeper local expertise can increase the relevance of local actors for the R4Ds, and thus their participation. These arguments propose that stakeholder participation in R4Ds is higher at local level. In this paper, we investigate whether this proposition is valid in our cases to better understand the nature of participation and articulate the implications of using an MSP approach in R4Ds.
2.4.2 Intervention-related factors.
As with the locational-related factors, the literature includes multiple studies focusing on the intervention-related factors influencing participation. Examples include detail level of planning, i.e. clarity of impact pathways and theory of change [96], flexibility in implementation [98] and the type of organization managing the R4Ds project and/or MSP [3]. Among the intervention-related factors, three major aspects are more commonly instanced: [1] funding [4,28,96], [2] human resources [96,101,103,104] and [3] type of event [26,105].
It has been argued that the amount of funding provided to the MSPs increases participation by providing monetary incentives and improving the quality of MSP processes. In addition, allocated human resources have been argued to influence participation. The existence and quality aspects of champions who promote and improve the legitimacy of the MSPs [106–109], facilitators who manage communication and negotiation in the MSPs [16], organizers who arrange logistics and infrastructure [19], and documentation people who monitor and communicate R4D events, decisions and successes [3,16,19] have been especially argued to play important roles. Moreover, the type of R4D event is argued to influence participation. For instance, an implementation event that focuses on administrative and project management issues [e.g. financial planning meeting] typically includes few stakeholders, most of whom are employed by the organization managing the R4Ds, whereas a promotional event [product demonstration event] that aims to disseminate knowledge and innovations often includes many stakeholders.
2.4.3 Temporal factors.
The R4D literature discusses two major temporal factors that influence participation in R4Ds when MSPs are used. The first temporal factor is calendar-based periods. It is argued that participation increases or decreases in specific time periods. For instance, participation is argued to decrease in holiday periods [110,111] and increase in planting periods in R4Ds focusing on agriculture [112]. Consequently, the first factor we consider in the study is specific calendar-based periods, such as cultural events [e.g. festive seasons] and agricultural seasons. We will investigate several of such periods.
The second temporal factor is based on the time period R4Ds are implemented. Participation was discussed to increase as R4D implementation gets older [21,113, 114]. Our review on the temporal factors indicated two major specific commonly discussed ways of defining time periods, i.e. phasing. The first phasing is based on the changes in the R4Ds’ objectives and innovations. R4Ds have multiple objectives–such as improving productivity, decreasing malaria incidence–and work on different innovations that can serve these objectives, such as developing new higher yield seed varieties or new mosquito traps. The stakeholder configurations that best fit these objectives and innovations are different, and a change in these also causes change in participation. As R4Ds focus on specific objectives and innovations for a period [hereafter R4D objective phases] these periods influence participation.
The second phasing is based on the type of work done on a single innovation. In R4Ds, development of innovations goes through different periods. In general, initially, several potential innovations that can serve R4D objectives are discussed and prioritized. Later, R4Ds work on the prioritized innovations and improve them until they are disseminated and marketed to the stakeholders who can use them to improve their livelihoods. As these periods require the participation of different stakeholders to achieve different goals, a change in the periods influences stakeholder participation. In other words, the periods [hereafter, innovation development phases] influence participation.
In our review of temporal factors, we came across standard ways of capturing calendar-based periods and used a few of them. However, for periods of R4D objectives and innovation development phases, there are different arguments about how to define the phases. Therefore, we rapidly synthesized multiple literatures and used the conceptualizations articulated in the following paragraphs.
R4Ds aim to achieve multiple livelihood objectives such as improving food security, nutrition and health status of smallholder farmers, natural resource management, empowering women and youth. As targeting these objectives will typically require more resources than they have at their disposal, R4Ds often identify specific entry points to focus their activities. In addition, each of these objectives can be achieved through different innovations, leading to a process of prioritization and focusing.
The R4D literature suggests several ways to phasing for the time following the prioritization and focusing [27,69,93,94]. For this study, we synthesized two ways of phasing using the literature, i.e. R4D objectives and innovation development to study this time period. For the phases based on R4D objectives, we suggest a four-phase approach [Table 2]. The phases start with [1] an entry phase, when stakeholders prioritize and select the objectives and innovations on which the R4Ds will initially focus [56,99], followed by [2] a vertical progress phase, when stakeholders generate deeper understanding and ‘improve’ selected innovations [115], [3] a horizontal progress phase, when stakeholders focus on improving complementary innovations that are typically within the same value chain [103], or [4] a system progress phase, when stakeholders focus on improving the contribution of selected innovations to other objectives [67,116]. Depending on the MSP participants’ preferences, R4Ds might focus on vertical, horizontal or system progress.
For the phases based on innovation development literature suggested several options [1,91,95,117]. Whereas Markard and Truffer [118] identified multiple phases ranging from formation to market growth, Edquist [91] proposed three phases: innovation generation, diffusion and use. In policy interventions, a four-phase approach including generation of a promising innovation, showing a business case for it, adoption/adaption by first movers and widespread adoption has been proposed [1]. The stepwise technology development assumption suggested that an innovation process starts with an idea, continues with basic research and technology formulation, followed by applied research, prototyping and demonstration, and ends with early and full commercial application [117]. As these phases require different stakeholder capacities, their existence will imply different stakeholder configurations and different sizes of participation during R4Ds [25,28]. In brief, these phases of innovation development might affect participation. In this paper, we utilize the three phases suggested by Edquist [i.e. generation, diffusion and use] because of their simplicity and generic character. To better accommodate innovation development in R4Ds, we add an initial phase in which the MSP goes through a participatory process of prioritization of innovations, resulting in the four phases described in Table 3.
2.5 Methodological framework
2.5.1 Study context and study sites.
We studied three MSPs in Uganda that were implemented under the CGIAR Research Programme on Integrated Systems for the Humid Tropics [Humidtropics], between 2013 and end 2016. The programme covered four regions across the globe, one of which was East and Central Africa, including Uganda. Humidtropics aimed to improve livelihood systems by reducing rural poverty, increasing food security, improving nutrition and health, and strengthening the sustainable management of natural resources. To reach these targets, the programme focused on improving agricultural productivity, access to affordable food and consumption of nutritious foods, decreasing environmental harm from agriculture, and improving the innovation capacity of the local innovation systems [107]. The programme aimed to optimize these outcomes to improve smallholder farmers’ livelihoods.
The Humidtropics programme in Uganda was organized through three MSPs. One of them was organized in Kampala–the economic and cultural capital of Uganda–another one in Kiboga and Kyankwanzi Districts, 123 and 155 km northwest of Kampala respectively, and the third one in Wakiso and Mukono Districts, 16 and 24 km east of Kampala respectively [Fig 1]. The programme included a diverse set of events, including meetings, field visits, experiments, capacity-building workshops and promotion events. The implementation and management of the events were supported by teams hosted within the organizations managing Humidtropics. The events in three sites kicked off in August 2013 with a big event, after which they were officially established in February 2014. The implementation and management teams included [1] champions–influential people who were financially and conceptually supportive of the MSPs in the organization leading the programme in Uganda–and district opinion leaders, [2] facilitators, both at national and at district scale, [3] organization and monitoring staff employed by the intervention working across the programme and their local assistants.
2.5.2 Data collection.
Data for the study were collected throughout the period 2013 to 2016, using two short surveys from the Learning System for Agricultural Research for Development [LESARD] [19]. The first survey was a LESARD Participant Profile, which recorded the characteristics of individuals participating in the R4D events [S2 File]. The participant profile was administered to all stakeholders who participated more than once in the events. The questions included the stakeholders’ organizations, their professional background, expertise subjects, scales of operation and role in the value chain group. The second survey was a LESARD Event Log, which recorded information about all the events in the R4D intervention [S3 File]. This included formal MSP meetings, but also smaller informal meetings where a subset of stakeholders met to pursue specific tasks agreed upon by the MSPs, field visits, workshops and promotional events. It covered location, start and end dates, event type and organizations funding different expenses of the events, as well as the information tools and documentation applied to the event. For the variables related to event types, minutes and audio and video materials were used to validate the content of the event.
2.5.3 Data analysis.
Data analysis was organized in two ways, namely, descriptive statistics and linear [time-series] regression. Descriptive statistics were used to present changes in participation in nominal and in unique terms throughout the R4D intervention. Linear time-series regression was done using ARIMA [119] in SPSS v.23. Specifically, we used IBM SPSS Time Series Modeller Procedure. The procedure uses partial autocorrelation functions, identify autoregressive orders [p], differencing [d], moving average orders [q] and outliers and transforms independent variables automatically [120] and shown to be a very good substitute for manual modelling [121]. Significant model coefficients are reported in Section 3. Analysis of participatory observations provided additional insights that enabled interpretation and discussion of our results.
Our study included two different sets of variables, namely, participation and factors affecting participation [Table 4]. Participation was measured by counting the number of stakeholders at an event. To study factors affecting participation, we focused on seven major factors, i.e. [1] geographical location, [2] funding, [3] human resources for the management and implementation of the MSP, [4] MSP events, [5] the phases of R4D development, [6] the phases of the innovation development process and [7] other time aspects. Each of these factors was investigated by analysing relevant variables in Table 4.
We calculated rural/urban characteristics of the location by checking the administrative classification of the location provided as a response to the location question in the LESARD Event Log. Share of funding in overall and in specific terms were calculated by checking the responses given to the fund provision question in the Event Log. Each cost item, i.e. mobilization, transportation cost, food, daily allowance, event venue and facilitation, was given equal weight. Type of event was identified using the response given to multiple selection event type questions in the Event Log. Phases of periods based on different objective-innovation bundles and innovation development process were identified using the responses given to the event type question and validated with controlling the minutes and pictures of the events. Other time periods, i.e. production seasons and special breaks, were identified by the stakeholders within the social and geographical specifications of the locations in which the MSP was active.
3. Results
3.1 Participation and changes in participation
Participation in 411 R4D events ranged between a monthly average of 4 and 51. It presented similar trends within five different periods during the course of the R4D. In order to present the data in a more informative manner we named these periods from A to E and described the averages in each period. The periods are formulated using the observed visual patterns in participation plots; they are not results of any statistical inquiry. Nevertheless, the periods inform our inquiry on the changes in the participation especially in terms of the patterns the change in participation present.
The average participation up to the tenth month after MSP establishment [Period A] was 31 per month [Fig 2]. Between months 11 and 18 [Period B] and between months 22 and 26 [Period D], there was an increase in average participation. In Period B, the average monthly participation was 173, and it was 288 in Period D. Between months 19 and 21 [Period C] and from month 27 onwards [Period E], average participation per month was lower than in the preceding months, with averages of 157 and 112.
Nominal and unique participation in MSPs had similar patterns throughout the intervention period. Stable periods [A, C, E] were followed and preceded by periods of increase [B, D].
During the three years of implementation, 4767 nominal and 1123 unique stakeholders participated in the R4D. Nominal and unique participation presented similar patterns across the MSP periods. During Periods A, C and E, participation in the MSPs was mostly stable, except during months 6 and 31 [Fig 2]. During Periods B and D, participation increased. In Period B, the increase accelerated continuously, whereas in Period E it peaked in month 24. In the first half of Period A, participation stagnated.
3.2 Factors affecting participation and changes over time
Location-related, intervention-related and temporal factors were significant in affecting participation [Table 5]. In terms of location-related factors, participation was relatively lower in urban areas, medium in peri-urban areas and higher in rural areas. Regarding the intervention-related factors, human resources, i.e. participation of facilitation, organization and monitoring staff in the R4D events, were significant. Higher investments in human resources were positively correlated with stakeholder participation. Among temporal factors, phases of R4D objectives and innovation development were significant. Participation was higher in later phases of both temporal factors [Table 5].
Numbers in the third column represent the coefficients for statistically significant factors of participation with 0.01 [**] and 0.05 [*] confidence levels.
Among the intervention-related factors, [1] share of funding provided for the events by the R4D project, [2] share of specific intervention funding to different expenditures including mobilization, transportation cost, food, daily allowance, event venue, facilitation, [3] type of event based on detailed event type, [4] type of event based on research and delivery and [5] participation of champions were insignificant. Among temporal factors, [1] production season and [2] other special periods such as long holidays were insignificant.
4. Analysis and discussion
4.1 Is participation dynamic?
Our study confirms that MSP participation is dynamic, as found also in previous studies [24,27,56,91]. The participation dynamics presented a cyclical pattern in the cases that we investigated. Periods of stagnation in the number of new stakeholders were followed by periods of expansion [Fig 2]. Participation in both nominal and unique terms was mostly stable in Periods A, C and E and changing in Periods in B and D. These changes in participation are partially caused by the combined effect of the significant factors discussed later in detail in section 4.2 that led to these dynamics aspects. As the explanatory power of the model is 36% [Table 5], other important factors beyond the scope of this study contributed to the patterns observed in these periods.
Our study showed that total participation increased both in nominal and in unique terms. More than 1100 unique stakeholders participated in the three MSPs [366 per MSP], totalling approximately 4750 nominal participations [Fig 2]. This confirms that both nominal and unique participation during R4Ds can increase when it is organized through MSPs [69,101] although it does not imply any direct causality between using the MSPs and increase in participation. As these numbers reflect the stakeholders that participated in two events or more, the outreach of R4Ds that apply the MSP approach when participation is organized through MSPs is likely to be higher than the numbers reported in this study. However, in non-cumulative terms, participation did not show a trend. In other words, average participation in a month was not higher or lower in the later period of the intervention in comparison to the earlier period.
4.2 How was participation influenced by location-related, intervention-related and temporal factors?
In the MSPs studied, different location-related, intervention-related and temporal factors influenced participation.
4.2.1 Location-related factors.
The investigated locational factor, geographical location along a rural–urban gradient, was significantly related to participation [Table 5]. Participation was highest in Kiboga-Kyankwanzi, medium in Mukono-Wakiso and lowest in Kampala. As Kiboga and Kyankwanzi are rural districts far from Kampala, and Mukono-Wakiso is a relatively close peri-urban area [Fig 1], we can argue that rurality is a relevant aspect that makes location significant. The decrease in participation along the rural–urban gradient confirms the locality argument. In other words, the higher interest of the local communities and the local media based in the rural areas might lead to higher participation in rural areas. The decrease in participation along the rural–urban gradient might be also due to the availability of alternative options for stakeholders. Stakeholders might have more opportunities and demands to allocate their time in the urban locations in comparison to peri-urban and rural locations where these opportunities are scarcer [122–124]. In addition, relative higher relevance [125] and urgency [126,127] of the R4D activities for the livelihoods of the residents of rural areas, in which agriculture is more relevant in relative terms, might lead to high prioritization of MSP events to other alternatives for allocation time.
4.2.2 Intervention-related factors.
Our study also showed that human resource allocation to manage and implement the MSP was significantly correlated to participation [Table 4]. Among the specific human resource contributions, the availability of facilitation, organization and monitoring was significant. Although statistical relations do not necessarily imply causality, the results support the commonly reported important role of facilitation for participation [1]. Although not frequently referred to in the literature, organization and monitoring are mentioned as important in many events. For instance, a field researcher in the Mukono-Wakiso platform in month 13 mentioned that ‘Provisions of Mukono local administration for organizing of field and farmer visits helped to reach many farmers in different sub-counties’.
Funding provided by the R4D intervention [in this case Humidtropics] to events as a share of the total cost of the events was not correlated with participation. Similarly, specific funding for mobilizing stakeholders to participate in the event, transportation of the stakeholders to and from the venue, provision of a daily allowance and food to participating stakeholders, using the event venue and facilitation of the event were not significantly correlated with participation when the influence of confounding factors is removed. This result does not confirm the arguments of some earlier studies [4,28,96] regarding the positive relation between funding and participation. It also indicates that monetary incentives might not be sufficient to increase participation, as opposed to the arguments in some of the literature [14,128].
One reason why funding is not significant could be that the amount of funding provided by the R4Ds is not sufficient to cover the opportunity costs of participation. MSPs consist of a continuous set of activities, and influencing the MSPs’ agenda might necessitate participation in the R4D events multiple times. When combined, participation in multiple events can create a large demand on stakeholders’ time. The time spent on the MSPs could be used on many other activities that would give stakeholders more direct benefit. Earlier studies have indicated that this may be particularly relevant for private sector stakeholders [25].
Type of event organized by the R4Ds through MSPs was not significantly correlated with participation. Participation was not significantly different between research and delivery events. In addition, participation varied for all 12 event types over the course of the R4D interventions. The insignificant result implies that geographical location where the event was organized was more relevant for participation than the type of event. The same type of event might attract different participation in rural and urban areas. This was indeed the case in our study. On average, both research and delivery events attracted more participation in rural areas [18.5 and 24.2 persons per event, respectively] than in urban areas [6.6 and 6.7 persons per event, respectively]. Similarly, in 9 of the 10 events organized both in rural and in urban areas, participation was higher in rural areas.
4.2.3 Temporal factors.
In our study, calendar-based periods were not significantly related to participation. Production season and specific time periods like the December festive season were insignificant. However, both R4D objective and innovation development phases were significant [Table 5]. This confirms findings from the literature in which changing MSP participation over time is related to phases of R4D objectives and innovation development [1,91,93] and validates that work content based phasing can explain the changes in participation in R4Ds.
Although agriculture was central in our empirical cases, high dependency of agriculture in climatic seasons was not enough for making calendar-based period significant. In addition, considering that influence of calendar based periods in other R4Ds focusing on natural resource management, health and environmental dimensions of livelihoods would be less than agricultural dimensions presenting strong seasonality [129,130], it can be argued that MSPs can make participation in R4Ds more continuous.
5. Conclusion
Our study has shown that MSPs are conducive to increasing the cumulative number of stakeholders participating in the events of R4Ds. However, the increase is not valid for average terms. Moreover, locational and temporal factors might have a higher influence than intervention factors, implying that participation is more autonomous in nature than induced or influenced by the R4D intervention. Therefore, we conclude that MSPs’ contribution to the performance of R4Ds in increasing participation is constrained by “locational and temporal context” and conditional, depending on the duration of the participation necessary to achieve the R4D objectives. In other words, MSPs might not increase participation significantly in some locations and in some periods and R4Ds might not have the ability to increase participation in those places and times.
The findings in our study imply that different stakeholders can participate throughout the course of the R4D. However, MSPs are not effective in increasing average participation. Thus, if the R4D objective requires continued participation by a high number of stakeholders, such as enhancing collective action or supervising R4D events, MSPs’ contribution is limited. In addition, our study has shown that participation fluctuates or is cyclical. Therefore, the potential contribution of MSPs to R4Ds changes over time. Consequently, monitoring participation and scheduling R4D events based on participation might be necessary to make best use of the MSP approach.
We conclude that since the contribution of MSPs to the participation of stakeholders in R4D interventions are constrained, conditioned and limited, their use need to be contemplated critically, especially by R4Ds aiming transformative changes in different dimensions of livelihoods, e.g. agriculture, health, environment and natural resource management that require intense participation of diverse stakeholders. Consequently, other complementary tools and approaches need to be used strategically and tactically to increase participation in R4D projects, programs, policies and other initiatives.
In this study, we aimed to provide one of the early systematic quantitative evidence on the contribution of MSPs to the participation in R4D intervention and therefore used a metric approach. Since a metric approach requires reduction of the detailed information to identify and test standard measures, we could only partially complement our results using the rich qualitative data we accumulated. We intend to use this rich qualitative evidence to achieve a more complete picture and further improve the validity of the results in follow up research. Another potential study to improve the result of this study is to investigate broader aspects of participation and investigate how improving the complexity of the definition of participation influence the results. A third research endeavour that can improve the result of this study is to systematically explore the reasons why the participation followed the cyclical pattern presented by the results.
Supporting information
S1 File. Annex 1 List of Interventions.
A list of interventions, domains they operate and factors referred in explaining participation in the interventions.
https://doi.org/10.1371/journal.pone.0223044.s001
(DOCX)
S2 File. Annex 2 LESARD participant profile Uganda.
Characteristics of the participants are documented by using a short survey, LESARD Participant Profile.
https://doi.org/10.1371/journal.pone.0223044.s002
(DOCX)
S3 File. Annex 3 LESARD event log Uganda.
Characteristics of the events were documented by using a short survey, LESARD Event Log.
https://doi.org/10.1371/journal.pone.0223044.s003
(PDF)
Acknowledgments
The authors highly appreciate all multi-stakeholder platform facilitators and members who collaborated with us and provided data and insights necessary for this study. Special thanks to Solange Zawadi, Dieuwke Lamers, Rachael Zozo and Julie Lunzihirwa who supported data collection and analysis.
References
- 1. Beers PJ, Geerling-Eiff F. Networks as policy instruments for innovation. The Journal of Agricultural Education and Extension. 2014 Aug 8;20[4]:363–79.
- 2. Schut M, Soares NC, van de Ven G, Slingerland M. Multi-actor governance of sustainable biofuels in developing countries: The case of Mozambique. Energy Policy. 2014;65:631–43.
- 3. Schut M, Klerkx L, Sartas M, Lamers D, Campbell MM, Ogbonna I, et al. Innovation platforms: experiences with their institutional embedding in agricultural research for development. Ex Agric. 2016 Oct;52[04]:537–61.
- 4. Faysse N. Troubles on the way: An analysis of the challenges faced by multi‐stakeholder platforms. Nat Resour Forum. 2006;30[3]:219–29.
- 5. Warner J. Multi-stakeholder platforms: integrating society in water resource management? Ambiente & sociedade. 2006;8[2]:4–28.
- 6. Kaiser DB, Köhler T, Weith T. Knowledge management in sustainability research projects: Concepts, effective models, and examples in a multi-stakeholder environment. Applied Environmental Education & Communication. 2016 Jan 2;15[1]:4–17.
- 7. McHugh M, Shi Y, McClellan SR, Shortell SM, Fareed N, Harvey J, et al. Using multi-stakeholder alliances to accelerate the adoption of health information technology by physician practices. Healthc [Amst]. 2016 Jun;4[2]:86–91. pmid:27343156
- 8. Reypens C, Lievens A, Blazevic V. Leveraging value in multi-stakeholder innovation networks: A process framework for value co-creation and capture. Industrial Marketing Management. 2016 Jul;56:40–50.
- 9. Spielman DJ, Hartwich F, Grebmer K. Public–private partnerships and developing‐country agriculture: Evidence from the international agricultural research system. Public Administration and Development. 2010;30[4]:261–76.
- 10. Muñoz-Erickson TA, Cutts BB. Structural dimensions of knowledge-action networks for sustainability. Current Opinion in Environmental Sustainability. 2016 Feb;18:56–64.
- 11. Verhagen J, Butterworth J, Morris M. Learning alliances for integrated and sustainable innovations in urban water management. Waterlines. 2008 Apr;27[2]:116–24.
- 12. Bawden RJ. Systems approaches to agricultural development: The Hawkesbury experience. Agric Syst. 1992 Jan;40[1–3]:153–76.
- 13. Hall A, Clark N. What do complex adaptive systems look like and what are the implications for innovation policy? Journal of International Development: The Journal of the Development Studies Association. 2010;22[3]:308–24.
- 14. Foran T, Butler JRA, Williams LJ, Wanjura WJ, Hall A, Carter L, et al. Taking Complexity in Food Systems Seriously: An Interdisciplinary Analysis. World Dev. 2014 Sep;61:85–101.
- 15.
Martin A, Chancellor T, Posthumus H. A systematic review on the impacts of capacity strengthening of agricultural research systems for development and the conditions of success. London: EPPI-Centre, Social Science Research Unit, Institute of Education, University of London; 2013.
- 16.
Klerkx L, van Mierlo B, Leeuwis C. Evolution of systems approaches to agricultural innovation: concepts, analysis and interventions. In: Darnhofer I, Gibbon D, Dedieu B, editors. Farming Systems Research into the 21st Century: The New Dynamic. Dordrecht: Springer Netherlands; 2012. p. 457–83.
- 17. Norman DW. The farming systems approach: A historical perspective. International Farming Systems Association; 2002. p. 17–20.
- 18.
Hemmati M. Multi-stakeholder processes for governance and sustainability: beyond deadlock and conflict. Routledge; 2012.
- 19. Sartas M, Schut M, Leeuwis C. Learning System for Agricultural Research for Development Interventions [LESARD]—Effective Documenting, Reporting and Analysis of Performance Factors in Multistakeholder Processes. Integrated Systems Research for Sustainable Intensification of Smallholder Agriculture Ibadan: Earthscan. 2016.
- 20. Thiele G, Devaux A, Reinoso I, Pico H, Montesdeoca F, Pumisacho M, et al. Multi-stakeholder platforms for linking small farmers to value chains: evidence from the Andes. International Journal of Agricultural Sustainability. 2011;9[3]:423–33.
- 21. Biermann F, Man-san Chan AM, Pattberg P. 11. Multi-stakeholder partnerships for sustainable development: does the promise hold? Partnerships, governance and sustainable development: Reflections on theory and practice. 2007;239.
- 22. Turnhout E, Van Bommel S, Aarts MNC. How participation creates citizens: participatory governance as performative practice. Ecology and Society. 2010;15[4]:26–41.
- 23.
Sartas M. Do multi-stakeholder platforms work?: contributions of multi-stakeholder platforms to the performance of research for development interventions [Doctoral dissertation]. 2018.
- 24.
Amerasinghe P, Cofie OO, Labri TO, Drechsel P. Facilitating outcomes: multi-stakeholder processes for influencing policy change on urban agriculture in selected West African and South Asian cities. International Water Management Institute [IWMI].; 2013.
- 25. Lamers D, Schut M, Klerkx L, van Asten P. Compositional dynamics of multilevel innovation platforms in agricultural research for development. Sci and Pub Pol. 2017 Dec 1;44[6]:739–52.
- 26. Kongo VM, Kosgei JR, Jewitt GPW, Lorentz SA. Establishment of a catchment monitoring network through a participatory approach in a rural community in South Africa. Hydrology and Earth System Sciences. 2010;14[12]:2507–25.
- 27. Roloff J. Learning from Multi-Stakeholder Networks: Issue-Focussed Stakeholder Management. J Bus Ethics. 2008 Sep;82[1]:233–50.
- 28. Johnson NL, Lilja N, Ashby JA. Measuring the impact of user participation in agricultural and natural resource management research. Agricultural systems. 2003;78[2]:287–306.
- 29. García-Barrios LE, Speelman EN, Pimm MS. An educational simulation tool for negotiating sustainable natural resource management strategies among stakeholders with conflicting interests. Ecol Modell. 2008 Jan;210[1–2]:115–26.
- 30. Bowen S, Martens PJ. A model for collaborative evaluation of university-community partnerships. J Epidemiol Community Health. 2006 Oct;60[10]:902–7. pmid:16973540
- 31. Klintenberg P, Seely M. Land degradation monitoring in namibia: A first approximation. Environ Monit Assess. 2004 Jan;99[1–3]:5–21. pmid:15641368
- 32. Armstrong MJ, Payne AIL, Deas B, Catchpole TL. Involving stakeholders in the commissioning and implementation of fishery science projects: experiences from the U.K. Fisheries Science Partnership. J Fish Biol. 2013 Oct;83[4]:974–96. pmid:24090558
- 33. Curtis A, Coomber K, Droste N, Hyder S, Palmer D, Miller PG. Effectiveness of community-based interventions for reducing alcohol-related harm in two metropolitan and two regional sites in Victoria, Australia. Drug Alcohol Rev. 2017 Apr 20;36[3]:359–68. pmid:28429402
- 34. Wang L, Hu G, Zhou T. Semantic analysis of learners’ emotional tendencies on online MOOC education. Sustainability. 2018 Jun 8;10[6]:1921.
- 35. Ali MK, Wharam F, Kenrik Duru O, Schmittdiel J, Ackermann RT, Albu J, et al. Advancing Health Policy and Program Research in Diabetes: Findings from the Natural Experiments for Translation in Diabetes [NEXT-D] Network. Curr Diab Rep. 2018 Nov 20;18[12]:146. pmid:30456479
- 36. Cole CB, Pacca J, Mehl A, Tomasulo A, van der Veken L, Viola A, et al. Toward communities as systems: a sequential mixed methods study to understand factors enabling implementation of a skilled birth attendance intervention in Nampula Province, Mozambique. Reprod Health. 2018 Aug 3;15[1]:132. pmid:30075791
- 37. Delisle H, Roberts JH, Munro M, Jones L, Gyorkos TW. The role of NGOs in global health research for development. Health Res Policy Syst. 2005 Feb 21;3[1]:3. pmid:15723694
- 38.
Laws S, Harper C, Jones N, Marcus R. Research for development: A practical guide. Sage; 2013.
- 39. Abate T, Shiferaw B, Gebeyehu S, Amsalu B, Negash K, Assefa K, et al. A systems and partnership approach to agricultural research for development. Outlook Agric. 2011 Sep;40[3]:213–20.
- 40.
Ashby J. Introduction: uniting science and participation in the process of innovation–research for development. Managing natural resources for sustainable livelihoods. Routledge; 2013. p. 21–39.
- 41. Davis J, Whittington D. “Participatory” research for development projects: A comparison of the community meeting and household survey techniques. Economic Development and Cultural Change. 1998;47[1]:73–94.
- 42. Nuyens Y. Setting priorities for health research: lessons from low- and middle-income countries. Bull World Health Organ. 2007 Apr;85[4]:319–21. pmid:17546314
- 43. Rosenfield PL. The potential of transdisciplinary research for sustaining and extending linkages between the health and social sciences. Soc Sci Med. 1992 Dec;35[11]:1343–57. pmid:1462174
- 44. Steins NA, Edwards VM. Platforms for collective action in multiple-use common-pool resources. Agriculture and human values. 1999;16[3]:241–55.
- 45.
Röling NG, Woodhill AJ. From Paradigms to Practice: Foundations, Principles and Elements for Dialogue, Water, Food and the Environment. the Consortium Dialogue on Water, Food and the Environment; 2001.
- 46. Abbott KW. Engaging the public and the private in global sustainability governance. Int Aff. 2012 May;88[3]:543–64.
- 47. Eggersdorfer M, Bird JK. How to Achieve Transparency in Public-Private Partnerships Engaged in Hunger and Malnutrition Reduction. World Rev Nutr Diet. 2016 May 19;115:224–32. pmid:27198563
- 48. Hall A. Public-private sector partnerships in an agricultural system of innovation: concepts and challenges. International Journal of Technology Management & Sustainable Development. 2006;5[1]:3–20.
- 49. Yildirim O, Gottwald M, Schüler P, Michel MC. Opportunities and Challenges for Drug Development: Public-Private Partnerships, Adaptive Designs and Big Data. Front Pharmacol. 2016 Dec 6;7:461. pmid:27999543
- 50.
Dror I, Cadilhon J-J, Schut M, Misiko M, Maheshwari S. Innovation platforms for agricultural development: evaluating the mature innovation platforms landscape. Routledge; 2015.
- 51. Hermans F, Sartas M, van Schagen B, van Asten P, Schut M. Social network analysis of multi-stakeholder platforms in agricultural research for development: Opportunities and constraints for innovation and scaling. PLoS ONE. 2017 Feb 6;12[2]:e0169634. pmid:28166226
- 52. Sanyang S, Taonda SJ-B, Kuiseu J, Coulibaly N, Konaté L. A paradigm shift in African agricultural research for development: the role of innovation platforms. International Journal of Agricultural Sustainability. 2016 Apr 2;14[2]:187–213.
- 53. Tenywa MM, Rao KPC, Tukahirwa JB, Buruchara RA, Adekunle AA, Mugabe J, et al. Agricultural innovation platform as a tool for development-oriented research: Lessons and challenges in the formation and operationalization. World Applied Sciences. 2011;16[7]:981–8.
- 54.
Zuurbier PJP. Sustainability and certification in the biofuels sector: some critical observations. Wageningen University; 2010.
- 55. Kachel U, Jennings G. Exploring tourists’ environmental learning, values and travel experiences in relation to climate change: A postmodern constructivist research agenda. Tourism and Hospitality Research. 2010;10[2]:130–40.
- 56. Adekunle AA, Fatunbi AO. Approaches for setting-up multi-stakeholder platforms for agricultural research and development. World Appl Sci J. 2012;16[7]:981–8.
- 57. Giuliani E. Network dynamics in regional clusters: Evidence from Chile. Research Policy. 2013 Sep;42[8]:1406–19.
- 58. Bäckstrand K. Multi-stakeholder partnerships for sustainable development: rethinking legitimacy, accountability and effectiveness. Eur Env. 2006 Sep;16[5]:290–306.
- 59. Derak M, Taiquib L, Aledoc A, Cortina J. Similarities in stakeholder identification of restoration targets in a semiarid area. J Arid Environ. 2016 May;128:30–9.
- 60. Reed M, Evely AC, Cundill G, Fazey IRA, Glass J, Laing A, et al. What is social learning? Ecology and society. 2010;
- 61. Hermans FLP, Haarmann WMF, Dagevos JFLMM. Evaluation of stakeholder participation in monitoring regional sustainable development. Reg Environ Change. 2011 Dec;11[4]:805–15.
- 62. Hämäläinen RP, Kettunen E, Ehtamo H, Marttunen M. Evaluating a Framework for Multi-Stakeholder Decision Support in Water Resources Management. Group Decis Negot. 2001;10[4]:331–53.
- 63. Prell C, Hubacek K, Reed M. Stakeholder analysis and social network analysis in natural resource management. Soc Nat Resour. 2009 Jun 4;22[6]:501–18.
- 64. Walker B, Sayer J, Andrew NL, Campbell B. Should enhanced resilience be an objective of natural resource management research for developing countries? Crop Sci. 2010;50[Supplement_1]:S-10.
- 65. Kasonde JM, Campbell S. Creating a Knowledge Translation Platform: nine lessons from the Zambia Forum for Health Research. Health Res Policy Syst. 2012 Oct 3;10:31. pmid:23034056
- 66. Whitworth JAG, Kokwaro G, Kinyanjui S, Snewin VA, Tanner M, Walport M, et al. Strengthening capacity for health research in Africa. Lancet. 2008 Nov 1;372[9649]:1590–3. pmid:18984193
- 67. Barlow J, Bayer S, Curry R. Implementing complex innovations in fluid multi-stakeholder environments: Experiences of ‘telecare.’ Technovation. 2006 Mar;26[3]:396–406.
- 68. Bebbington A, Farrington J. Governments, NGOs and agricultural development: Perspectives on changing inter‐organisational relationships. Journal of Development Studies. 1993 Jan;29[2]:199–219.
- 69. Roloff J. A life cycle model of multi-stakeholder networks. Bus Ethics Eur Rev. 2008 Jul;17[3]:311–25.
- 70. de Zeeuw H. Cities farming for the future—multi-stakeholder policy formulation and action planning on urban agriculture in developing countries. Acta Hortic. 2010 Nov;[881]:97–109.
- 71. Fleury P, Petit S, Dobremez L, Schermer M, Kirchengast C, De Ros G, et al. Implementing sustainable agriculture and rural development in the european alps. Mt Res Dev. 2008 Aug;28[3/4]:226–32.
- 72. Thompson GD, Dutton R, Sparks TC. Spinosad–a case study: an example from a natural products discovery programme. Pest Management Science: formerly Pesticide Science. 2000;56[8]:696–702.
- 73. Pretty JN, Noble AD, Bossio D, Dixon J, Hine RE, Penning de Vries FWT, et al. Resource-Conserving Agriculture Increases Yields in Developing Countries. Environ Sci Technol. 2006 Feb;40[4]:1114–9. pmid:16572763
- 74. Bosher L, Dainty A, Carrillo P, Glass J, Price A. Attaining improved resilience to floods: a proactive multi‐stakeholder approach. Disaster Prev and Management. 2009 Feb 20;18[1]:9–22.
- 75. Meyer S, Glaser B, Quicker P. Technical, economical, and climate-related aspects of biochar production technologies: a literature review. Environ Sci Technol. 2011 Nov 15;45[22]:9473–83. pmid:21961528
- 76. Saysel AK, Barlas Y, Yenigün O. Environmental sustainability in an agricultural development project: a system dynamics approach. J Environ Manage. 2002 Mar;64[3]:247–60. pmid:12040958
- 77.
Warner J. The beauty of the beast: Multi-stakeholder participation for integrated catchment management. Multi-stakeholder platforms for integrated water management. Routledge; 2016. p. 17–36.
- 78. Fliervoet JM, van den Born RJG, Meijerink SV. A stakeholder’s evaluation of collaborative processes for maintaining multi-functional floodplains: a Dutch case study. International Journal of River Basin Management. 2017 Apr 3;15[2]:175–86.
- 79. Agarwal B. Participatory exclusions, community forestry, and gender: an analysis for south asia and a conceptual framework. World Dev. 2001 Oct;29[10]:1623–48.
- 80. Magesa SM, Lengeler C, deSavigny D, Miller JE, Njau RJA, Kramer K, et al. Creating an “enabling environment” for taking insecticide treated nets to national scale: the Tanzanian experience. Malar J. 2005 Jul 22;4:34. pmid:16042780
- 81. Yasuoka J, Levins R. Impact of deforestation and agricultural development on anopheline ecology and malaria epidemiology. Am J Trop Med Hyg. 2007 Mar;76[3]:450–60. pmid:17360867
- 82. Berti PR, Krasevec J, FitzGerald S. A review of the effectiveness of agriculture interventions in improving nutrition outcomes. Public Health Nutr. 2004 Aug;7[5]:599–609. pmid:15251050
- 83. Huang X, Faysse N, Ren X. A multi-stakeholder platform involving a mining company and neighbouring villages in China: Back to development issues. Resources Policy. 2017 Mar;51:243–50.
- 84. Mayangsari L, Novani S. Multi-stakeholder co-creation Analysis in Smart city Management: An Experience from Bandung, Indonesia. Procedia Manufacturing. 2015;4:315–21.
- 85. Beall J, Todes A. Gender and integrated area development projects: lessons from Cato Manor, Durban. Cities. 2004 Aug;21[4]:301–10.
- 86. Balan V, Chiaramonti D, Kumar S. Review of US and EU initiatives toward development, demonstration, and commercialization of lignocellulosic biofuels. Biofuels, Bioproducts and Biorefining. 2013;7[6]:732–59.
- 87. Badibanga T, Ragasa C, Ulimwengu J. Assessing the effectiveness of multistakeholder platforms: agricultural and rural management councils in the Democratic Republic of the Congo. Intl Food Policy Res Inst; 2013.
- 88. Leeuwis C. Reconceptualizing participation for sustainable rural development: towards a negotiation approach. Development and change. 2000;31[5]:931–59.
- 89. Sumberg J. Systems of innovation theory and the changing architecture of agricultural research in Africa. Food policy. 2005;30[1]:21–41.
- 90. Cook T, Boote J, Buckley N, Vougioukalou S, Wright M. Accessing participatory research impact and legacy: developing the evidence base for participatory approaches in health research. Educational Action Research. 2017 Aug 8;25[4]:473–88.
- 91. Edquist C. Systems of innovation perspectives and challenges. African Journal of Science, Technology, Innovation and Development. 2010;2[3]:14–45.
- 92. Richardson J, Grose J. An action learning approach to partnership in community development: a reflection on the research process. Action Learning: Research and Practice. 2013 Nov;10[3]:254–63.
- 93. Comby E, Le Lay Y-F, Piégay H. The achievement of a decentralized water management through stakeholder participation: An example from the Drome River catchment area in France [1981–2008]. Environmental management. 2014;54[5]:1074–89. pmid:25257044
- 94. Nikitina E, Ostrovskaya E, Fomenko M. Towards better water governance in river basins: some lessons learned from the Volga. Regional Environmental Change. 2010;10[4]:285–97.
- 95. Verma D, Ramirez-Marquez J. From TRL to SRL: The concept of systems readiness levels. boardmansauser.com; 2006.
- 96. Aw-Hassan AA. Strategies for out-scaling participatory research approaches for sustaining agricultural research impacts. Development in Practice. 2008;18[4–5]:564–75.
- 97. Hale TN, Mauzerall DL. Thinking globally and acting locally: can the johannesburg partnerships coordinate action on sustainable development? The Journal of Environment & Development. 2004 Sep;13[3]:220–39.
- 98. Joy KJ, Paranjape S, Kulkarni S. Multi-stakeholder participation, collaborative policy making and water governance: The need for a normative framework. Governance of Water: Institutional Alternatives and Political Economy. 2008;269.
- 99. Head BW. Assessing network-based collaborations: effectiveness for whom? Public Management Review. 2008;10[6]:733–49.
- 100. Hekkert MP, Suurs RAA, Negro SO, Kuhlmann S, Smits REHM. Functions of innovation systems: A new approach for analysing technological change. Technol Forecast Soc Change. 2007 May;74[4]:413–32.
- 101. Zornes D, Ferkins L, Piggot-Irvine E. Action research networks: role and purpose in the evaluation of research outcomes and impacts. Educational Action Research. 2016 Jan 2;24[1]:97–114.
- 102. Raford N. Online foresight platforms: Evidence for their impact on scenario planning & strategic foresight. Technol Forecast Soc Change. 2015 Aug;97:65–76.
- 103. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q. 2004;82[4]:581–629. pmid:15595944
- 104. Hall A, Rasheed Sulaiman V, Clark N, Yoganand B. From measuring impact to learning institutional lessons: an innovation systems perspective on improving the management of international agricultural research. Agric Syst. 2003 Nov;78[2]:213–41.
- 105. Lamb JN, Moore KM, Norton J, Omondi EC, Laker-Ojok R, Sikuku DN, et al. A social networks approach for strengthening participation in technology innovation: lessons learnt from the Mount Elgon region of Kenya and Uganda. International journal of agricultural sustainability. 2016;14[1]:65–81.
- 106.
Boogaard BK, Schut M, Klerkx L, Leeuwis C, Duncan AJ, Cullen B. Critical issues for reflection when designing and implementing research for development in innovation platforms. Wageningen, The Netherlands: Wageningen UR; 2013.
- 107.
CGIAR. Proposal for integrated systems for Humid Tropics Research Program. 2012 [cited 2018 Dec 12]; Available from: http://library.cgiar.org/bitstream/handle/10947/2554/crp_1.2_humid_tropics_proposal_jan24_2012.pdf?sequence=1
- 108. Fazey I, Bunse L, Msika J, Pinke M, Preedy K, Evely AC, et al. Evaluating knowledge exchange in interdisciplinary and multi-stakeholder research. Global Environmental Change. 2014 Mar;25:204–20.
- 109. Kilelu CW, Klerkx L, Leeuwis C. Unravelling the role of innovation platforms in supporting co-evolution of innovation: Contributions and tensions in a smallholder dairy development programme. Agricultural systems. 2013;118:65–77.
- 110. Barrientos S, McClenaghan S, Orton L. Stakeholder participation, gender, and codes of conduct in South Africa. Development in Practice. 2001;11[5]:575–86.
- 111. Aaltonen K, Jaakko K, Tuomas O. Stakeholder salience in global projects. International Journal of Project Management. 2008 Jul;26[5]:509–16.
- 112. Assaye A, Melak A, Ayalew B, Teshale D, Mazengia Y. Assessment of seed systems in North Western Ethiopia; with special emphasis on community based seed multiplication scheme. World Scientific News. 2015;12:100–10.
- 113. Bizikova L, Nijnik M, Kluvanková-Oravská T. Sustaining multifunctional forestry through the developing of social capital and promoting participation: A case of multiethnic mountain communities. Small-scale Forestry. 2012 Sep;11[3]:301–19.
- 114.
Payne SL, Calton JM. Towards a managerial practice of stakeholder engagement: Developing multi-stakeholder learning dialogues. Unfolding stakeholder thinking. Routledge; 2017. p. 121–35.
- 115. Akpo E, Crane TA, Vissoh PV, Tossou RC. Co-production of Knowledge in Multi-stakeholder Processes: Analyzing Joint Experimentation as Social Learning. The Journal of Agricultural Education and Extension. 2015 Aug 8;21[4]:369–88.
- 116. Svendsen AC, Laberge M. Convening stakeholder networks: A new way of thinking, being and engaging. Journal of Corporate Citizenship. 2005;[19]:91–104.
- 117.
European Commission. Horizon 2020 work programme 2014–2015. 19 General annexes revised. 2014;
- 118. Markard J, Truffer B. Technological innovation systems and the multi-level perspective: Towards an integrated framework. Research Policy. 2008 May;37[4]:596–615.
- 119. Wei WW. Time series analysis. The Oxford Handbook of Quantitative Methods in Psychology: Vol 2. 2006.
- 120.
IBM Corporation. IBM SPSS Forecasting. IBM; 2012.
- 121. Yang H. The case for being automatic: introducing the automatic linear modeling [LINEAR] procedure in SPSS statistics. Multiple Linear Regression Viewpoints. 2013;39[2]:27–37.
- 122. Chambers R. The origins and practice of participatory rural appraisal. World Dev. 1994 Jul;22[7]:953–69.
- 123. Tacoli C. Rural-urban interactions: a guide to the literature. Environ Urban. 1998 Apr;10[1]:147–66.
- 124. North D, Smallbone D. Innovative activity in SMEs and rural economic development: Some evidence from England. European Planning Studies. 2000;8[1]:87–106.
- 125. Eloy L, Bilbao B A., Mistry J, Schmidt IB. From fire suppression to fire management: Advances and resistances to changes in fire policy in the savannas of Brazil and Venezuela. Geogr J. 2019 Mar;185[1]:10–22.
- 126. Carter SE, Currie-Alder B. Scaling-up natural resource management: insights from research in Latin America. Dev Pract. 2006 Apr;16[2]:128–40.
- 127. Benjaminy S, Macdonald I, Bubela T. “Is a cure in my sight?” Multi-stakeholder perspectives on phase I choroideremia gene transfer clinical trials. Genet Med. 2014 May;16[5]:379–85. pmid:24071795
- 128. Gupta S. Worlds apart? Challenges of multi-agency partnership in participatory watershed development in Rajasthan, India. Development Studies Research An Open Access Journal. 2014;1[1]:100–12.
- 129. Sautier M, Piquet M, Duru M, Martin-Clouaire R. Exploring adaptations to climate change with stakeholders: A participatory method to design grassland-based farming systems. Journal of environmental management. 2017;193:541–50. pmid:28262418
- 130. Ohe Y. Evaluating the complementarity of the educational function in agriculture. WIT Transactions on Ecology and the Environment. 2010;131:247–55.