http://orcid.org/0000-0003-4011-8217
Figures
Abstract
Background
Quality Improvement (QI) approaches are increasingly used to bridge the quality gap in maternal and newborn care (MNC) in Sub Saharan Africa. Health workers typically serve as both recipients and implementers of QI activities; their understanding, motivation, and level of involvement largely determining the potential effect. In support of efforts to harmonise and integrate the various QI approaches implemented in parallel in Tanzanian health facilities, our objective was to investigate how different components of a collaborative QI intervention were understood and experienced by health workers, and therefore contributed positively to its mechanisms of effect.
Materials and methods
A qualitative process evaluation of a collaborative QI intervention for MNC in rural Tanzania was carried out. Semi-structured interviews were conducted with 16 health workers in 13 purposively sampled health facilities. A deductive theory-driven qualitative content analysis was employed using the integrated Promoting Action on Research Implementation in Health services (i-PARIHS) framework as a lens with its four constructs innovation, recipients, facilitation, and context as guiding themes.
Results
Health workers valued the high degree of fit between QI topics and their everyday practice and appreciated the intervention’s comprehensive approach. The use of run-charts to monitor progress was well understood and experienced as motivating. The importance and positive experience of on-site mentoring and coaching visits to individual health facilities was expressed by the majority of health workers. Many described the parallel implementation of various health programs as a challenge.
Conclusion
Components of QI approaches that are well understood and experienced as supportive by health workers in everyday practice may enhance mechanisms of effect and result in more significant change. A focus on such components may also guide harmonisation, to avoid duplication and the implementation of parallel programs, and country ownership of QI approaches in resource limited settings.
Citation: Baker U, Petro A, Marchant T, Peterson S, Manzi F, Bergström A, et al. (2018) Health workers' experiences of collaborative quality improvement for maternal and newborn care in rural Tanzanian health facilities: A process evaluation using the integrated 'Promoting Action on Research Implementation in Health Services' framework. PLoS ONE 13(12): e0209092. https://doi.org/10.1371/journal.pone.0209092
Editor: Tanya Doherty, Medical Research Council, SOUTH AFRICA
Received: July 19, 2018; Accepted: November 28, 2018; Published: December 19, 2018
Copyright: © 2018 Baker et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data are available from the Ifakara Health Institute (IHI) Institutional Review Board for researchers who meet the criteria for access to confidential data. Requests can be sent to Dr. Mwifadhi Mrisho, Chairman of the Ifakara Health institute Institutional Review Board (IRB) by email: mmrisho@ihi.or.tz.
Funding: The study was funded by the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement no. 265827 (TM, SP, FM, CH). UB also received funding for her salary from Karolinska Institutet and Stockholm County Council, Sweden. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Abbreviations: EQUIP, Expanded Quality Management Using Information Power; HW, Health Worker; i-PARIHS, integrated Promoting Action on Research Implementation in Health services framework; PDSA, cycles–Plan Do Study Act cycles; QI, Quality Improvement; QIT, Quality Improvement Team
Introduction
The quality gap is recognised as a critical limiting factor in accelerating the reduction of maternal and newborn deaths in Sub Saharan Africa [1–3]. This gap implies that while an unprecedented proportion of women seek care for themselves and their newborns during pregnancy, childbirth and the postpartum period, the content of care received is often of insufficient quality to have a significant impact on mortality and morbidity [2, 4].
Various approaches to quality improvement (QI) are used to address this quality gap in Sub Saharan African countries and include for example the “5 S”, “Standards-Based Management and Recognition” and collaborative QI using “Plan-Do-Study-Act” (PDSA) cycles, introduced into routine health care settings [5–9]. These approaches can be seen as implementation interventions, in that they aim to increase the use of existing knowledge and its implementation in practice [10]. Health workers typically serve as both recipients and implementers of these interventions; their understanding, motivation, and level of involvement therefore largely determining their potential impact [11]. In the context of limited resources, where there is a severe lack of skilled health workers and conditions for care provision are unpredictable, the implementation of such interventions may be challenging [12–15].
In Tanzania, the application of QI approaches has been spearheaded by HIV/AIDS programs and in more recent years also used by programs aimed to improve maternal and newborn care [16–19]. The various approaches promote distinct methodologies while often having several components in common [6–8]. In practice, different QI approaches may be implemented in parallel in the same district causing duplication, inefficiency and at worst, confusion among health workers [16]. This uncoordinated parallel implementation may impact negatively on the health system’s absorptive capacity, its ability to learn and incorporate new practices, and therefore potentially limit the positive effects of implemented interventions [20]. To share experiences and best practice, a National Quality Improvement Forum has therefore been established with the vision to harmonise the different QI approaches implemented in Tanzania [16]. To support this agenda, more evidence is needed on the role of the different components of QI approaches in this context: which aspects are understood and fulfil the perceived needs of health workers in every day practice and therefore contribute positively to the mechanisms of effect, i.e. how the QI interventions produce change [21]?
The EQUIP (Expanded Quality Management Using Information Power) was a collaborative QI intervention targeting community, health facility and district levels, implemented in one rural district in Tanzania and one rural district in Uganda in 2011–2014 [17, 22]. The aim of EQUIP was to increase coverage and quality of a number of essential evidence-based interventions for maternal and newborn care as outlined by WHO and partners [23]. Examples of interventions included promotion of mothers’ preparedness for birth and administration of Oxytocin within 1 minute of childbirth. The latter was one of four primary outcomes for which the outcome evaluation showed a positive effect: an increase of 26% and 8% of the proportion of mothers receiving this intervention in Tanzania and Uganda respectively [24]. This was achieved despite significant contextual challenges, the most notable being poor readiness of health facilities in terms of lack of drugs and equipment and also the project’s limited implementation strength [14, 15, 24, 25]. In Tanzania, improvements were also seen in two locally identified improvement topics: mothers’ preparation of clean birth kits and the frequency of district supervision of lower level health facilities [24].
In this study, we report on a qualitative process evaluation conducted to gain a deeper understanding of how the EQUIP intervention worked in rural health facilities in Tandahimba district in Tanzania. Utilising the recently published integrated Promoting Action on Research Implementation in Health Services (i-PARIHS) framework [26] as a lens during analysis, our objective was to investigate how the different components of this collaborative QI intervention were understood and experienced by health workers and therefore, contributed to the mechanisms of effect. The perspective of health workers was chosen in recognition of their central role as both recipients and implementers of the EQUIP intervention.
Materials and methods
Study design
This study was a qualitative process evaluation of the EQUIP intervention in health facilities, applying the i-PARIHS framework as a lens during analysis [21, 26].
The EQUIP intervention in health facilities: logic model and analytical framework
The EQUIP intervention and results have previously been described in detail [17, 24]. Here, we provide a summary of the health facility component with a more detailed description in S1 File.
EQUIP was modelled on the Institute for Healthcare Improvement’s (IHI) Breakthrough series for collaborative QI, an approach which includes seven elements envisaged to work in synergy to achieve improvement [22]. A schematic logic model, describing the components of the EQUIP intervention, its hypothesised mechanisms of effect and intended outcomes, is outlined in Fig 1. The same figure also illustrates how the elements of collaborative QI and the constructs of the i-PARIHS framework relate to the EQUIP intervention.
The grey-shaded boxes contain the core intervention components [17], with arrows representing the relationship between these. Hypothesised mechanisms of effect are contained within the dashed boxes and the intended outcomes in the red-shaded box. On either side of the logic model are mapped the constructs of the i-PARIHS framework [26] and the seven elements of Collaborative QI which are mapped according to their numbering in the paper by Kilo et al [22]. The red dashed square illustrates the focus for this process evaluation.
The i-PARIHS framework poses four constructs involved in implementation, the characteristics of which will determine its success. These constructs include the innovation to be implemented, the recipients of this innovation and the context in which the innovation is introduced. Facilitation, the core construct, is widely defined as a process of enabling, helping or making something easier and is considered to activate implementation through its interaction with the other three constructs [26]. In essence, successful implementation is more likely where an innovation, e.g., a clinical guideline or a triage system, is clearly defined, has a high degree of fit but at the same time a comparative advantage to existing practice [26, 27]. Recipients need to be motivated, have a sufficient level of skills, knowledge, and authority to engage with the innovation in a context with supportive leadership and sufficient resources [28]. The method of facilitation should ensure a high level of participation of key stakeholders, be integrated and iterative and sensitive to the context where it is introduced [28].
Applying the i-PARIHS constructs, the innovation introduced by EQUIP can be conceptualised as the use of Plan-Do-Study-Act (PDSA) cycles to structure problem solving and testing change ideas for prioritised improvement topics and tracking progress over time using run-charts. Although the aim of EQUIP was to improve the care provided to mothers and newborns, the primary recipients of the innovation were health workers of different cadres, organised into Quality Improvement Teams (QITs) in the 32 public and faith-based health facilities providing maternal and newborn care. The context where implementation took place was the district health service in Tandahimba district [17]. Facilitation in EQUIP, the process through which implementation of the innovation was enabled, consisted of 3–4 monthly learning sessions for a cluster of health facilities and monthly mentoring and coaching visits to individual health facilities. These activities were carried out by an EQUIP mentor, a member of the EQUIP implementing team, together with a district mentor, one of the members of the Council Health Management Team. Another part of the facilitation in EQUIP, not immediately visible to health workers, was the extensive prework carried out by EQUIP and district mentors, during which improvement topics were prioritised, learning sessions prepared and potential change ideas developed.
The hypothesised mechanisms of effect of EQUIP can be divided into two stages. In the first stage, the hypothesis is that learning sessions together with mentoring and coaching lead to increase in health workers’ knowledge and skills of the improvement topics and also their motivation and capacity to implement QI using PDSA-cycles and run-charts. In the second stage, this implementation of QI would lead to improved quality of care provided for mothers and newborns in the health facilities, through for example improved procurement of drugs and supplies and increased adherence to clinical guidelines.
Study setting
EQUIP was implemented in Tandahimba district in Mtwara region in south-east Tanzania. Characteristics of the district have been reported on in detail elsewhere [14, 29]. Briefly, the population of approximately 200,000 is predominantly subsistence farmers and serviced by one district hospital, three health centres and 29 dispensaries, all except one providing maternal and newborn care. At the time of our study, several external partners were involved in implementing various health programs in the district health facilities. Challenges for care provision include a lack of health workers, drugs and equipment [15]. In a previous paper, we explored health workers’ experiences of the conditions for care provision which were perceived as inherently unpredictable and largely outside of their control [29]. Health workers also expressed often being alone and feeling unsupported in their work [29].
Data collection
In February 2014, semi structured interviews were conducted with 16 health workers in 13 health facilities in Tandahimba, purposively sampled to reflect all levels of facilities (hospital, health centre and dispensary) and the varying degree to which QITs engaged in the EQUIP intervention, described as their functionality. The EQUIP mentors conducted the assessment of this functionality based on rating seven characteristics of the QITs including their knowledge of new improvement topics, PDSA cycles, work plans, change ideas and the use of run-charts (S2 File). Respondents were included from the hospital and all three health centres while ten out of the twenty-eight dispensaries were sampled through a two-stage process. Dispensaries were first sorted according to their caseload of deliveries in the previous month and from those with the highest case load, five facilities with a high functionality of their QITs and five with a low QIT functionality were sampled (Table 1).
Respondents were contacted in advance, by telephone where possible, else by going to see them in the health facility. They were asked if they would be willing to participate and when a suitable time for the interview would be. At the time of the interview, informed written consent was then sought from all respondents. Interviews were held in a private area of the health facility. Interruptions to allow respondents attending to their patients were made and for this reason, some interviews took place over a longer period. The median effective interview time was 1 hour 12 minutes (range 1 hr 3 min to max 2 hrs 7 min). Interviews were co-conducted in Swahili by the first (UB) author together with a female Tanzanian social scientist (FH). UB is a Swedish female medical doctor with a master’s degree in Public Health who had two years’ work experience as a clinician and program manager in rural Tanzania and could easily establish rapport with the respondents. This study was the final study of her PhD. FH had experience from several qualitative studies in southern Tanzania. Both are fluent Swahili-speakers and minimal translation into English by FH to UB was done during the interviews. Neither of the interviewers was known to the respondents beforehand.
The interview guide (S3 File) was adapted after the first interviews as some questions did not yield sufficient response and to reflect new ideas emerging during the data collection. The very first interview was considered a pilot and not included in the analysis for this study. While the number of interviews was pre-determined, it was felt that saturation in the interview material was reached before the last few interviews were conducted as no or little new information emerged. No repeat interviews with respondents were done. The interview guide was divided into two parts, of which only the second part was analysed and reported on in this study. Results from the analysis of the first part have been reported on elsewhere [29] and are referred to in the discussion and interpretation of results.
Interviews were audio recorded and transcribed verbatim. Subsequent translation into English was conducted with careful attention to ensure the preservation of the original meaning. During analysis, any sections with unclear meaning were compared with the Swahili transcripts for clarification. Transcripts were not shared with respondents for validation.
Data analysis
Data analysis was led by the first author (UB) with review by three of the co-authors (AP, AB, and CH) at different stages of the analysis. Qualitative content analysis was conducted by applying a theory-driven deductive approach [30] using the i-PARIHS framework as a lens. All transcripts were sorted into content areas correlating to the four i-PARIHS constructs [26]applied as themes for the analysis, then pasted into Microsoft Excel spreadsheets. Sorted paragraphs were further analysed deductively applying the characteristics of each i-PARIHS construct [28]as categories. The text was condensed into meaning units with codes applied which were grouped into sub-categories. Not all characteristics of each i-PARIHS construct were represented in the transcripts and the same sub-categories were sometimes reflected in more than one category resulting in some categories being merged. The emphasis during data analysis was on the themes of innovation and facilitation, and less on the recipients and context as these themes have been explored in detail in a previous paper [29].
Results
Table 2 outlines an overview of the results along the main themes, corresponding to the four i-PARIHS constructs of innovation, recipients, context and facilitation. Detailed results for each theme are described below.
1. INNOVATION: Experience and understanding of the EQUIP intervention
1.1 Clarity
The clarity of the EQUIP intervention was reflected in sub-categories describing health workers’ perceptions of its purpose, their memory of improvement topics and understanding and use of the PDSA-cycles and run-charts (Table 2).
The purpose of EQUIP was perceived as improving maternal and child health by the majority of health workers although some described the purpose as providing training. Some health workers could not remember the improvement topics, whereas others were able to list them and described continuing to work on all of them during EQUIP.
There was a broad range of understanding and reported use of PDSA-cycles by health workers. Some health workers could describe the steps involved in a PDSA cycle and how to use it. A few described never having used a PDSA-cycle and some never having heard of it. Many health workers were unsure of its meaning and unable to explain the steps.
I would say you plan something and do, and act and what is another; I forget one. S is for setting, you set. I have never used it. (HW #15)
While a few health workers reported never having used a run-chart, the majority reported using them regularly, and the understanding was overall better than that of PDSA cycles. This understanding was reflected in the perceptions of their benefits and motivating effect described in the section on relative advantage (see below).
1.2 Degree of fit /with existing practice/
Health workers’ perceptions of the degree of fit between EQUIP and existing practice was reflected in two sub-categories: “it is within our responsibilities” and “EQUIP brought what we needed”.
The majority of health workers perceived that what EQUIP introduced was already within their responsibilities, indicating a high degree of fit with existing practice. Several health workers mentioned that, as opposed to programs dealing with one area of care, for example Prevention of Mother to Child Transmission (PMTCT), EQUIP dealt with all mothers regardless of HIV status. This more comprehensive approach was experienced as something positive.
But for EQUIP all mothers, whether she is positive or negative, must be touched by EQUIP […] the same mother included in PMTCT, if you didn’t provide proper care she can die from other things apart from HIV and at the same time, the mother who is HIV negative, if you don’t provide proper care you may lose her due to other things and not HIV. Therefore other programs are for that, but …..I would say EQUIP is dealing with daily activities and touch every mother. (HW #16)
Improvement topics were part of everyday practice, and health workers described having worked on these before. In some instances, the same problems as those identified during EQUIP had been the focus of previous improvement efforts; a prominent example of which was the high infection rates after Caesarean sections in the District hospital. The tools and support introduced by EQUIP were therefore welcome, also reflected in health workers’ appreciation of being reminded of skills learned during their pre-service training.
We were using these postnatal strategies even in the past, but they were not like the way they are now. […]. The significance and sensitization of postnatal care have increased. (HW #3)
1.3 Degree of novelty
Although the perceived high degree of fit could be viewed as the opposite of novelty, health workers expressed that they had become aware of “new strategies for old problems”. One of these was the involvement of fathers during Antenatal care, to increase birth preparedness, which was expressed as positive by several health workers. The importance of mothers staying longer in the health facility after childbirth was also described as new knowledge, something that health workers were not aware of before.
The use of run-charts was a new tool for some health workers who described never having used it before EQUIP.
1.4 Relative advantage
Health workers’ experiences of EQUIP’s relative advantage compared to standard practice included knowing one’s performance, being able to provide better care to mothers who bring equipment and an increased awareness of being able to provide care despite challenges such as lack of drugs.
Knowing one’s performance through the use of run-charts was experienced as motivating by many health workers; they were perceived as help to evaluate one’s work, to know when to celebrate successes and when and where to put in more effort.
You can assess yourself the way you have given services by looking at what you did the previous month and where you are now. I ask myself if I have been improving or not; have I delivered better services this month or not? What can I do so that I offer better services? (HW#9)
Health workers also experienced the increased birth preparedness among mothers, perceived as an effect of EQUIP, to make their work easier and that it enabled them to provide better care.
[…] when they come they have everything ready, therefore it becomes easy to offer service. (HW #3)
1.5 Observable results
Health workers had noticed several changes following the implementation of EQUIP. The majority of these were changes in mothers’ behaviour, described in the sub-category “mothers come”. Health workers experienced that more mothers were coming to deliver in health facilities and that more fathers were accompanying their pregnant partners for ANC visits and at the time of delivery. These developments were viewed as positive, even though more mothers coming to deliver meant more work.
Things are good [since starting to work with EQUIP], we get many mothers for deliveries, we only refer a mother after failing completely […] I feel very happy. (HW #15)
The increase observed in mothers’ birth preparedness was also part of this, as was the observation that mothers who had delivered at home would now come for follow up in the health facilities.
Complications in mothers and newborns were perceived to have decreased since the start of EQUIP. As mentioned previously, this was particularly emphasised in the hospital where a marked reduction of septicaemia in mothers following caesarean section was achieved.
Health workers also noted changes in their practice in that they had become better at documenting their work.
2. RECIPIENTS: Characteristics of health workers involved in quality improvement teams
2.1 Motivation, values and beliefs
When asked about the reasons behind their choice of profession, the majority of health workers expressed a strong internal motivation, illustrated in the sub-categories “this job is my heart” and “I wanted to help the community”. They described feeling encouraged when being thanked by pregnant women and mothers, and a desire to follow up those who had been referred. While a few health workers perceived that community beliefs and behaviours caused the main problems they were faced with; health workers overall expressed a belief of needing to commit to providing good services. This could include working overtime, something which was sometimes needed in order to complete additional reports for projects like EQUIP.
Because I love my work, I provide care until I feel satisfied. (HW #15)
2.2 Time, resources and support
Health workers described often having to “work alone in difficult conditions". Examples given included not having colleagues available to help during emergency situations, of having to perform many tasks by yourself and of not getting enough sleep due to constantly being on duty in the small health facilities. One health worker had been stationed alone in a dispensary for three consecutive years.
In 2006 I was transferred again, to [name deleted] health centre […] I think I worked alone for three years. Then I was transferred to this place in 2009 and there was a medical attendant who was working here and she was about to retire. After her retirement, I worked alone for about a year […] probably she [current colleague] will also get transfer…I can see it, even now she is normally away for two months or one and a half month as she is involved in a distance learning programme; that it is the way it is. (HW #5)
Health workers also described a sense of limited support from the district health officials, reflected in the category “we request but nothing is done”. This was not mentioned in relation to the EQUIP facilitation, but in general. Although they would convey their requests for assistance, they would not always receive this in a timely manner.
We do not understand what is being discussed there [at the district] because once we speak here that is it [nothing happens] (HW# 3)
2.3 Collaboration and team work
The category “we cooperate” reflects a sense of good cooperation among health workers and perceptions of helping each other when needed. Examples included direct patient care, where health workers would help each other to manage a difficult case such as PPH, distributing tasks when enough health workers were present, or sitting together to plan and make requests for drugs and equipment.
We cooperate, together we check what we don't have and we request together (HW#6)
The team work was also reflected in a few health workers expressing that colleagues who had attended training would report to the others about what they had learnt when they came back.
2.4 Power, authority and presence of boundaries
Health workers’ power and authority was also expressed as limited. “I was posted” was frequently described as the reason for working in the district or a particular health facility. Many issues, such as the frequent lack of drugs and equipment, were experienced as beyond health workers’ influence. Informal “task sharing” was illustrated by health workers perceptions of having to take on the role of higher cadre health workers.
I'm a medical attendant [lower cadre health worker] but I have to do deliveries. I can’t tell a patient that the midwife is not around so it is impossible I must receive the patient. (HW #14)
While this informal task sharing was necessary, lower cadre health workers described that their higher cadre colleagues sometimes perceived this as them interfering.
3. FACILITATION: Perceptions of EQUIP mentoring and coaching and learning sessions
3.1 Participation and ownership
Health workers’ sense of participation and ownership in implementing EQUIP was mixed, reflected in the sub-categories “we talk together”, “they come to look at our records” and “they direct us” (Table 2).
Some health workers experienced a high level of interaction and engagement with EQUIP, described as sitting and talking together with EQUIP mentors, of being asked questions and of being able to provide one’s views.
We sit and discuss where we are, what to do, and way forward. (HW#9)
A few health workers emphasised EQUIP mentors coming to look at the records in the health facility as a core activity, reflecting a more passive stance with a limited sense of ownership.
“They direct us” reflect a guiding role of the EQUIP mentoring and coaching which was mainly experienced as helpful. Health workers described being provided with solutions but felt that these were appropriate for their circumstances.
[…] we were instructed by the people of EQUIP […] we didn’t have an idea on what we should do, but they are the ones who came to advise us […] we found out that the idea was suitable. (HW #8)
3.2 Integration and empowerment
The extent to which health workers experienced the facilitation in EQUIP as integrated and empowering was reflected in four sub-categories; “they are good in follow-up”, “they remind you”, “they gave us ways to fight the problems” and “good opportunity to learn from colleagues”.
Many health workers articulated satisfaction, often unprompted, with the frequency and content of mentoring and coaching visits in individual health facilities. These were experienced to be integrated and health workers emphasised the iterative quality of the frequent follow-up and reminders.
For example like yesterday we were there [at the learning session] and we were given other objectives. After two weeks they [EQUIP mentors] will come to visit and to see if we are implementing or not, if not why? And if we implement do we do that correctly? If there is any limitation, we are reminded how to do in order to succeed. They do not abandon us these people. (HW #6)
While routine visits for supervision from the district to lower level health facilities would sometimes be experienced as negative, a positive change was felt following the implementation of EQUIP.
Before, I was very scared when you heard about supervision [from the district], you felt like running away because when they came here they complained […] but when they come [EQUIP mentors] […] the supervision perspective has changed […], it’s very polite. (HW#7)
Health workers expressed feeling empowered by the training provided by EQUIP; having increased their skills in problem-solving and enabling them to do things they couldn’t do previously.
The thing which makes me really happy is the strategy that if something is missing in your facility–do not just sit. Just try to go to another facility, maybe you will find it. (HW #17)
Learning sessions were mainly discussed by health workers when prompted. Experiences were overall positive with the opportunity to discuss and learn from others expressed as something good.
Your challenge may differ from others or their challenges may not be yours. (#16)
4. CONTEXT: Experience of innovation and change and absorptive capacity
4.1 Experience of innovation and change
The sub-category “projects often overlap” reflects the situation that health facilities in Tandahimba district were the recipients of several concurrent health programs run by research and non-governmental organisations; some of which like EQUIP focused on maternal and newborn care.
I don’t know if there is a difference because, if you don’t know these organisations how they work, it is not easy to understand the difference. (HW #17)
There could be as many as four external programs implemented alongside each other in some health facilities, experienced as a challenge in facilities with few, or sometimes only one, health worker.
Health workers were also not always clear about which programs did what and expressed the lack of integration between programs and their single-focus as something negative.
The problem is that everyone [every program] is proud of they have […] but here we are not dealing with one thing, but many. (HW #2)
At the same time, health workers experienced the presence of these various programs as positive and their activities as helpful. Specific examples included a positive feeling of being visited in one’s health facility, of receiving training and of being brought needed equipment.
From my experience these projects are helpful (mmm). I think it is because that they come to visit time to time…I mean if it would be that you are just working without them to pass by, I think that would not be good (mmm) for sure projects are helpful. (HW#16)
4.2 Absorptive capacity
Some health workers’ expressed that the presence of many different programs resulted in additional work load, especially in terms of documentation, as each project demands their own reporting.
We normally fill the forms at the end of the month. We write so many reports and every project demand their report to be sent. […] It is a bit complicated in working performance, yes, it is a bit difficult. (HW#14)
Health workers’ experiences of often working alone (described under the Recipients theme) and of being exposed to many different projects in the same health facilities can be interpreted as limiting the absorptive capacity, the degree to which health facilities have the capacity to incorporate new activities and implement projects.
Discussion
In this qualitative process evaluation, we have explored the experiences and understanding among health workers of EQUIP, a district-wide collaborative QI intervention in rural Tanzanian health facilities. The results illustrate positive perceptions of several components such as a high degree of fit between the improvement topics in EQUIP and health workers’ priorities in every day practice and an appreciation of the intervention’s comprehensive, as opposed to a single-focus, approach. A clear motivating effect of using run-charts to monitor progress and an emphasis on the perceived importance of on-site mentoring and coaching visits to individual health facilities has also been found. Coupled with a high level of internal motivation to provide good care among health workers, in a context where many feel unsupported and lonely in their work as illustrated in a previous analysis of the same interviews [29], it is likely that these components have contributed particularly positively to the mechanisms of effect of the EQUIP intervention.
The identified lack of clarity around the use of PDSA-cycles has possibly led to this intervention component contributing less to mechanisms of effect in this setting. The context where health workers were exposed to overlapping projects implemented in the same health facilities may have a negative impact on the absorptive capacity to implement QI, reducing the EQUIP intervention’s potential effect.
The improvement topics which were selected during the EQUIP prework had purposefully been aligned to national guidelines and health workers’ routine work and responsibilities, something which was clearly noticed and appreciated. Similar findings were made in another study on training and health workers’ motivation in Vietnam [31]. The integrated approach of EQUIP through its focus on all mothers and newborns, was another expression of the high degree of fit, and health workers pointed out the tension of having to manage all patients and conditions while many programs would only focus on one or two.
While the reported use of run-charts varied, the expressions of their relative advantage and motivating effect were strong. The understanding of this QI tool was described as good, an important finding in a context characterised by many lower cadre health workers with limited pre-service training [32]. Run-charts as a standalone discrete intervention have been evaluated in high-income settings, also with positive effects on motivation, [33] and their motivating effect has also been reported by a QI initiative in a district hospital in Rwanda [34].
Contrary to run-charts, there was a wide range of understanding and reported use of PDSA-cycles, also found in a recent study investigating health worker perspectives of a QI intervention for institutional childbirth care in the same area of Tanzania [18]. In our study, health workers would only mention PDSA-cycles when prompted, suggesting that this intervention component was of subordinate importance to them. While EQUIP mentors consistently used PDSA-cycles, our findings suggest that the use of PDSA-cycles by QITs outside learning sessions or mentoring and coaching visits were limited. Conceptual understanding of the steps in the PDSA-cycle, especially by lower cadre health workers, may be challenging and one reason for its limited use [32]. However, a recent systematic review of the application of PDSA-cycles in high-income settings also found that application was inconsistent, negatively affecting this approach’s potential to improve quality of care [35]. An analogy can be made with the use of Partographs to monitor labour, where understanding and application by health workers is an ongoing challenge [36].
The positive experiences of the mentoring and coaching in EQUIP was an area of strong consensus among health workers, reflecting findings from previous studies where the level of senior support was identified as a critical determinant of health worker motivation and retention [37–39]. The supportive supervision typically carried out periodically in lower level facilities (health centres and dispensaries) in the Tanzanian and other Sub Saharan African health system has traits in common with the mentoring in coaching in EQUIP in that it provides task-focused facilitation [28, 40]. This kind of facilitation is characterised by a focus on certain goals/tasks, provides episodic contact and technical help and is usually of low intensity but with a high coverage [28]. There is however a wide range of roles that supportive supervision in the district is supposed to fulfil and, from a health worker perspective, it is often experienced as a control function rather than support [41, 42]. These findings were mirrored in a multi-country study where no correlation could be established between supportive supervision and quality of maternal care [43]. It is possible however that this finding was due to poor quality supervision or the way that this supervision was measured and defined. A recent study from two Tanzanian hospitals found that the quality of supportive supervision, together with other aspects of management, can have a significant impact on health worker motivation and performance [44]. The same was found in a Rwandan study on improving quality of Integrated Management of Childhood Illness [45], in a Nigerian study where significant improvements were achieved in service delivery for maternal, newborn and child care through clinical mentoring as a single component intervention [46], and in India where mentoring significantly improved the skills and knowledge of labour and delivery nurses [47]. Our results mirror these findings and emphasise the importance of good quality supervision, or rather mentorship, as an important mechanism of effect to improve the quality of maternal and newborn care in Tanzanian districts [48].
While the experience of learning sessions in EQUIP was positive overall, similarly to PDSA cycles, health workers only shared their experiences of them when prompted. Their perceptions suggest subordinate importance of these meetings compared to the mentoring and coaching visits to their facilities. This was also found in the study on health worker perspectives of QI in the same area of Tanzania mentioned above, and similarly from high-income settings [18, 49].
The context in which EQUIP was implemented had many challenges which have been reported on in detail elsewhere and which were likely a strong determinant of the limited measurable effect of the intervention [12]. What came out additionally in this study was the potential negative influence of other concurrent health programs implemented in the same district. Tandahimba had several partners operating health programs targeting mothers and newborns simultaneously. This is commonly found in Tanzania despite efforts put in place to coordinate programs and likely affects the health system’s absorptive capacity to incorporate change [18, 50, 51]. Health workers in our study felt ambivalent towards this presence of alternate agendas and approaches, which have also been found in other areas [52]. While the attention provided by external partners was experienced as motivating, health workers were not always able to distinguish between programs and sometimes felt overburdened by their demands. A recent review of the plethora of varying QI methodologies in the Tanzanian setting also pointed out the potentially harmful effects of this lack of coordination; including duplication, inefficiency and a feeling of confusion among front-line health workers [16].
Methodological considerations
While time and funding for this study did not make it possible to conduct interviews in all 32 health facilities in Tandahimba district, our purposive sample aimed to reflect the range of functionality of QITs, the level of health facilities and cadres of health workers. As such, we believe that the results illustrate health workers’ experiences of the implementation of EQUIP in a comprehensive way.
While the interviewers were not involved in the implementation of EQUIP, health workers may still have been subjected to a social desirability bias, potentially skewing results to be more positive or limit results to areas that health workers felt comfortable to talk about. Our study could not distinguish facilitation from EQUIP mentors from that from district mentors, as frequently health facilities were visited by the two together. EQUIP mentors, as opposed to district mentors, had dedicated time for their task which could have enhanced the health workers’ positive experiences.
While the i-PARIHS framework was applied as a lens for analysis, it was not used to develop the interview guide and the results therefore did not reflect all characteristics of the i-PARIHS constructs. As in any deductive analysis, it is possible that some information which didn’t fit the predetermined categories were excluded. Although the qualitative analysis was led by the first author, three more authors were also contributing and we therefore believe the risk of omitting significant information to be small.
At the same time, we believe that the use of i-PARIHS strengthened our analysis in that it relates to theory of the key constructs known to affect implementation and also links this to the underlying theory of collaborative QI as outlined in our schematic logic model. This analytical approach fills an important gap in the improvement and implementation science literature [53, 54].
This process evaluation was purposefully conducted from the perspective of health workers and other perspectives, such as those of the EQUIP mentors or district managers, was left out. Another limitation was also that it only considered the first stage of mechanisms of effect, i.e. the interaction between health workers and the EQUIP project, and left out the interaction between health workers and the mothers or patients. It is possible that certain components of the EQUIP intervention impacted positively on the first level of mechanisms but not the second, which we were not able to assess. The evaluation was also based on health workers’ accounts with no inclusion of observations that could have deepened the understanding of how the intervention worked.
Lastly, no measure or examination of the sustainability of the collaborative QI approach as implemented by EQUIP was included in this process evaluation; limiting the potential understanding of integration and application in routine health services over time.
Conclusions
Quality improvement components that are well understood and experienced as supportive by health workers in everyday practice include strong alignment of health topics with local priorities, use of run-charts to monitor progress and mentoring and coaching in individual health facilities. Emphasising these components may enhance mechanisms of effect and result in more significant change; something which could also guide harmonisation between various methodologies and increase country ownership of QI approaches. Additionally, the implementation of concurrent, overlapping, external health programs in lower level health facilities should be limited.
Declarations
Ethics approval and consent to participate
Ethical approval to conduct the study was obtained from Ifakara Health Institute Institutional Review Board (Ref: IHI/IRB/No: 30–2012) and from the National Institute for Medical Research (NIMR/HQ/R.8a/Vo.IX/1704), both in Tanzania. Permission to conduct the interviews in sampled facilities was also obtained from the District Medical Officer in Tandahimba district. Health workers whose main responsibility was to provide maternal and newborn care were approached and scheduled for an interview if they agreed to participate. Informed written consent was obtained from all respondents at this time. Respondents were carefully informed about their right to refuse participation but all chose to participate.
Supporting information
S1 File. Detailed description of the EQUIP intervention in health facilities.
https://doi.org/10.1371/journal.pone.0209092.s001
(PDF)
S2 File. Functionality assessment of Quality Improvement Teams in EQUIP.
https://doi.org/10.1371/journal.pone.0209092.s002
(PDF)
Acknowledgments
Thank you to all health workers who lent their time to share their experiences during the interviews. Thank you to Farida Hassan who acted as a lead interviewer during data collection and transcribed many of the interviews. She was also participated in adapting the interview guide after the pilot interview. Thank you also to Ingrid Hylander who contributed to the development of the interview guide and acted as a mentor during the data collection phase. Thank you to the whole EQUIP team.
References
- 1. Campbell OM, Calvert C, Testa A, Strehlow M, Benova L, Keyes E, et al. The scale, scope, coverage, and capability of childbirth care. Lancet. 2016. Epub 2016/09/20. pmid:27642023.
- 2. Koblinsky M, Moyer CA, Calvert C, Campbell J, Campbell OM, Feigl AB, et al. Quality maternity care for every woman, everywhere: a call to action. Lancet. 2016;388(10057):2307–20. Epub 2016/09/20. pmid:27642018.
- 3. Tuncalp , Were WM, MacLennan C, Oladapo OT, Gulmezoglu AM, Bahl R, et al. Quality of care for pregnant women and newborns-the WHO vision. BJOG. 2015;122(8):1045–9. Epub 2015/05/02. pmid:25929823; PubMed Central PMCID: PMCPmc5029576.
- 4. Graham WJ, McCaw-Binns A, Munjanja S. Translating Coverage Gains into Health Gains for All Women and Children: The Quality Care Opportunity. PLoS Med. 2013;10(1):e1001368. pmid:23335862
- 5. Kanamori S, Sow S, Castro MC, Matsuno R, Tsuru A, Jimba M. Implementation of 5S management method for lean healthcare at a health center in Senegal: a qualitative study of staff perception. Global health action. 2015;8:27256. Epub 2015/04/10. pmid:25854781; PubMed Central PMCID: PMCPmc4390558.
- 6. Franco LM, Marquez L. Effectiveness of collaborative improvement: evidence from 27 applications in 12 less-developed and middle-income countries. BMJ Qual Saf. 2011;20(8):658–65. Epub 2011/02/15. pmid:21317182.
- 7. Necochea E, Tripathi V, Kim YM, Akram N, Hyjazi Y, da Luz Vaz M, et al. Implementation of the Standards-Based Management and Recognition approach to quality improvement in maternal, newborn, and child health programs in low-resource countries. International journal of gynaecology and obstetrics: the official organ of the International Federation of Gynaecology and Obstetrics. 2015;130 Suppl 2:S17–24. Epub 2015/06/28. pmid:26115852.
- 8. Kamiya Y, Ishijma H, Hagiwara A, Takahashi S, Ngonyani HA, Samky E. Evaluating the impact of continuous quality improvement methods at hospitals in Tanzania: a cluster-randomized trial. Int J Qual Health Care. 2016. Epub 2016/12/07. pmid:27920249.
- 9.
WHO. Standards for improving quality of maternal and newborn care in health facilities. Geneva, Switzerland: World Health Organization, 2016.
- 10. Eldh AC, Almost J, DeCorby-Watson K, Gifford W, Harvey G, Hasson H, et al. Clinical interventions, implementation interventions, and the potential greyness in between -a discussion paper. BMC Health Serv Res. 2017;17(1):16. Epub 2017/01/08. pmid:28061856; PubMed Central PMCID: PMCPmc5219812.
- 11. Campbell J. The route to effective coverage is through the health worker: there are no shortcuts. The Lancet. 2013;381(9868):725.
- 12. Blacklock C, Goncalves Bradley DC, Mickan S, Willcox M, Roberts N, Bergstrom A, et al. Impact of Contextual Factors on the Effect of Interventions to Improve Health Worker Performance in Sub-Saharan Africa: Review of Randomised Clinical Trials. PLoS One. 2016;11(1):e0145206. Epub 2016/01/06. pmid:26731097; PubMed Central PMCID: PMCPmc4701409.
- 13.
Baker UH, F; Hanson, C; Manzi, F; Marchant, T; Peterson, S; Hylander, I. Unpredictability dictates quality of maternal and newborn care provision in rural Tanzanian: a qualitative study of health workers' perspectives (Accepted in BMC Pregnancy and Childbirth). 2016.
- 14. Baker U, Peterson S, Marchant T, Mbaruku G, Temu S, Manzi F, et al. Identifying implementation bottlenecks for maternal and newborn health interventions in rural districts of the United Republic of Tanzania. Bulletin of the World Health Organization. 2015;93(6):380–9. Epub 2015/08/05. pmid:26240459; PubMed Central PMCID: PMCPmc4450702.
- 15. Penfold S, Shamba D, Hanson C, Jaribu J, Manzi F, Marchant T, et al. Staff experiences of providing maternity services in rural southern Tanzania—a focus on equipment, drug and supply issues. BMC Health Serv Res. 2013;13:61. pmid:23410228
- 16. Mwidunda PE, Eliakimu E. Creating a national culture of quality: the Tanzania experience. AIDS (London, England). 2015;29 Suppl 2:S175–7. Epub 2015/06/24. pmid:26102628.
- 17. Hanson C, Waiswa P, Marchant T, Marx M, Manzi F, Mbaruku G, et al. Expanded Quality Management Using Information Power (EQUIP): protocol for a quasi-experimental study to improve maternal and newborn health in Tanzania and Uganda. Implement Sci. 2014;9(1):41. pmid:24690284
- 18. Jaribu J, Penfold S, Manzi F, Schellenberg J, Pfeiffer C. Improving institutional childbirth services in rural Southern Tanzania: a qualitative study of healthcare workers' perspective. BMJ open. 2016;6(9):e010317. Epub 2016/09/24. pmid:27660313.
- 19. Garcia ME, Li MS, Siril H, Hawkins C, Kaaya S, Ismail S, et al. Health-care worker engagement in HIV-related quality improvement in Dar es Salaam, Tanzania. Int J Qual Health Care. 2011;23(3):231–8. Epub 2011/03/29. pmid:21441571; PubMed Central PMCID: PMCPmc3092691.
- 20. Keugoung B, Macq J, Buve A, Meli J, Criel B. The interface between health systems and vertical programmes in Francophone Africa: the managers' perceptions. Tropical medicine & international health: TM & IH. 2011;16(4):478–85. Epub 2011/01/12. pmid:21219552.
- 21. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258. Epub 2015/03/21. pmid:25791983; PubMed Central PMCID: PMCPmc4366184.
- 22. Kilo CM. A framework for collaborative improvement: lessons from the Institute for Healthcare Improvement's Breakthrough Series. Quality management in health care. 1998;6(4):1–13. Epub 1999/05/29. pmid:10339040.
- 23.
The Aga Khan University, and PMNCH, Essential interventions, commodities and guidelines. A global review of key interventions related to reproductive, maternal, newborn and child health (RMNCH). Geneva: WHO; 2011.
- 24. Waiswa P, Manzi F, Mbaruku G, Rowe AK, Marx M, Tomson G, et al. Effects of the EQUIP quasi-experimental study testing a collaborative quality improvement approach for maternal and newborn health care in Tanzania and Uganda. Implement Sci. 2017;12(1):89. pmid:28720114
- 25. Hanson C, Ronsmans C, Penfold S, Maokola W, Manzi F, Jaribu J, et al. Health system support for childbirth care in Southern Tanzania: results from a health facility census. BMC Res Notes. 2013;6(1):435. pmid:24171904
- 26. Harvey G, Kitson A. PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice. Implement Sci. 2016;11(1):33. Epub 2016/03/26. pmid:27013464; PubMed Central PMCID: PMCPmc4807546.
- 27. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53. Epub 2015/04/22. pmid:25895742; PubMed Central PMCID: PMCPmc4406164.
- 28.
Harvey G, Kitson A. Implementing Evidence-Based Practice in Health Care: A Facilitation Guide. New York: Routledge; 2015.
- 29. Baker U, Hassan F, Hanson C, Manzi F, Marchant T, Swartling Peterson S, et al. Unpredictability dictates quality of maternal and newborn care provision in rural Tanzania-A qualitative study of health workers' perspectives. BMC Pregnancy Childbirth. 2017;17(1):55. Epub 2017/02/09. pmid:28166745; PubMed Central PMCID: PMCPmc5294891.
- 30. Elo S, Kyngas H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15. Epub 2008/03/21. pmid:18352969.
- 31. Thu NT, Wilson A, McDonald F. Motivation or demotivation of health workers providing maternal health services in rural areas in Vietnam: findings from a mixed-methods study. Human resources for health. 2015;13:91. Epub 2015/12/03. pmid:26626015; PubMed Central PMCID: PMCPmc4667451.
- 32. Jaribu J, Penfold S, Green C, Manzi F, Schellenberg J. Improving Tanzanian childbirth service quality. International journal of health care quality assurance. 2018;31(3):190–202. Epub 2018/04/25. pmid:29687759.
- 33. Wells S, Rafter N, Eggleton K, Turner C, Huang Y, Bullen C. Using run charts for cardiovascular disease risk assessments in general practice. Journal of primary health care. 2016;8(2):172–8. Epub 2016/08/02. pmid:27477560.
- 34. Kotagal M, Lee P, Habiyakare C, Dusabe R, Kanama P, Epino HM, et al. Improving quality in resource poor settings: observational study from rural Rwanda. BMJ. 2009;339:b3488. Epub 2009/11/03. pmid:19880528.
- 35. Taylor MJ, McNicholas C, Nicolay C, Darzi A, Bell D, Reed JE. Systematic review of the application of the plan-do-study-act method to improve quality in healthcare. BMJ Qual Saf. 2014;23(4):290–8. Epub 2013/09/13. pmid:24025320; PubMed Central PMCID: PMCPmc3963536.
- 36. Souza JP, Oladapo OT, Bohren MA, Mugerwa K, Fawole B, Moscovici L, et al. The development of a Simplified, Effective, Labour Monitoring-to-Action (SELMA) tool for Better Outcomes in Labour Difficulty (BOLD): study protocol. Reprod Health. 2015;12:49. Epub 2015/05/27. pmid:26006758; PubMed Central PMCID: PMCPmc4448543.
- 37. Manzi F, Schellenberg JA, Hutton G, Wyss K, Mbuya C, Shirima K, et al. Human resources for health care delivery in Tanzania: a multifaceted problem. Human resources for health. 2012;10:3. Epub 2012/02/24. pmid:22357353; PubMed Central PMCID: PMCPmc3311084.
- 38. Mbaruku GM, Larson E, Kimweri A, Kruk ME. What elements of the work environment are most responsible for health worker dissatisfaction in rural primary care clinics in Tanzania? Human resources for health. 2014;12:38. Epub 2014/08/05. pmid:25086596; PubMed Central PMCID: PMCPmc4131055.
- 39. McAuliffe E, Daly M, Kamwendo F, Masanja H, Sidat M, de Pinho H. The critical role of supervision in retaining staff in obstetric services: a three country study. PLoS One. 2013;8(3):e58415. Epub 2013/04/05. pmid:23555581; PubMed Central PMCID: PMCPmc3605440.
- 40. Zinnen V, Paul E, Mwisongo A, Nyato D, Robert A. Motivation of human resources for health: a case study at rural district level in Tanzania. The International journal of health planning and management. 2012;27(4):327–47. Epub 2012/06/21. pmid:22714251.
- 41. Mkoka DA, Mahiti GR, Kiwara A, Mwangu M, Goicolea I, Hurtig AK. "Once the government employs you, it forgets you": Health workers' and managers' perspectives on factors influencing working conditions for provision of maternal health care services in a rural district of Tanzania. Human resources for health. 2015;13:77. Epub 2015/09/16. pmid:26369663; PubMed Central PMCID: PMCPmc4570215.
- 42. Nkomazana O, Mash R, Wojczewski S, Kutalek R, Phaladze N. How to create more supportive supervision for primary healthcare: lessons from Ngamiland district of Botswana: co-operative inquiry group. Global health action. 2016;9:31263. Epub 2016/06/28. pmid:27345024; PubMed Central PMCID: PMCPmc4921783.
- 43. Leslie HH, Gage A, Nsona H, Hirschhorn LR, Kruk ME. Training And Supervision Did Not Meaningfully Improve Quality Of Care For Pregnant Women Or Sick Children In Sub-Saharan Africa. Health affairs (Project Hope). 2016;35(9):1716–24. Epub 2016/09/09. pmid:27605655.
- 44. Tibandebage P, Kida T, Mackintosh M, Ikingura J. Can managers empower nurse-midwives to improve maternal health care? A comparison of two resource-poor hospitals in Tanzania. The International journal of health planning and management. 2016;31(4):379–95. Epub 2015/02/26. pmid:25712238.
- 45. Magge H, Anatole M, Cyamatare FR, Mezzacappa C, Nkikabahizi F, Niyonzima S, et al. Mentoring and quality improvement strengthen integrated management of childhood illness implementation in rural Rwanda. Archives of disease in childhood. 2015;100(6):565–70. Epub 2014/05/14. pmid:24819369.
- 46. Okereke E, Tukur J, Aminu A, Butera J, Mohammed B, Tanko M, et al. An innovation for improving maternal, newborn and child health (MNCH) service delivery in Jigawa State, northern Nigeria: a qualitative study of stakeholders' perceptions about clinical mentoring. BMC Health Serv Res. 2015;15:64. Epub 2015/04/17. pmid:25879544; PubMed Central PMCID: PMCPmc4335453.
- 47. Bradley J, Jayanna K, Shaw S, Cunningham T, Fischer E, Mony P, et al. Improving the knowledge of labour and delivery nurses in India: a randomized controlled trial of mentoring and case sheets in primary care centres. BMC Health Serv Res. 2017;17(1):14. Epub 2017/01/08. pmid:28061783; PubMed Central PMCID: PMCPmc5219705.
- 48. Kiplagat A, Musto R, Mwizamholya D, Morona D. Factors influencing the implementation of integrated management of childhood illness (IMCI) by healthcare workers at public health centers & dispensaries in Mwanza, Tanzania. BMC Public Health. 2014;14:277. Epub 2014/03/29. pmid:24666561; PubMed Central PMCID: PMCPmc3987128.
- 49. Gustafson DH, Quanbeck AR, Robinson JM, Ford JH 2nd, Pulvermacher A, French MT, et al. Which elements of improvement collaboratives are most effective? A cluster-randomized trial. Addiction (Abingdon, England). 2013;108(6):1145–57. Epub 2013/01/16. pmid:23316787; PubMed Central PMCID: PMCPmc3651751.
- 50. Sundby J. A rollercoaster of policy shifts: global trends and reproductive health policy in The Gambia. Global public health. 2014;9(8):894–909. Epub 2014/09/10. pmid:25203251; PubMed Central PMCID: PMCPmc4166968.
- 51. Mkoka DA, Kiwara A, Goicolea I, Hurtig AK. Governing the implementation of emergency obstetric care: experiences of rural district health managers, Tanzania. BMC Health Serv Res. 2014;14:333. Epub 2014/08/05. pmid:25086597; PubMed Central PMCID: PMCPmc4124475.
- 52. Prytherch H, Kagone M, Aninanya GA, Williams JE, Kakoko DC, Leshabari MT, et al. Motivation and incentives of rural maternal and neonatal health care providers: a comparison of qualitative findings from Burkina Faso, Ghana and Tanzania. BMC Health Serv Res. 2013;13:149. Epub 2013/04/27. pmid:23617375; PubMed Central PMCID: PMCPmc3648439.
- 53. Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Quality & Safety. 2015;24(3):228–38. pmid:25616279
- 54.
Peters DH, TRan NT, Adam T. Implementation Research in health. A practical guide. Geneva, Switzerland: Alliance for Health Policy and Systems Research, World Health Organization.; 2013.