Study design

The Interrupted Time Series (ITS) study design was used to conduct the study. This is a quasi-experimental longitudinal study design which involves statistical comparison of time trends before and after an intervention. It was used since there was a definite point in time when the implementation of the policy began and data could be obtained from the District Health Information System (DHIS-2) for the time period before and after policy implementation. There was no parallel event that would have affected utilization of maternal health services for deliveries supervised by skilled birth attendants in the county. Devolution of health services was delayed and mainly focused on provision of services. However, the study design and the method of analysis would remove the effect of any confounding factors or biases that may affect the results of the study. This is because ITS analysis involves a before-after comparison within a single population, rather than a comparison with a control group. This has the advantage that selection bias and confounding due to between-group differences are limited. The manner in which deliveries from health facilities were captured and reported to the DHIS-2 did not change in any way to affect study findings (i.e. there was no change in mode of measurement). A trend was established for before and after the implementation of the policy (intervention) for each indicator with a goal of determining whether the policy had a significantly greater effect than any underlying secular or seasonal trends.

Setting

The study was conducted in Nyamira County, one of the forty-seven counties in Kenya. Nyamira County is in the former Nyanza Province and borders Homabay County to the north, Kisii County to the west, Bomet County to the south east and Kericho County to the east. Nyamira County was selected for the study because of its maternal health indicators, particularly uptake of maternal health services and maternal mortality. The County covers an area of 899.4 km² and is home to five sub-counties: Nyamira (South), Nyamira North, Borabu, Manga and Masaba North [15]. The County had a population of 598,252 persons in the 2009 housing and population census report but is projected to have 667,716 persons in 2015 [16].

There are 130 health facilities in the county and eight of these are hospitals. The rest are dispensaries and health centers (i.e. PHCF). The average distance to a health facility in Nyamira County is 7 km and agriculture is the main economic activity with tea and coffee being the main cash crops grown [17].

Study population and sampling strategy

All the data on the variables of interest (deliveries attended by skilled health personnel; attendance of fourth ANC; institutional maternal mortality ratio (iMMR); and birth related complications) were collected from all the health facilities. These were incorporated in the study so as to achieve population-level outputs & outcomes, and representativeness. Data was therefore collected for all the health facilities in the county.

Data collection

Data was abstracted from the District Health Information System-2 (DHIS-2) for each variable of interest using a data abstraction form. These were then verified using facility level data. Data was collected for the period between June 1, 2011 and November 30, 2014. Therefore, forty-two data sets were collected for each outcome variable i.e. 24 pre- and 18 post-intervention. A records officer was trained and employed to collect the data from the DHIS-2 and to enter the findings in an excel spreadsheet. Data from hospitals and the PHCF were summated to find the total from hospitals and PHCF respectively. Data was also retrieved by sub-county. The retrieved data was verified by a second officer who counter checked if the entries were similar with what was in the DHIS-2 and with facility level data.

Data analysis

After data verification and cleaning the data was transferred to the IBM-SPSS version 17 for an interrupted time series (ITS) analysis using the auto-regressive integrated moving-average (ARIMA) model to estimate the intervention effects.

The regression model used for the study given the coefficient β1 is for time, β2 for phase and β3 for interact was:

Variable (e.g. births attended by skilled attendant-SCD) = β0 (constant) + β1time + β2phase + β3interact

Where:

1. β0 estimates the baseline SCD at the beginning of the pre FMS period
2. β1 estimates the change in number of SCD that occurs with each month before the FMS policy (pre-slope)
3. β2 estimates the change in SCD immediately after the FMS policy (interact)
4. β3 estimates the change in the trend of SCD of the post-FMS period compared to the pre-FMS period (post-slope i.e. β₁+β₂)

Estimates for regression coefficients corresponding to two standardized effect sizes are obtained: a change in level (step change i.e. β2) and a change in trend before the intervention (β1). The change in trend after the intervention (β3) is the sum of the pre-intervention slope and the change in level i.e. β1 + β2. Other coefficients generated from the ‘ARIMA model Parameters’ including the corresponding standard error (SE) and the t-values were important in calculating the 95% confidence intervals. The p-values demonstrated the significance of the effect of the free maternity services policy. Percentages were calculated to estimate the relative effect of the policy. This facilitates comparison between the sub-counties and health facilities. The analysis was done to determine the first- and third-month level effect after the policy execution date to monitor any immediate and short-term effects of the policy. Analysis for the sixth and ninth months was to demonstrate any mid-term effects whereas the analyses done at the twelfth and eighteenth month post intervention was to assess whether the policy had a long-term effect. Statistical significance was set at a p≤0.05.

Key findings

Following the implementation of the policy there was an immediate and significant increase in deliveries attended by skilled health personnel at the county level. This increase in SCD was long-term as it remained significant in the 18^th month. This was however associated with a decline in attendance of fourth ANC which was not statistically significant. At the CRH, there was a long-term increase in SCD and a medium-term increase in the number of CSs. At the same facility, there had been a significant decline in iMMR. However, there was an increase in iMMR after the policy that became significant from the 9^th month onwards. There was a statistically significant medium-term (up to one year) reduction in number of obstetric complications. However, there was no significant effect of the policy on neonatal mortality. In addition, there was a significant long-term effect of the policy on SCD in hospitals (up until the 18^th month). However the effect of the policy on SCD in PHCF was short-term (up to the 6^th month).

Discussion of key findings

The FMS policy improved the utilization of maternal health services, particularly the deliveries attended by skilled birth attendants. The relative effect of the policy was more marked in the hospitals than in PHCF. This improvement in deliveries attended by skilled personnel in the county was expected as studies have shown that facility deliveries usually increase following similar fee exemption policies [18–20]. The larger improvement of deliveries in hospitals was not expected though since the PHCF are most accessible to the populace [21]. The initial increase in utilization of maternity services at the PHCF may be due to their proximity and comparative ease of accessibility [21, 22]. Many pregnant mothers in labor would have been reluctant to seek care at the hospitals because of distance. There could also have been a misconception that the free services were only available at the PHCF and not at the hospitals.

Later on, due to quality of care (occasioned by staff numbers and qualifications) and the 24 hour availability of staff at the hospitals, there was a shift of clients from PHCF to hospitals resulting in the sustained rise in number of deliveries attended by skilled health personnel in the hospitals [23]. In addition, the hospitals received a relatively larger reimbursement per delivery compared to the PHCF [24]. Hospitals could therefore offer better quality services, better staff rewards and better gift hampers to mothers that deliver at their facilities.

Previous studies have shown that facility deliveries usually increase following similar fee exemption policies [18–20]. This is despite methodological limitations including a lack of secular time trends (temporal trends) and population representativeness of the sample for these studies [1]. Nonetheless, a study that estimated effects net of temporal trends reported an increase of 7.3 monthly facility deliveries in district hospitals in Afghanistan immediately following fee removal [8]. This was not statistically significant and the initial increase was followed by a slight decline in trend [8]. The improvement in deliveries conducted by skilled attendants is higher in our study because in Afghanistan 83.6% of deliveries were free before the ban. Similar to our findings this study also reported more marked increase in deliveries attended by skilled attendants in the hospitals [8].

Cost had been a significant barrier to maternal health care services in Kenya. Studies done in 2003 and 2006 in three African countries (Burkina Faso, Kenya and Tanzania) reported that the mean cost of a normal or a complicated delivery was US$18.4 compared to US$ 7.9 and US$5.1 for Burkina Faso and Tanzania respectively [4]. The mean out-of-pocket expenses for normal delivery care at Kenyan health centers and dispensaries was US$7.4 and US$4.3 respectively whereas it was US$ 13.5 in government hospitals with costs at private/mission facilities being twice as high as those incurred at government facilities [4]. The average cost of a complicated delivery (defined as either a delivery complicated by hemorrhage, eclampsia or obstructed labor) was US$26.0 and US$68.7 at government hospitals and private facilities respectively, which were very high compared to US$6.7 and US$8.0 incurred at health centers and dispensaries respectively [4]. Therefore the FMS policy eliminated a significant barrier to access in a county where many people are poor as attested to by the human development indices in the county which are actually lower than the national indices [15]. This explains the long-term increases in SCD.

Another effect of the policy is the sustained decrease in delivery-related maternal complications (i.e. post-partum hemorrhage and obstructed labor). Better labor management from the available staff contributed to the long-term decline in obstetric complications at the facility. However, there was no effect on neo-natal mortality ratio. This differs from findings in other studies conducted in the region. In Ghana, Senegal and Nepal there was an increase in the number of pregnant women presenting with pregnancy- and delivery-related complications including hypertension, hemorrhage and obstructed labor. This was attributed to increased access to emergency obstetric services [25–27]. These studies however did not take into account any underlying trends.

Increased referral of emergency cases requiring surgical intervention from PHCF could have resulted in the short-term increase in the number of caesarean sections (CSs) conducted at the CRH. This later waned in significance when there was a shift of clients from the PHCF to the hospital. These patients had their labor managed better than in the PHCF hence the stalled effect of the policy on CSs. In Senegal, a fee ban on caesarean deliveries resulted in the increase of caesarean deliveries from 4.2% to 5.6% over one year. This was a cross-sectional study that failed to take care of any underlying trends [27]. Meanwhile, an increase in caesarean deliveries from 0.9% to 2.3% was reported in Mali following fee ban on caesarean sections in 2009 [5]. There was limited access to caesarean deliveries to a section of the population (the poor) in Mali since there was no fee ban on normal deliveries.

Lack of a maternity theatre and fatigue due to high workload could have caused delays in performing emergency CSs. There could have been deterioration in quality of maternity services due to staff shortage. In addition, the obstetrician at the CRH resigned. This explains the worsening of the facility’s iMMR which had been declining before the policy implementation. The rising iMMR is in agreement with a study done at tertiary hospitals in South Africa that reported an increasing iMMR following a fee ban. The authors also postulated that this was due to a decline in quality of maternal health services resulting from an increase in patient load with no increase in staff or facility resources [28]. However, the study did not estimate the effect attributable to the fee change. These findings do not agree with similar studies in Sub-Saharan Africa that reported a decline in iMMR following fee exemptions. In Ghana there was a non-significant decline from 34% to 10% [26]. Findings in Mali reported a decline in post-caesarean maternal deaths that was attributed to early seeking of emergency care following the fee ban [5].

Contrary to the increase in facility utilization for deliveries seen since the introduction of the free maternity services policy, there was a trend towards lower attendance for the fourth ANC clinic. ANC services had been free in the country [29]. This is comparable to the findings of a similar study in Afghanistan where there was an increase in ANC services which had been free before the ban (8). Moreover, user fees on curative services had also been banned in PHCF (dispensaries and health centers). The resultant congestion together with the reduction in consultation time due to the increased workload caused the reductions in 4^th ANC by discouraging some women from attending the required number of ANC visits [6].

This study’s main strength is that deliveries attended by skilled personnel in all the facilities in the study area were summated to achieve population representativeness. Others include the quality and quantity of data that was collected to draw the time series. The interrupted time series study design and analysis method eliminated the effect of the trends before policy implementation. There was no other intervention during the study period that could interfere or confound the effect of the policy. The study analyzed the effect of the policy on a multiplicity of health indicators so as to assess the complex nature of real-world environments. It encompassed 24 months pre-intervention and 18 months post-intervention with the longer pre intervention period vital for modeling any cyclical trends hence limiting errors in analysis. The post-intervention period allowed assessment of short-term and long-term effects of the policy.

The study’s limitations include the relatively shorter period and lack of a comparison group since the policy was implemented nationally. This is mitigated by the quasi-experimental study design (interrupted time series) that compares the trends before and after the study i.e. there is still a comparator. This study was in a rural county and its findings may therefore not be generalizable to urban areas and towns. Most of the country remains rural though. The roll out of the policy occurred independent of other factors that could have affected maternal mortality or that could have increased access to skilled health attendance at birth but given its ecological nature other factors that may contribute to the observed change may not be easily identified.