Introduction

Millennium Development Goal four (MDG-4) is to reduce the number of under-five year old deaths from 93 to 31 per 1000 by 2015. Prompt and appropriate management of febrile illnesses in under-five years old children is one of the interventions that would contribute to the achievement of MDG-4 [1], [2]. Currently the management of fevers in low-resource settings is based on the WHO and UNICEF-supported Integrated Management of Childhood Illnesses (IMCI). The IMCI guidelines aim to reduce childhood mortality and disability due to common illnesses, and improve growth and development. It also aims at improving health-worker performance, strengthening health systems, and enhancing family and community practices [3], [4], [5], [6].

Prior to the introduction of IMCI, guidelines for the management of childhood illnesses were disease-specific [7], [8]. The major justifications for introducing IMCI were: (1) pneumonia, diarrhoea, malaria, measles and malnutrition accounted for over 70% of child deaths in many settings; (2) high frequency of co-morbidities (malnutrition is known to underlie 40–60% of other illnesses); (3) considerable overlap in clinical presentations of several common childhood illnesses; (4) first-level facilities generally lack laboratory capacity and practitioners rely only on clinical signs and symptoms to manage childhood illnesses [9], [10], [11].

Under IMCI, the case management of a sick child is expected to follow a stepwise approach that includes proper assessment, classification, treatment and counselling [3], [12], [13]. The assessment step involves checking vital signs, identifying danger signs, and enquiring about the presence of four main symptoms. It also includes enquiring about other problems and assessing the nutritional and immunization status of the child [12]. IMCI has recommended that in high malaria risk areas (where more than 5% of the fever cases in children are due to malaria), any child who presented with fever should be given an ant malarial [12], [14].

The Ministry of Health in Ghana adopted the IMCI approach in 1998 and by 2000 it was rolled out in all primary care facilities within the Ghana Health Service (GHS) [15]. The case management guidelines have been adapted and incorporated into the curricula of health worker training programmes in the country.

The presumptive approach to managing febrile illness has contributed to the over-diagnosis of malaria and the excessive use of antimalarials, including the relatively expensive artemisinin-based combination treatments (ACTs) [16], [17]. There is now however a shift to test-based management of malaria. The successful application of this shift in under-five children inevitably requires that it is incorporated into IMCI, the standard approach to managing childhood illnesses. WHO has accordingly reviewed the IMCI guidelines in order to incorporate test-based management of malaria as a requirement in endemic countries. For health systems and clinicians who have for many years followed the presumptive treatment approach of IMCI, adhering to the new guidelines will be a challenge [18]. In order to understand the potential barriers in introducing the new IMCI guidelines, we investigated current IMCI-based assessments of sick children in primary care facilities in rural Ghana. The study was part of a larger study to evaluate the health system determinants of RDT-based ACT use in rural Ghana.

Methods

Results

The catchment population of the studied health centres ranged from 1,789 to 47,025, while that for hospitals ranged from 5,214 to 202,409 (Table 1). The majority of facilities were government-owned, and headed by non-physicians. Malaria smear microscopy was available in three health centres and in all five district hospitals. While in 4 facilities, no staff had had training in IMCI, in 7 other facilities, only one staff had received such training at the time of the study. The average OPD-staff to catchment population ratio was 1: 5,768 for health centres and 1: 6,774 for the hospitals.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of the participating facilities and assessment of IMCI performance.

https://doi.org/10.1371/journal.pone.0028944.t001

A total of 1983 structured observations were undertaken over a period of six months; 1122 in the health centres and 861 in the hospitals. Follow-up of children to assess health outcomes on day 7–10 post-management started later than the main study. This delay resulted in additional structured observations and therefore an increase in the overall number of observations conducted.

The mean age of the children observed was 21months (3–59 months), with 19.1% aged less than one year. There were slightly more males (53%) than females. The majority (88%) of children were brought in by their mothers, of whom 6% were teenagers. Approximately one third of carers had no formal education.

The most frequent time of registration for children was between the hours of 1000 hrs and 1200 hrs, with 45% of children registering between these hours. About 90% of children were registered with the national health insurance scheme. The sequence of activities followed by the majority (89%) of children was registration at the out-patient department, history-taking and vital signs assessment, consultation, collection of drugs, followed by exit from the facility. Only 11% of children were referred to the laboratory for investigations and 1% referred to higher level facilities for further assessment.

Common presenting symptoms besides fever (which was an inclusion criterion) were cough (33%), vomiting (30%), diarrhoea (20%) and loss of appetite (25%). Consistently, symptoms were reported more often during consultation than during history-taking and vital signs assessments. Temperature and weight were checked in 91% and 88% of children respectively. In only 4% of children presenting with cough or difficulty in breathing was respiratory rate measured.

The proportion of children assessed for each task in the national IMCI guidelines are shown in Table 2. Among the danger signs the assessment of ability to drink (21%) and history of convulsions (13%) was very low. Only 15% of children were assessed for ear problems and the immunisation status was checked in 20% of children.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. IMCI-based parameters and number of children in which each parameter was assessed (N = 1983).

https://doi.org/10.1371/journal.pone.0028944.t002

All 11 items in the IMCI checklist were assessed in only 1% (11) of children and all of these children were observed in the same facility. Better-than-average (≥7 required tasks performed) IMCI assessment was achieved in 706 (35%) children with considerable variation (3% to 85%) across the facilities (Table 1). The worst performance (3%) was recorded in the only private facility among the facilities studied. The estimated design effect for the primary outcome measure of better-than-average IMCI assessment was 23.9.

Whereas nearly all (95%) children received an antimalarial, only 30% received an antibiotic. Twenty-nine percent of those who received an antibiotic received an antimalarial in addition.

Predictors of better-than-average assessment

Compared with doctors, better-than-average assessment was likely to be achieved when a medical assistant conducted consultation. Although nurses appeared to perform better than doctors, the difference was not statistically significant. Similarly, the likelihood of better-than-average assessment did not differ according to whether a nurse or health assistant performed history-taking and took vital signs (Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Association between various factors and a better-than-average IMCI assessment (N = 1983).

https://doi.org/10.1371/journal.pone.0028944.t003

Achieving better-than-average assessment did not depend on whether a child was assessed at a health centre or hospital (OR = 0.69; 0.26–1.89), or whether the child was covered under the NHIS or not. The amount of time spent in history taking, vital signs, and clinical consultation had no significant effect on the extent of performance. (Table 3) The overall time spent at the facility was not a predictor of performance. The extent of assessment was similarly not significantly influenced by whether the child attended the clinic on a day when attendance was higher than the average facility attendance (0.97; 0.71–1.31) or whether the facility had at least one staff trained in IMCI (2.19; 0.71–6.70).

None of the carer-related factors explored had a significant effect on the extent of assessment (Table 3). Among the children-related factors however, the likelihood of better-than-average assessment was slightly higher in children who presented with a history of diarrhoea, compared to children without diarrhoea (1.13; 1.04–1.67).

There were no differences between data collectors in their IMCI assessments, adjusted for the facility that each of them conducted observations in (P = 0.16).

Discussion

The study findings apply directly to the facilities evaluated, and are representative of health centres in the 6 districts studied. Because the study was undertaken in a period just before the introduction of test-based management of malaria, findings may serve as baseline against which to assess the impact of test-based management of malaria on IMCI assessments in the future.

The Hawthorne Effect is an acknowledged limitation of non-participant structured observation, the key method used in this study [22], [23], [24]. Health workers who are conscious of the fact they were being observed may have altered their behaviour. We assume in this study that if the health workers behaviour was changed because they were being observed then it was changed in a positive direction so that what was observed was the health workers improved practices. In our analysis, we used the mean assessment across all children in all facilities to define a cut-off of IMCI assessment. Our choice of this cut-off does not represent an endorsement as either satisfactory or adequate, the mean number of tasks performed by the health workers. This cut-off was taken in the absence of any standard in the literature on the assessment of IMCI case management practices. This approach to analysing health worker performance, and the tendency of health workers towards improved practice increased the potential for high scores in the assessment of their performance. In spite of this, the extent of execution of the required tasks was very low. In only 1% of children were all 11 required tasks performed.

The findings of this study are consistent with the evaluations of IMCI case management practices in many other similar settings [13], [], [25], [26,27]. Inadequate health-worker performance is a very widespread problem [28]. It is perhaps reasonable to begin to question whether, at the current level of health worker motivation and the overburdened heath systems, the standards defined in IMCI-based case management are implementable. There is a need for qualitative studies to assess health worker perceptions of the usefulness of the various tasks included in IMCI-based case management, and the practical barriers to their implementation. The pattern of assessments found in this study is similar to that observed in Benin where fever was among the most assessed and ear problems among the least assessed [25]. Complaints of diarrhoea elicited better-than-average IMCI assessment suggesting that clinicians intuitively considered whether there was the need to assess all parameters in each case. This could result in overlooking one of the objectives of the IMCI case management process, which is to use clinic visits by sick children, irrespective of the type of illness, as an opportunity to deliver preventive interventions [29].

Our finding that clinical outcome was not significantly influenced by the extent of IMCI assessment raises questions about the efficacy of the tasks listed within IMCI guidelines. Ninety-four percent of children had a favourable outcome and this was irrespective of the extent of IMCI assessment. However, in our study we did not have expert re-evaluation of the study children to compare the treatment decision that would have been made if the IMCI guidelines had been strictly adhered to. Thus the lack of an association between the level of assessment of IMCI indicators and the health outcomes observed in this study should be interpreted with caution. Our observation is nevertheless consistent with another study in Benin where the children were re-evaluated by an expert clinician, it was found that although health workers often deviated from IMCI guidelines, it did not necessarily lead to ineffective treatment [25].

We were unable to assess IMCI performance matched against individual health worker attributes including whether IMCI trained or otherwise. However the presence of at least one staff member who had received training in IMCI at the facility did not make any difference to the assessments conducted at the facility. Although training and re-training has often been proposed as an intervention to improve health worker performance, its impact remains unclear. While some reports suggest a positive impact of training [25], [26], [29], others report no effect [30], [31].

After 15 years of IMCI implementation and little evidence of its effective implementation or effectiveness (possibly due to poor implementation), a re-evaluation of its guidelines, algorithms and approach is required. Alongside re-evaluation of the tools for IMCI assessments, studies of its feasibility of implementation under a variety of health systems contexts, and assessment of factors that increase feasibility of effective implementation should be studied. Any revision to the IMCI guidelines should be undertaken in consultation with stakeholders, including clinicians and other health workers who are the users of the guidelines. A convincing evidence-base should be established to make the case for the inclusion or exclusion of each of the tasks on the IMCI assessment checklist.

Respiratory infections are an important cause of febrile illnesses in children in sub-Saharan Africa. In the absence of x-ray and laboratory support, IMCI case management guidelines stress the need to use respiratory rate in assessing and classifying acute infections. This is to provide a rational basis for the use of antibiotics and other interventions [11], [12], [32], [33]. Therefore our finding that respiratory rate is rarely measured in the assessment of febrile children suggests that the use of antibiotics in the management of acute respiratory infections is, in many cases, not based on rigorous clinical assessment.

In the process of assessing febrile children in health facilities in Ghana, there are two stages at which thorough history-taking and assessment is possible. These are during vital signs assessment and during consultation. Our finding that symptoms tend to be reported more frequently during consultation than during vital signs assessment suggest a need to strengthen the stage of history-taking and vital signs assessments. A probable solution may be the use of structured symptoms and signs forms that make it a requirement to record the findings of specific assessments at each stage. The use of such a form will offer clinicians and other health personnel a tool for self-circumspection. It will also provide real-time objective assessment of the extent of completeness of IMCI-based assessment procedures and a trail for the audit of case management procedures.

It has been argued that the management of non-malarial febrile illnesses will be improved by confirming the diagnosis of malaria [15], [17], [34]. However because of the possibility of false results and co-morbidity, using rapid tests to confirm malaria would still require clinicians to conduct thorough assessment in arriving at a definitive or working diagnosis [10], [35]. In the era of test-based management of malaria, safeguards need to be instituted to prevent the substitution of thorough clinical assessment with sole reliance on rapid test for malaria to classify febrile illnesses. That path would appear effortless and in the face of dwindling health staff and increasing workload on a few, there is danger that overreliance on rapid test for malaria could result in a further slip in the quality of clinical assessments. Our finding that antibiotics were nearly always given with an antimalarial (but not vice versa) suggests a lack of confidence with the diagnosis of non-malarial illnesses, together with wasteful use of antimalarials.

A striking feature in the results of this study is the lack of statistically significant associations between nearly all the factors that were investigated as potential predictors of the extent of IMCI assessment. However, the ORs showed trends towards association that may have been statistically significant given a larger sample size. We assume the lack of statistically significant associations was due to the loss of power as result of the large design effect of 23.6, for which adjustment was made in the analysis. Although typically, a design effect of 1–3 is anticipated and adjusted for in cluster surveys it is not uncommon for the design effect to be much larger [36], [37]. This level of design effect was not anticipated at the time of designing the study. However had such a large effect been anticipated, it would have been logistically impossible to conduct a study of a sufficient sample size to achieve a power of 80% in estimating the parameters of better-than-average IMCI assessment. The lesson in this regard is similar to that reported in the multi-country evaluation of IMCI [38]. We believe future studies that explore the quality of IMCI assessment across health facilities should consider the potential for a high design effect to undermine the power to detect significant associations and explore alternative approaches. The large design effect also points to considerable heterogeneity across health facilities in IMCI assessments and makes the case for facility-tailored interventions.

Our finding that most of the history-taking and vital signs assessments were conducted by health assistants, and medical assistants (in hospitals), and nurses and other cadres of staff conducted clinical consultations is a pointer to the extent that task-shifting is being employed to overcome the shortage of health workers in these facilities. This observation is consistent with reports from other African countries where task-shifting has been implemented [39], [40], [41]. Note should however be taken of the significantly poorer performance by non-professionals during clinical consultation. By existing GHS regulations, staff such as health assistants are not expected to conduct consultations and are therefore not targeted in IMCI training programs. The finding requires further investigation as it may be highlighting the limits to the extent that task-shifting can be taken in the delivery of specialised services.

Acknowledgments

We are grateful to the parents and guardians who permitted the assessments of their children to be observed in the various health facilities. We are similarly grateful to the heads and staff at the various facilities used for data collection, and the respective District and Hospital Management Teams. We wish to acknowledge the contribution of staff who undertook data collection, management and statistical analyses. We are particularly grateful to Mr. George Adjei. Our sincere thanks also go to the KHRC Scientific Review Committee, the ethics committees (KHRC, GHS LSHTM), project DSMB and the ACT Consortium.