http://orcid.org/0000-0002-8502-2318
Figures
Abstract
Background
The World Health Organization (WHO) in collaboration with partners is developing a toolkit of resources to guide lymphatic filariasis (LF) morbidity management and disability prevention (MMDP) implementation and evaluation. Direct health facility inspection is the preferred method for documenting the readiness of a country programme to provide quality lymphoedema management services, one of the three MMDP criteria used to demonstrate the elimination of LF as a public health problem.
Methodology/Principal findings
As component of tool development, a Delphi consultation was implemented to gain consensus on six proposed domains and fourteen proposed tracer indicators to measure national programme readiness to provide quality health facility-based lymphoedema management services. A seven-point Likert-type scale was used to rank the importance of proposed domains and tracer indicators. Consensus for inclusion of the indicator was defined a priori as 70% or more of respondents ranking the proposed indicator in the top three tiers (5–7). Purposive sampling was used to select 43 representative experts including country representatives, programme implementers, and technical experts. A 55.8% response rate (n = 24) was achieved for the survey. Analysis of the responses demonstrated that consensus for inclusion had been reached for all proposed domains including trained staff (mean = 6.9, standard deviation (SD) = 0.34), case management and education materials (mean = 6.1, SD = 0.65), water infrastructure (mean = 6.3, SD = 0.81), medicines and commodities (mean = 6.3, SD = 0.69), patient tracking system (mean = 6.3, SD = 0.85), and staff knowledge (mean = 6.5, SD = 0.66).
Significance
The Delphi consultation provided an efficient and structured method for gaining consensus among lymphatic filariasis experts around key lymphoedema management quality indicators. The results from this analysis were used to refine the indicators included within the direct inspection protocol tool to ensure its ability to assess health facility readiness to provide quality lymphoedema management services.
Author summary
Prior to this assessment, there was a need for clearly defined, measurable indicators for global lymphoedema management programmes to use to evaluate their lymphoedema morbidity management and disability prevention services. The results presented in this report outline a framework for indicators to assess healthcare facility-based readiness to provide quality lymphoedema management services. We describe our use of the Delphi methodology to obtain consensus on programme evaluation metrics as a model for elimination or control metrics and targets. The quality indicators developed from this Delphi consultation will be used to improve the direct inspection protocol tool that can be used to assess health facility readiness to provide quality lymphoedema management services LF endemic countries.
Citation: Walsh VL, Fox LM, Brady M, King J, Worrell CM (2018) A Delphi consultation to assess indicators of readiness to provide quality health facility-based lymphoedema management services. PLoS Negl Trop Dis 12(9): e0006699. https://doi.org/10.1371/journal.pntd.0006699
Editor: Marc P. Hübner, University Hospital Bonn, GERMANY
Received: March 28, 2018; Accepted: July 19, 2018; Published: September 18, 2018
This is an open access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Data Availability: All relevant data are within the paper.
Funding: MB was supported thanks to the generous support of the American People through the United States Agency for International Development (https://www.usaid.gov/) and the Morbidity Management and Disability Prevention project led by Helen Keller International (HKI) (AID-OAA-A-14-00054). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Lymphatic filariasis (LF) is a parasitic infection caused by filarial nematodes that are transmitted by mosquitoes. Chronic infection with LF can lead to clinical manifestations such as lymphoedema and hydrocele that have significant impacts on the mobility and quality of life of affected individuals. Further, individuals with lymphoedema are prone to painful and debilitating secondary bacterial infections, known as acute attacks, that are associated with diminished quality of life and progression of disease [1–4]. Approximately 947 million people are at risk for LF in more than 73 countries worldwide [5]. In an effort to reduce suffering, LF has been targeted for elimination as a public health problem by 2020 following the World Health Assembly Resolution 50.29 [6]. The Global Programme to Eliminate LF (GPELF) has established a two-pillar strategy for elimination: (1) interruption of transmission through mass drug administration (MDA) and (2) alleviating the suffering of individuals affected by the chronic manifestations of LF infection through the provision of morbidity management and disability prevention (MMDP) services. To meet the criteria established by the Word Health Organization (WHO) for the MMDP pillar for elimination, national LF elimination programmes are asked to provide data on the number of patients with lymphoedema (or elephantiasis) and hydrocele, the number of health facilities designated to provide care, and the readiness and quality of the care provided [7]. Quality of care assessments can be used to understand what resources are needed to improve services as well as advocate for other sectors or departments within the Ministry of Health to supplement these services.
While the implementation and evaluation activities for MDA have been clearly defined, there is a need for clearer guidance on the provision and assessment of MMDP services for national LF elimination programmes. In order to meet this need, WHO is developing a toolkit to guide LF MMDP implementation and evaluation. One component of the toolkit is a direct inspection protocol, a tool designed to measure readiness to provide quality health facility-based lymphoedema management services in accordance with WHO recommendations. Here, we summarize an expert consultation following the Delphi methodology to reach consensus on domains and indicators that should be used to evaluate health-facility readiness to provide quality lymphoedema management services [10]. The results of the Delphi consultation informed the refinement of the direct inspection protocol tool and will assist national LF elimination programmes demonstrating that they have achieved the requirements for validation of elimination of LF as a public health problem.
Methods
Delphi methodology
Indicator development.
To establish consensus on indicators to assess the quality of MMDP services, a Delphi methodology was implemented. The Delphi methodology has been utilized by others in the neglected tropical disease (NTD) field to obtain consensus on programmatic targets as well as indicators for programme monitoring and evaluation [8, 9]. It is a quantitative mechanism to gain consensus on a particular topic among a panel of subject matter experts [10]. The Delphi methodology that we implemented (Fig 1) was based on a framework proposed by Deribe and colleagues in the context of establishing indicators to assess endemicity, elimination, and clinical outcomes of podoconiosis [9, 11].
Based on a literature review, an expert panel proposed six key domains of facility readiness and 14 tracer indicators to measure readiness of health facilities to provide quality MMDP services for lymphoedema care. The six domains selected were modeled after the Service Availability and Readiness Assessment (SARA), a WHO tool for evaluating health facilities, and included availability of: trained staff, case management and education materials, water infrastructure, medicines and commodities, patient tracking system, and staff knowledge [12]. Fourteen tracer indicators were proposed to evaluate the domains (Table 1).
Participant selection.
Forty-three experts in LF and NTDs representing various stakeholders globally were identified to participate in this consultation (Table 2). This target was determined assuming a 20% loss to follow-up over two steps and aimed to achieve 15 to 25 final participants. The process for selecting these participants was modeled on the recommendations for participant selection outlined in previous literature utilizing the Delphi methodology [10, 13]. Participants were from more than ten countries across all WHO regions with LF-endemic countries and represented country programmes, non-governmental organizations, bi-lateral and multi-lateral organizations, donor organizations, and academic institutions.
Questionnaire.
Participants were invited by email to participate in the consultation via a link where they could access an online questionnaire. Participants were asked to report demographic information including professional role, educational background, area of expertise, and years of experience in their respective area of expertise. Participants were then asked to evaluate the domains and corresponding indicators using a seven-point Likert-type scale to optimize discriminating power [14, 15]. Finally, participants were asked to rank the importance of each of the domains in determining quality of MMDP services a health facility could provide, ranging from “1 = not at all important” to “7 = extremely important”. Participants were asked to rank how well each of the indicators evaluated the respective domain ranging from “1 = strongly disagree” to “7 = strongly agree”. Participants were also invited to provide open-ended feedback on the domains and indicators.
Seven indicators measured general health facility readiness and quality (e.g. water infrastructure, provision of medications and commodities) and seven measured lymphoedema-specific readiness and quality of services provided by health facilities (e.g. staff training and knowledge). In an effort to harmonize and integrate with ongoing WHO initiatives, it was determined that if the water infrastructure domain was deemed relevant, the tracer indicators for water infrastructure outlined by WHO’s water, sanitation, and hygiene (WASH) in health-care facilities initiative also would be used in the direct inspection protocol for consistency. Therefore, the water infrastructure tracer indicator was not evaluated, thus only thirteen indicators were evaluated in the Delphi methodology versus the total 14 indicators that would be assessed as a component of the direct inspection protocol.
Data analysis.
Data were analyzed using Microsoft Excel and SAS 9.3 (Cary, NC). Consensus criteria for each domain and tracer indicator were defined a priori as follows: consensus for inclusion was achieved if ≥70% of participants ranked the item in the top three categories (5–7); consensus for exclusion was achieved if ≥70% of participants ranked the item in the bottom three categories (1–3); and no consensus was achieved if neither of the above conditions were met.
Counts of respondents’ selections were used to calculate the frequency of selected answers to inform whether consensus had been reached. In addition, the sample mean, median, and range were calculated to assess central tendency for each tracer indicator and domain and to characterize the responses. Central tendency was included in the assessment as a secondary measure of consensus among the respondents. If consensus was not achieved by the above criteria, further refinement and evaluation of the domains and indicators in subsequent rounds of questionnaires was planned until consensus was reached.
As a sensitivity analysis, we evaluated the impact of using more stringent consensus criteria as follows: consensus for inclusion was achieved if ≥70% of participants ranked the item in the top two categories (6–7); consensus for exclusion was achieved if ≥70% of participants ranked the item in the bottom two categories (1–2); and no consensus was achieved if neither of the above conditions were met.
Results
The response rate for the online survey component of the Delphi consultation was 55.8% (n = 24). The individuals who participated in the survey represented a range of professions (Table 2). A third of participants (n = 8, 33.3%) had more than 25 years of experience in their respective field.
Participants’ responses to the domains and tracer indicators are presented in Tables 3 and 4. In the first round, there was consensus that all six domains of readiness and quality of MMDP services in health care facilities were important. None of the respondents ranked the domains in the bottom three categories. The strongest agreement was observed for trained staff and staff knowledge with 87.5% and 62.5% of respondents respectively indicating they felt that these domains were extremely important. Furthermore, consensus was reached for all thirteen evaluated tracer indicators, though a wider range of ranking was observed. The strongest consensus was observed for tracer indicators related to medicines and commodities—primarily the availability of medicines—as well as the tracer indicators for case management and education materials.
Based on the sensitivity analysis using stricter criteria for consensus, all domains met consensus criteria under stricter conditions with between 79.2% and 100% of respondents ranking the domains in the top two categories. All of the tracer indicators met the stricter consensus criteria except for staff training in the last two years (66.7%) and at least one patient with lymphoedema recorded in the reporting system in the last 12 months (50.0%).
Common themes from the qualitative feedback included: the need for a more robust definition of training and refresher training, the importance of clinic staff being able to identify more than one sign, symptom, and management strategy, and a need for a more clearly stated definition for a patient tracking system.
Discussion
The World Health Assembly resolution to eliminate LF was built on a desire to mitigate the harm caused by LF, both by preventing future infection as well as by alleviating the suffering experienced by individuals who present with clinical manifestation as a result of infection [6]. Since the clinical sequelae of LF develop many years after infection and are chronic, national LF programmes must work closely within the health care system to ensure that MMDP services are well integrated, available, and sustainable. While the components of a minimum package of care for lymphoedema and hydrocele patients has been clearly defined [9], there is a need for standardization in the evaluating and reporting of the availability and quality of MMDP services in the provision of the minimum package of care at healthcare facilities. This Delphi consultation allowed input from multiple stakeholders and improved the practicality and acceptability of a standard survey for direct inspection of health facilities to assess readiness and quality of MMDP. Based on previous literature, framing questions using a Likert-type scale for Delphi consultations facilitates straightforward statistical analysis to assess for consensus across respondents [13, 14, 16].
A strength of the Delphi methodology is that it allows stakeholders from a variety of perspectives to offer their expert opinion on the key elements that need to be included in an evaluation of quality services. Using a Delphi consultation, we were able to gather input for indicator development from a range of stakeholders. Our hope is that this approach will lead to broader stakeholder support and acceptability. Based on the diversity of participants, we feel that the consensus achieved reflects the priorities of global partners working towards elimination of LF as a public health problem. However, due to limitations in accessibility we were unable to include the perspectives of two important stakeholders: health facility level staff and affected patients. Steps were taken to include feedback from staff and patients during the pilot testing of the direct inspectional protocol as discussed later.
Though consensus was reached for the domains and indicators, through open-ended feedback experts proposed more stringent criteria to strengthen the indicators to measure the readiness of health facilities to provide quality lymphoedema management care. Citing the critical need for appropriate identification of lymphoedema in patients, experts suggested that a greater emphasis should be placed on the evaluation of staff knowledge. To address this, questions assessing staff knowledge were modified to require two correct responses instead of one for each tracer indicator. No significant changes were made to the components of the remaining domains and indicators.
The fourteen tracer indicators, refined as part of the Delphi consultation, are intended to comprise the questionnaire component of a health facility inspection tool, allowing LF programmes to evaluate the readiness of health facilities to provide quality lymphoedema management services as a component of MMDP programmes. The inspection comprises a facility walkthrough and interview with key health personnel at randomly selected health facilities providing lymphoedema management services. The surveyor evaluates if the facility meets the criteria for each indicator, through direct observation where relevant (e.g. the presence of medicines and commodities). The results of the questionnaire generate a health facility score, by which the programme can evaluate highly performing and poorly performing health facilities. Programmes can also evaluate indicator scores across facilities to evaluate systematic strengths and weakness, in order to implement informed process-improvement steps to strengthen the quality of lymphoedema management services. In addition, the standardization offered through these tracer indicators provides programmes with the ability to compare lymphoedema management services across settings.
We recognize that while the primary focus of this Delphi consultation was assessing lymphoedema management services, hydrocele care is also important in LF endemic countries. We feel confident that we could replicate similar procedures to develop indicators to assess health facility readiness to provide quality hydrocele care. Due to the unique components of care required for hydrocele patients, expert consultation with urologists with expertise in hydrocele management will be conducted to determine the domains and tracer indicators for evaluating readiness and quality of health-facility based hydrocele care. A standardized protocol, the WHO Surgical Assessment Tool (SAT), provides information on general surgical capacity including the availability of hydrocelectomy and is under revision. Efforts are under consideration to include a module specifically evaluating the readiness and quality of hydrocele care.
By demonstrating that the global community is in agreement about the components of lymphoedema management that healthcare facilities must be prepared to provide their patients, there is evidence to support the inclusion of the direct inspection protocol tool in the MMDP toolkit for countries implementing the MMDP component of the GPELF strategy. In an effort to assess the measurability of these indicators in healthcare facilities, the quality domains and tracer indicators identified were included in a pilot direct inspection tool to assess the readiness of healthcare facilities to provide quality lymphoedema management services in Mali, Vietnam, and Haiti [17]. The pilot demonstrated that these indicators were feasible to implement and yielded useful information about the quality of services; however minor changes were incorporated to the survey based on the results of the pilot. The direct inspection protocol is intended to supplement SARA assessments, provide more detailed information on lymphoedema management services, and to ensure programme managers have the information needed to plan for services to meet the needs of individuals with LF.
Acknowledgments
We would like to thank the LF and NTD experts that participated in the Delphi consultation.
Disclaimer: The authors' views expressed in this publication do not necessarily reflect the views of the United States Centers for Disease Control and Prevention, the United States Agency for International Development or the United States Government.
References
- 1. Das PK, Srividya A, Pani SP, Ramaiah KD, Vanamail P, Dhanda V. Cumulative exposure and its relationship with chronic filarial disease in bancroftian filariasis. Southeast Asian J Trop Med Public Health. 1994;25(3):516–21. pmid:7777918.
- 2. Mues KE, Deming M, Kleinbaum DG, Budge PJ, Klein M, Leon JS, et al. Impact of a community-based lymphedema management program on episodes of Adenolymphangitis (ADLA) and lymphedema progression—Odisha State, India. PLoS Negl Trop Dis. 2014;8(9):e3140. pmid:25211334; PubMed Central PMCID: PMCPMC4161333.
- 3. Pani SP, Yuvaraj J, Vanamail P, Dhanda V, Michael E, Grenfell BT, et al. Episodic adenolymphangitis and lymphoedema in patients with bancroftian filariasis. Trans R Soc Trop Med Hyg. 1995;89(1):72–4. pmid:7747314.
- 4. Wijesinghe RS, Wickremasinghe AR. Quality of life in filarial lymphoedema patients in Colombo, Sri Lanka. Trans R Soc Trop Med Hyg. 2010;104(3):219–24. pmid:19796782.
- 5. World Health Organization. Global programme to eliminate lymphatic filariasis: progress report 2014. Weekly epidemiologic record. 2015;(90):489–504.
- 6.
Ninth plenary meeting, 13 May 1997—Committee A, third report: WHA50.29 Eliminatino of lymphatic filariasis as a public health problem. Geneva, Switzerland: 1997 May 13, 1997. Report No.
- 7.
World Health Organization. Validation of Elimination of Lymphatic Filariasis as a Public Health Problem. Geneva, Switzerland: World Health Organization; 2017.
- 8. van 't Noordende AT, Kuiper H, Ramos AN Jr., Mieras LF, Barbosa JC, Pessoa SM, et al. Towards a toolkit for cross-neglected tropical disease morbidity and disability assessment. Int Health. 2016;8 Suppl 1:i71–81. pmid:26940312.
- 9. Deribe K, Wanji S, Shafi O, Muheki Tukahebwa E, Umulisa I, Davey G. Measuring elimination of podoconiosis, endemicity classifications, case definition and targets: an international Delphi exercise. Int Health. 2015;7(5):306–16. pmid:26185194; PubMed Central PMCID: PMCPMC4550552.
- 10.
Chia-Chien Hsu, Brian A Sandford. The Delphi Technique: Making Sense of Consensus. 2007. In: Practical Assessment, Research & Evaluation [Internet]. Available from: http://pareonline.net/getvn.asp?v=12&n=10.
- 11. Cross H. A Delphi exercise to refine the WHO three-point disability grading system for leprosy, and to develop guidelines to promote greater accuracy and reliability of WHO disability recording. Lepr Rev. 2014;85(1):18–28. pmid:24974439.
- 12.
Health Statistics and Information Systems WHO. Service Availability and Readiness Assessment (SARA): an annual monitoring system for service delivery: Reference Manual, Version 2.2. Geneva, Switzerland: 2015.
- 13.
Yousuf MI. Using Experts' Opinions through Delphi Technique. Practical Assessment, Research & Evaluation [Internet]. 2007; 12(4). Available from: http://pareonline.net/getvn.asp?v=12&n=4.
- 14. Preston CC, Colman AM. Optimal number of response categories in rating scales: reliability, validity, discriminating power, and respondent preferences. Acta Psychol (Amst). 2000;104(1):1–15. pmid:10769936.
- 15.
Vagias Wade M. Likert-type scale response anchors. In: Clemson International Institute for Tourism & Research Development DoP, Recreation and Tourism Management., editor. Clemson University 2006.
- 16. James MT, Pannu N, Barry R, Karsanji D, Tonelli M, Hemmelgarn BR, et al. A modified Delphi process to identify process of care indicators for the identification, prevention and management of acute kidney injury after major surgery. Can J Kidney Health Dis. 2015;2:11. pmid:26060575; PubMed Central PMCID: PMCPMC4460967.
- 17.
Binh Vu T, Worrell Caitlin M, Dung Do T, Brady Molly, Walsh Victoria, Yajima Aya, et al. Implementation of a facility-based inspection tool to assess quality of lymphedema management services in Vietnam. American Society of Tropical Medicine & Hygiene; Atlanta, Georiga2016.