Introduction

Accurate diagnosis of infectious diseases is essential for appropriate targeting of treatment and disease control. Rapid diagnostic tests (RDTs) are quick and easy to perform, they give results during one clinic visit, and they can be used in settings with little infrastructure or trained personnel. RDTs are promising tools to improve diagnosis in remote or low-resource settings. In the field of neglected infectious diseases, new manufacturers, RDTs, and users are coming onto the scene [1, 2]. As access to RDTs improves, the need for quality assurance and postmarket surveillance increases.

The International Medical Device Regulators Forum has formulated guidelines about quality assurance of medical devices, including RDTs, which have been adopted as regulatory standards in Australia, Canada, the European Union, Japan, and the United States [3]. Specific quality standards for in vitro diagnostic tests (IVDs) (ISO 13485:2003) and medical laboratories (ISO 15189:2012) have been published by the International Organisation for Standardisation [4, 5]. In less-regulated settings, the World Health Organisation (WHO) has stepped in to promote IVD quality [3, 6, 7].

Participation from various stakeholders is required to assure RDT quality. Manufacturers must ensure that their products are ready for the market—i.e., that product design, development, testing, manufacturing, packaging, and labelling meet the required standards of safety and performance. The role of RDT users is, among other responsibilities, to know indications, contraindications, and operating procedures of the devices. Most regulatory authorities recognise that efficient communication between manufacturers and users is key to postmarket surveillance [3–7]. In low-resource settings and in the field of neglected infectious diseases, this communication between manufacturers and users may be suboptimal, as well as the pre- and postmarketing oversight of national regulatory authorities.

The Neglected Infectious Diseases dIAGnosis (NIDIAG) consortium aims to improve diagnostic approaches for different clinical syndromes in low-resource settings where neglected infectious diseases are prevalent. In this case study, we assessed several quality aspects of RDTs used in the NIDIAG study about persistent fever: we focused on RDT labelling and instructions for use (IFU) and on product-related incidents, including communication with manufacturers about these incidents.

Methods

The NIDIAG study about persistent (≥7 days) fever recruited 1,926 patients in four countries (Sudan, the Democratic Republic of the Congo, Nepal, and Cambodia). The study evaluated 11 RDTs from seven manufacturers targeting human African trypanosomiasis, leptospirosis, malaria, typhoid fever, and visceral leishmaniasis (Table 1). These RDTs were selected by an expert committee based on their potential to improve prognosis of patients with persistent fever in the study areas. To be included as index tests, the RDTs had to target severe and treatable infectious diseases that cause persistent fever and they had to be ready for evaluation in phase III or phase IV diagnostic studies.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Characteristics of 11 rapid diagnostic tests used in the NIDIAG persistent fever study.

https://doi.org/10.1371/journal.pntd.0004655.t001

The assessment of RDT labelling and IFU was done by one trained person using a checklist that had been developed previously based on international regulatory documents, a review of the literature, field observations, and comments from manufacturers and implementers. This checklist is given as supporting information (S1 Table) [8]. The product-related incidents were reported by the laboratory technicians in the study sites. They had received on-the-job training to correctly manipulate and interpret the RDTs and to follow standard operating procedures in line with good clinical laboratory practice guidelines. The diagnostic accuracy (sensitivity and specificity) of the RDTs is not included in this report.

The following definitions were used. Test results were invalid if there was no control line. Migration was considered to be delayed if the sample/buffer mixture did not start to migrate immediately (this usually takes only a few seconds). Poor background clearing was defined as the presence of a red-coloured background in the result window of the test strip at the time of result interpretation.

Results

The results for adherence to recommended RDT labelling and IFU are shown in Table 2. Five of the 11 RDT products had a meaningful name (i.e., mentioning at least the target condition and the type of test). Ten products displayed internationally recognized symbols, but none of them provided a complete symbol key. Four products displayed all essential information on the box. The cassettes from three products and buffer bottles from five products were correctly labelled. Fig 1 shows cassettes and buffer bottles for leptospirosis and typhoid fever tests from the same manufacturer, which are indistinguishable from each other. Four RDT products included appropriately labelled lancets. The IFU of eight RDT products included all required information but were not sufficiently readable (font size too small, closed letter type, and/or too difficult phrasing precluding easy reading and understanding). In summary, none of the 11 RDTs complied with all the specifications.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Cassettes and buffer bottles from different products but indistinguishable from each other.

https://doi.org/10.1371/journal.pntd.0004655.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Assessment of labelling and instructions for use of 11 rapid diagnostic tests used in the NIDIAG persistent fever study.

https://doi.org/10.1371/journal.pntd.0004655.t002

We registered 15 quality-related incidents for five RDT products: one related to supply (insufficient buffer volume), three to shipment (lack of temperature monitoring or shipment at too high temperature), and one to storage (elevated temperature). The remaining 10 incidents were product related, seven of which were technical: change in buffer consistency (n = 1), delayed migration (n = 2) (Fig 2), high invalid test rate (n = 1), poor background clearing (n = 2), and dust in cassette packaging (n = 1). The three remaining incidents concerned changes in IFU: two changes related to sample volume and one to reading time. None of the concerned IFU displayed track changes or a revision history, which would allow quick and accurate identification of the changes. Only one IFU displayed a new version number, and only two IFU showed a different date of issue (Fig 3). The changes had not been communicated to the user by writing or by an alert on the box.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2.

Above: Rapid diagnostic test with correct lateral flow and timely migration of sample/buffer mixture. Below: Rapid diagnostic test with blocked lateral flow and no or delayed migration. The sample/buffer mixture is not flowing through the test strip but is staying behind in the sample well. This is considered as an invalid result.

https://doi.org/10.1371/journal.pntd.0004655.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Two successive versions of the instructions for use of a rapid diagnostic test (most recent version on the right).

The change in reading time (indicated with a red arrow on the picture) was not communicated to the user in writing, by an alert on the box, or through a new version number, “track changes,” or a revision history of the instructions for use. Only the date of issue of the instructions for use had changed.

https://doi.org/10.1371/journal.pntd.0004655.g003

For six of the incidents, we contacted the manufacturers (n = 3). For four incidents, the manufacturers’ reply was swift and correct: they provided information on changes in the IFU (n = 1) and on temperature stability (n = 1), and they performed corrective actions (n = 2, replacement of affected RDT kits, provision of additional buffer vials). For two incidents, the reply was inadequate. One reply was correct but delayed for 12 days (ascribed to summer holidays). For the second incident, the manufacturer replied that the validation of the new reading time should be done by us or that, alternatively, we could simply disregard the new IFU.

Overall, at least one quality problem was detected for each of the 11 RDTs. There was no clear overlap between different types of incidents: problems occurred with all RDT types and all manufacturers. To our knowledge, the observed shortcomings did not cause serious adverse events for participants (most of these RDTs were used as index tests and not to guide patient care), nor did they affect study data reliability.

Discussion

This case study raises the issue of quality and postmarket surveillance of RDTs for neglected infectious diseases, irrespective of their diagnostic performance. When RDTs are put under scrutiny, shortcomings appear to be common. Some of the problems also occur with CE-marked RDTs. This is not surprising because in the current European Union regulation, IVDs for neglected infectious diseases are not considered to be high risk, and consequently, CE marks can be obtained by so-called self-certification by the manufacturer [10]. Of note, only small investments would be required from the manufacturers to correct the shortcomings here described.

The most worrying shortcomings are the changes in reading time and sample volume without notification to the user. Such changes are categorised as substantial because they may influence RDT performance [11]. One of these incidents illustrates the failure of the manufacturer to correctly handle a customer complaint: the attempt to shift responsibility for the validation of the changed reading time towards the user and the recommendation to disregard the IFU are unacceptable. At the users’ end, quality assurance systems should clearly include proactive monitoring of IFU changes. In line with this, the ISO 15189 guideline that has been developed for medical laboratories (end users of diagnostic devises) recommends “document control”—i.e., the prospective monitoring, updating, and archiving of all documents that may vary in versions or time, including IFU [4].

Limitations of this case study include the relatively small number of RDTs evaluated and the fact that not all elements of RDT quality were assessed. Likewise, one might debate the relevance of the findings, as the reported shortcomings did not cause any apparent serious adverse events. However, one should note that the possible effects of these shortcomings actually were mitigated by the study-related standard operating procedures in case of the shortcomings in labelling and IFU and by quick and appropriate action of the study team in case of RDT-related incidents. In support of its relevance, this study assessed real-life RDT products, and the product-related shortcomings reported here were observed thanks to the quality system of the clinical study. We expect that in the daily practice of a remote or low-resource setting, these shortcomings would not have been detected nor reported and may affect RDT performance and impact patient outcome.

Similar quality problems have been reported for RDTs targeting other infectious diseases. For example, in a study of 42 different malaria RDTs, shortcomings were found at the level of devices, buffers, packaging, labelling, and IFU [12]. For malaria and other frequent infections such as HIV and hepatitis B and C, WHO has taken a leading role in quality assurance of IVDs through prequalification and guidance on postmarket surveillance [6, 7, 11]. In the case of RDTs for neglected infectious diseases, such formal and international quality assurance mechanisms are less strong [13]. Moreover, stakeholders in this field may be less coordinated and less informed than those in the fields of malaria or HIV, for which national control programmes, funding agencies, nongovernmental organizations, and product experts constitute influential stakeholders. In the context of RDTs for neglected infectious diseases, production volumes are low, commercial interests are poor, and manufacturers may have limited ISO competence. At the other end of the supply chain, users of such RDTs tend to be dispersed and poorly trained [14]. At the same time, the requirements for these RDTs are particularly high: they should be user-friendly, withstand extreme environmental conditions, and be valid in heterogeneous patient populations [10].

Based on this experience, we recommend proactive interest from IVD manufacturers in line with ISO 13485 guidelines. Likewise, users even in remote settings should be motivated and trained to take up their role as emancipated customers. This can be achieved through biomedical curricula, national health authorities, and regional professional and scientific organisations such as the African Society for Laboratory Medicine. In addition, stakeholders involved in selection, procurement, and implementation of IVDs should be aware about these quality issues. Finally, regulatory authorities could subject RDTs for moderate-risk infectious diseases to a more rigorous premarketing scrutiny [13].

Supporting Information

Acknowledgments

We thank Mohammed Osman Ibrahim Mohammedali, Modther Siddig Morsal Ismail, Mohamedelfateh Salaheldin Eljack Abdelwahab, Emilie Alirol, and Ninon S. Horié for substantial contributions to the NIDIAG persistent fever study in general and for their support of good clinical and laboratory practices in particular.