https://orcid.org/0000-0002-2681-4139
Figures
Abstract
Background
Infections caused by soil-transmitted helminths (STHs) are the most prevalent neglected tropical diseases and result in a major disease burden in low- and middle-income countries, especially in school-aged children. Improved diagnostic methods, especially for light intensity infections, are needed for efficient, control and elimination of STHs as a public health problem, as well as STH management. Image-based artificial intelligence (AI) has shown promise for STH detection in digitized stool samples. However, the diagnostic accuracy of AI-based analysis of entire microscope slides, so called whole-slide images (WSI), has previously not been evaluated on a sample-level in primary healthcare settings in STH endemic countries.
Methodology/Principal findings
Stool samples (n = 1,335) were collected during 2020 from children attending primary schools in Kwale County, Kenya, prepared according to the Kato-Katz method at a local primary healthcare laboratory and digitized with a portable whole-slide microscopy scanner and uploaded via mobile networks to a cloud environment. The digital samples of adequate quality (n = 1,180) were split into a training (n = 388) and test set (n = 792) and a deep-learning system (DLS) developed for detection of STHs. The DLS findings were compared with expert manual microscopy and additional visual assessment of the digital samples in slides with discordant results between the methods. Manual microscopy detected 15 (1.9%) Ascaris lumbricoides, 172 (21.7%) Tricuris trichiura and 140 (17.7%) hookworm (Ancylostoma duodenale or Necator americanus) infections in the test set. Importantly, more than 90% of all STH positive cases represented light intensity infections. With manual microscopy as the reference standard, the sensitivity of the DLS as the index test for detection of A. lumbricoides, T. trichiura and hookworm was 80%, 92% and 76%, respectively. The corresponding specificity was 98%, 90% and 95%. Notably, in 79 samples (10%) classified as negative by manual microscopy for a specific species, STH eggs were detected by the DLS and confirmed correct by visual inspection of the digital samples.
Conclusions/Significance
Analysis of digitally scanned stool samples with the DLS provided high diagnostic accuracy for detection of STHs. Importantly, a substantial number of light intensity infections were missed by manual microscopy but detected by the DLS. Thus, analysis of WSIs with image-based AI may provide a future tool for improved detection of STHs in a primary healthcare setting, which in turn could facilitate monitoring and evaluation of control programs.
Author summary
In this school survey conducted within a primary healthcare setting in rural Kenya, a deep-learning system (DLS) was developed for detection of eggs secreted in human stool from intestinal worms, so called soil-transmitted helminths (STHs). Infections caused by STHs are the most common neglected tropical diseases and a major cause of health impairment in low- and middle-income countries. Novel diagnostic methods are needed to improve disease control. In the current study, stool samples were collected from children attending local primary schools and digitized using a portable slide scanner and uploaded via mobile networks to a cloud repository. Using the digital samples, a DLS was trained and tested for the detection of eggs of the most common STHs (Ascaris lumbricoides, Tricuris trichiura and hookworms) and compared to conventional manual microscopy analysis of the samples. The results showed that the DLS was able to accurately detect STHs in the digital samples. Although the DLS detected some false positive parasites, it was able to detect a significant number of parasites that had been missed with conventional microscopy analysis. The study concludes that digital microscopy, supported by image-based artificial intelligence, can be implemented in a primary health care setting and may provide a future tool for improved STH detection and, thereby, better monitoring and evaluation of control programs. The digital method is especially promising in light intensity infections, with only a few parasite eggs per sample.
Citation: Lundin J, Suutala A, Holmström O, Henriksson S, Valkamo S, Kaingu H, et al. (2024) Diagnosis of soil-transmitted helminth infections with digital mobile microscopy and artificial intelligence in a resource-limited setting. PLoS Negl Trop Dis 18(4): e0012041. https://doi.org/10.1371/journal.pntd.0012041
Editor: Eva Clark, Baylor College of Medicine, UNITED STATES
Received: July 28, 2023; Accepted: February 28, 2024; Published: April 11, 2024
Copyright: © 2024 Lundin et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All data required to evaluate the conclusions in the article are included in the manuscript and/or the supplementary material. Additional data are available on request from the Data Access Committee (FIMM-DAC) at Institute for Molecular Medicine Finland (FIMM), University of Helsinki, Helsinki, Finland; fimm-dac@helsinki.fi. Further requests for sharing of deidentified data (digitized samples) will be considered by the FIMM-DAC abiding the following principles: data will be securely stored with appropriate documentation and not disposed into publicly accessible domains or otherwise shared without explicit permission from the FIMM-DAC, and data are only used with the aim to generate data for the public good.
Funding: This research was financially supported by The Erling-Persson Foundation (grant number 2021 0110) JL, Vetenskapsrådet (grant number 2021-04811) JL, Finska Läkaresällskapet r.f. JL, Medicinska Understödsföreningen Liv och Hälsa rf JL and Wilhelm och Else Stockmanns stiftelse JL. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: JL and ML are founders and co-owners of Aiforia technologies Plc. JL and AS reported having a patent for Mobile Microscope pending (no.WO2017037334A1; the invention is related to the use of fluorescence imaging filters combined with inexpensive plastic lenses; all rights are with the University of Helsinki) and JL having a patent for a slide holder for an optical microscope pending (no.WO2015185805A1; related to motorization of regular microscopes).
Introduction
Neglected tropical diseases (NTDs) are a group of infectious diseases that affect more than a billion people globally and result in considerable negative health impacts in already resource-constrained areas [1]. The most prevalent NTDs are infections caused by the soil-transmitted helminths (STHs), which include the roundworms Ascaris lumbricoides (giant roundworm), Trichuris trichiura (whipworm), Ancylostoma duodenale and Necator americanus (hookworms). Together these pathogens cause chronic infections that result in disability, stigmatization, and significant economic consequences for societies [2,3]. As the STHs cause loss of micronutrients, they can cause neurocognitive problems, impaired growth and development and chronic fatigue in affected children [4]. Furthermore, the STHs are significant causes of morbidity and complications during pregnancy [5,6]. Current strategies to reduce morbidity associated with STHs include mass drug administration (MDA) and interventions related to water, sanitation, and hygiene (WASH). The World Health Organization (WHO) has endorsed 2030 target goals for controlling STHs [7]. Currently, the recommended method of diagnostics of STHs is manual microscopy of stool samples to visualize and manually quantify parasites or parasite eggs [7,8]. However, there is a shortage of experts and access to microscopy equipment and laboratory infrastructure in regions with the highest STH prevalence [9,10]. Access to diagnostic tests is vital for efficient infection control and elimination of STHs as a public health problem, as well as and STH management, and there is a need for improved diagnostic tests that are accurate, feasible to use and deployable in regional laboratories, at the point-of-care (POC) and point-of-sampling in low-resource settings [11,12]. Although light microscopy has been widely implemented and validated, challenges with this method include the need for trained microscopists on-site and varying levels of sensitivity, especially in light-intensity infections [13]. The manual analysis of samples is furthermore time-consuming and labour-intensive, associated with inter- and intra-observer variability [14], and typically may require up to 10 minutes per slide [15].
There are a series of other diagnostic methods that have been evaluated, such as wet-mount methods, flotation-based methods (FLOTAC, Mini-FLOTAC, FECPAKG2, McMaster) and DNA-based methods (qPCR, LAMP) [16]. These have both advantages (typically higher sensitivity and specificity), but also significant disadvantages (lack of standardization, more time consuming, requirement for specialized equipment, higher costs) and manual Kato-Katz microscopy is still recommended by WHO for surveillance and epidemiologic surveys [1,7,8].
During the last decade, portable digital microscopes and cell phone-based microscopes have been increasingly explored as potential platforms for POC diagnostics of NTDs [17–21]. Studies have shown that the imaging performance achievable with these devices is sufficient to resolve the most common NTDs by both visual and automated analysis of the digitized samples [19,20]. Moreover, image-analysis methods, such as algorithms based on deep learning with artificial neural networks, have been applied to microscopy diagnostics for a large number of parasitic diseases [22] and shown to be suitable for diagnostics of NTDs with promising results [23–27]. Although the results demonstrate the potential for improved diagnostics of NTDs with these methods, clinical application of the methods in real-world settings has to date been limited and to our knowledge previously never been applied to assess diagnostic accuracy on a sample level based on digital scans of entire microscope slides, so called whole-slide images (WSIs).
In this study, we developed a digital diagnostic system for STHs and deployed it in a basic laboratory at a primary health care hospital in a rural region in Kenya with locally varying prevalence of STHs. We evaluated the system for detection of STHs in conventional Kato-Katz thick smears, prepared from stool samples of 1,335 children attending primary schools in the region. The samples were digitized with a portable WSI scanner, uploaded to a cloud-server over local data and mobile networks and used to train a deep learning system (DLS) to detect STHs. We evaluated the diagnostic accuracy of the DLS by comparison to expert manual microscopy assessment of the physical samples and to visual analysis of the digitized samples.
Methods
Ethics statement
Ethical approval for the current study (No: TUM ERC EXT/001/2020) was issued by the Chairperson of the Ethical Review Committee (Mombasa, Kenya) of the Technical University of Mombasa accredited by the National Commission for Science, Technology and Innovation Kenya. It was emphasized that participation in the study was voluntary and withdrawal possible at any stage without further obligation. Informed verbal assent for study participation was acquired from participants and their parents or legal guardian, and a written consent form (in Swahili and English) was signed by the parents or legal guardian of the participant, as well as by the headmaster of the primary schools. Following the study, children in all participating schools were offered preventive treatment (albendazole at a dose of 400 mg), according to national guidelines. All participants with detected STHs were also examined by a clinician, besides prescribing the deworming medication. Treatment was offered by a designated clinician from the study hospital (Kinondo Kwetu Hospital). There was a follow-up on treated infected patients that was conducted by the Kwale County NTD Program coordinator who also provided the necessary drugs. The National NTD Program management was included in the treatment for effective follow up. Furthermore, the families of children with positive tests were also offered deworming treatment.
Study design
This school survey was a diagnostic trial type I to establish the diagnostic accuracy of DLS with manual microscopy as the reference method [28], and reported in accordance with the STARD-guidelines (Fig 1) [29]. The study was based at a primary health care hospital (Kinondo Kwetu Hospital https://www.kinondokwetuhospital.com) in rural Kenya (Msambweni Subcounty, Kwale County). The hospital is fully owned and supported by a trust fund (Kinondo Kwetu Trust Fund) and serves the Kinondo region in Kwale County and provides basic health care, treatment of malaria, HIV, tuberculosis, parasitic diseases, antenatal care, deliveries, child care, vaccinations and health information. The laboratory has basic equipment including light microscopes, hematology, blood grouping, biochemistry and CRP analyzers, as well as rapid diagnostic test kits (e.g. dengue fever, HIV, hepatitis etc).
STH = Soil-transmitted helminth. DLS = deep-learning system.
The target population was children aged between 5 and 16 years attending local schools in the region (Zigira, Masindeni and Kilole primary schools). The schools were invited to participate in the study, and eligible participants recruited from grades 1–8 after informed assent and consent had been obtained. Inclusion criteria were the following: age between 5 and 16, informed assent from the participant and consent from the parents/guardians acquired. Exclusion criteria were the following: Symptoms and signs of acute, severe disease, subject refuses consent, analysis of samples failed due to unsatisfactory quality of the physical or digital sample.
Acquisition and preparation of samples
Stool samples were acquired from 1,335 children during a time of eight months (March to October 2020) with the assistance of a team of dedicated health care volunteers (HCVs). As the schools were temporarily closed due to the COVID-19 pandemic, study participants were provided with sample vials (30 mL) and instructions for sample collection and asked to collect the sample at home. Prior to the study, the research team based at the Kinondo Kwetu Hospital had briefed the team of HCVs with the intent of the study. Household mapping was done and each of the HCVs were allocated at least 20 households per day for the duration of the study. Sample vials were transported by the HCVs to the study laboratory at the Kinondo Kwetu Hospital for processing. During the study period, twenty samples were collected per day. Samples were processed in daily batches and Kato-Katz thick smears prepared by laboratory technologists following standard operating procedures (S1 Protocol). First, glass slides were labelled with a unique sample identifier number after which the sample was applied to the surface using a standardized wire mesh sieve for sample filtration. A standardized template was used to concentrate the sample in the centre of the slide. Following this, the sample was covered with a cellophane strip, immersed in glycerol-methylene blue solution (Kato’s solution), placed on filter paper, and pressed to form an even surface between the cellophane and the slide. Slides were read immediately after preparation, to avoid disintegration of hookworm eggs [30], using a standard light microscope (CX23, Olympus, Tokyo, Japan). Slides were reviewed by a laboratory technologist with experience in diagnostics of STHs (FK), blinded from results of the digital analysis. Only one observer was used in order to allow sample analysis before disintegration of hookworm eggs [8]. The entire cellophane-covered area was visually reviewed using 10x (NA 0.25) or 40x (NA 0.65) objectives and all visible parasite eggs of the species, A. lumbricoides, T. trichiura and hookworm eggs were recorded and counted. The Kato-Katz smear template is estimated to consist of 41.7 mg of stool. To calculate the number of eggs per 1 gram of stool (EPG) the egg count from the slide was multiplied by a factor of 24 (24 x 41.7 mg ≈ 1 g). Infection intensity was then categorized as light, moderate, and heavy infection according to the WHO guidelines [31]. Remaining stool of each sample was stored in a refrigerator for reference and discarded after a week on a ‘first-in, first-out’ basis.
Of the total number of 1,335 slides prepared, 155 (11.6%) were excluded due to inadequate quality for analysis and consequently 1,180 slides were included in the training and analysis of the method. Reasons for inadequate quality included insufficient amount of stool, sample contaminated with sand, too oily stool hence obscuring parasite eggs, stool too hard especially in constipation, fragmented filtrate which trapped water in the Kato-Katz smear causing areas of the sample to be empty or too dense, diarrhoea which could not form a firm smear, or high amounts of vegetable cells that obscured details including parasite eggs.
For development of the DLS, 388 (33%) slides were randomly selected to constitute the training set and the remaining 792 (67%) slides were used for testing of the algorithm. Manual microscopy assessment of samples was performed prior to the training and testing of the DLS, and results from the digital image analysis were, thus, not available during manual microscopy of the slides. Slides with discrepant results between the methods (manual microscopy and DLS-based analysis) in the test set were subjected to a visual review of the digital slides done by the expert microscopist (FK) and the researchers (AS, OH, SH, SV). During this phase visible STH eggs were detected in a number of samples that were not observed during the manual microscopy.
Digitization of slides
Following preparation of slides and manual microscopy, the Kato-Katz thick smears were digitized with a compact and portable whole-slide scanner (Grundium Ocus 20, Grundium, Tampere, Finland) that weighs approximately 3.5 kilograms (Fig 2). The scanner digitizes samples with a 6-megapixel image sensor, using a 20× objective (NA 0.40) with a pixel size of 0.48 μm. The device is operated from a laptop computer over the wireless local area network. Coarse focusing of samples was performed manually on one field of view before initiation of the scanning which takes on average 5–10 seconds, whereafter the scanning process is fully automated using a motorized stage and a built-in autofocus routine. Scanning the entire standard sized Kato-Katz thick smear typically took 5–10 minutes. Once the scan is complete, the scanner automatically stitches all field-of-view images into a single WSI. Digital slides were transferred from the internal storage on the scanner to local storage on the laptop computer and to an external hard drive back-up unit in a Tagged Image File Format (TIFF). A desktop application (Aiforia Connector, Aiforia Technologies, Helsinki, Finland) was then used to convert the files to a wavelet file format (Enhanced Compressed Wavelet, Hexagon Geospatial, Madison, AL, USA) with a compression ratio (1:16) previously shown to preserve sufficient detail to not alter image-analysis results [32]. The same application automatically compresses the file into a single zip-file and uploads the file to an image-management and machine-learning platform (Aiforia, Aiforia Technologies, Helsinki, Finland). Uploading of slides was performed with the in-house ADSL connection and using a 3G/4G mobile-network router (Huawei B525S, Huawei Technologies, Shenzhen, China), operating on the local mobile network (Safaricom, Nairobi, Kenya). The average size of the digitized slides in the compressed format was 502 MB (range 127 MB– 1,127 MB). Turnaround time for uploading of samples was 10–20 min per slide with the mobile network (upload speed 5–8 Mbps) or ADSL connection (upload speed 5–10 Mbps). On the cloud server, the slides were stored as JPEG-compressed tiles sized 512x512 pixels with pyramid-structure zoom levels (70% JPEG-quality). A typical WSI comprises approximately 10,000 tiles on the server. Access to the image server for slide viewing was established with a web browser using secure socket layer (SSL) encryption.
Figure showing: (1) laptop computer used to operate the whole-slide scanner and manage the digital slides, (2) the whole-slide scanner used to scan slides and (3) a 3G/4G mobile network router used to upload scanned slides.
Training of the deep learning system
For the development of the DLS used for the analysis of the digital samples to detect STHs, we utilized supervised deep learning. The image analysis was performed by an algorithm utilizing two sequential convolutional neural networks to detect STH eggs and classify them according to the different species (A. lumbricoides, T. trichiura and hookworm). Here, the first network (the “detector network”) initially detected all potential STH egg candidates and forwarded the results to a second network (the “classifier network”), which then classified the findings into one of four categories: A. lumbricoides, T. trichiura, hookworm or artefact (i.e., debris or other non-STH objects). This dual-step operation principle of the DLS is illustrated in Fig 3.
Panels showing (a) the digitized samples, (b) first step: localization of parasite egg candidates and (c) second step: classification of the parasite egg candidates into the soil-transmitted helminth and artefact categories. Scale bars: (a) 5 mm, (b) 50 μm, (c) 25 μm.
Digital samples were acquired by downloading the scanned whole-slide data from the image management platform (Aiforia Hub, Aiforia Technologies Oy, Helsinki, Finland), and each slide was converted into a single TIFF file for further processing and algorithm training in MATLAB (MathWorks Inc, Natick, MA). Digital samples in the training series (n = 388) were visually reviewed and objects of interest (parasite eggs) in the samples from the different categories manually annotated to constitute the ground truth for the training data. An initial set of 1,339 annotations were made by browsing the whole slide images of the training set and manually annotating visible A. lumbricoides (n = 371), T. trichiura (n = 530), hookworm (n = 51), and artefact (n = 446) findings. The initial annotations were done by the researchers (AS, OH, SH, SV), after thorough training on the morphology of the STHs provided by the laboratory technologist (FK) who performed the manual microscopy. After that, the rest of the object-level annotations were done by utilizing machine learning-assisted annotation (Fig 4). In this process, the image data in the training set was analysed by a previously trained algorithm, which then suggested undetected new potential objects of interest (e.g., parasites or artefacts) for the user to confirm or reject. The machine-learning annotations were completed by one researcher (AS) and in cases of uncertainty, confirmation was reached after consultations with the other researchers. These confirmed annotations (objects of interest and their locations in the image) were then incorporated into the training data for the DLS, improving the algorithm’s performance. This iterative annotation process was carried on until no new acceptable findings were found in the training slides. In total, a final set of 15,058 annotations, with manual and AI-assisted annotations for A. lumbricoides (n = 2,299 from 39 slides), T. trichiura (n = 2,727 from 94 slides), hookworm (n = 552 from 38 slides) and artefacts (n = 9,480 from 384 slides) were available for training the DLS.
Figure depicts how the algorithm automatically suggests new, undetected objects of interest with associated labels for the user to confirm or reject. These findings are then included in the training data, but also used to improve the performance of the algorithm itself to suggest new objects more accurately for the user.
For the object detector in the DLS, the you-only-look-once version 2 (YOLOv2) algorithm was used [33]. This algorithm predicts class probabilities for objects within bounding boxes in images. For feature extraction in the network, we utilized a pre-trained ResNet50 network [34]. Training of the algorithm was performed using training regions with a size of 512 x 512 px2 (246 x 246 μm) containing the objects of interest annotated in the training data. Training was performed with 20 training epochs with a minibatch size of 25, using a stochastic gradient descent solver with a momentum of 0.9 and initial learning rate of 0.0005.
For the object classification, transfer learning [35] was utilized using a pre-trained ResNet50 network. The algorithm was a multiclass classification convolutional neural network which was trained using manually selected image regions measuring 150 x 150 px2 (72 x 72 μm) in the training samples. These areas represented visible A. lumbricoides, T. trichiura, hookworm and artefacts. Training of the classifier was performed in 50 training epochs with a minibatch size of 32, using a stochastic gradient descent solver with a momentum of 0.9 and initial learning rate of 0.003 and dropping the learning rate by half after every 10 epochs. Image augmentations of the training data were utilized to increase the generalizability of the model and to prevent overfitting for both networks in the DLS. For the detection network, augmentations used were variation of scale (±10%), XY reflections, saturation offset (0.1), brightness offset (0.1), hue offset (0.05), and scale factor of contrast (0.1). The object classifier training data was augmented using variation of scale (±5%), rotation (0–360°), XY shear (±5°), XY reflections, XY translations (±10 pixels), saturation offset (0.1), brightness offset (0.15), hue offset (0.05), and scale factor of contrast (0.15).
Analysis of samples in the test set was performed using a 512 x 512 px2 (246 μm x 246 μm) sized scanning window with an overlap of 128 pixels (61 μm) with the adjacent regions to cover the whole sample image area. Overlapping bounding boxes of the detected objects were eliminated by selecting the box with the highest confidence (i.e. probability score). The detected bounding box was then resized to 150 x 150 px2 (72 μm x 72 μm), and that subregion image was forwarded to the second network for classification. The threshold for the confidence was set to 50/100 to reduce the initial false positive rate and computation cost. Similarly, a threshold for the parasite egg finding in the classifier was set to 90/100. We achieved an analysis speed of approximately 14 digital samples (whole-slide images measuring 25 x 25 mm2) per hour using a computer equipped with an Intel Xeon E3-1241 v3 processor, a NVIDIA GeForce GTX 1660 Super graphics processing unit, and 32GB of RAM running MathWorks MATLAB R2020b on the Microsoft Windows 10 operating system. If one WSI in average consists of 10,000 tiles sized 512x512 pixels, this corresponds to approximately 140,000 tiles (fields-of-view; FOV) per hour and 2,333 tiles (FOVs) per minute.
Statistical analysis
Results from analysis of samples were entered into a standardized spreadsheet table (Microsoft Excel, Microsoft, Redmond, WA, USA). We used a general-purpose statistical software suite (Stata 15.1, Stata, College Station, TX, USA) for analysis of the results. The sample sizes used in the study were determined based on calculations with a previously described sample size formula [36], assuming an estimated overall infection prevalence (prevalence of STHs in the population, Pr) of 20%, an α level of 0.05 and a precision parameter (ϵ) of 0.05 to determine whether sensitivity (SN) and specificity (SP) were equivalent to the reference standard (expert manual microscopy assessment of samples).
Based on these calculations, sample sizes of 692 (for sensitivity) and 173 (for specificity) were required, assuming a predetermined level of sensitivity and specificity of 0.90 [37]. All performed statistical tests were two-sided unless otherwise stated. Diagnostic accuracy was evaluated in terms of sensitivity, specificity, positive and negative predictive values. To measure the correlation between the manual and DLS-based counting of EPG, we calculated the Pearson correlation coefficient. The level of statistical significance was 0.05 and statistical estimates of diagnostic accuracy were reported with 95% confidence intervals (CI95%).
Results
Prevalence of soil-transmitted helminths
After exclusion of slides used for training (n = 388) of the DLS and slides with unsatisfactory quality, 792 slides remained for analysis (Fig 1). Prevalence of the STHs according to manual microscopy and the DLS are presented in Table 1.
Diagnostic accuracy for detection of soil-transmitted helminths by the deep learning system
The sensitivity, specificity, positive and negative predictive values for the detection of STHs by the DLS, as compared to manual microscopy assessment of samples are presented in Table 2.
Discrepancy between the methods was observed especially in the analysis of hookworm samples, where 33 of samples were false negative by the DLS, in which hookworm eggs had been detected by manual microscopy. The largest number of false positive samples was related to detection of T. trichiura, where the DLS detected parasites in 64 samples classified as negative by manual microscopy (Table 3).
Visual review of digitized samples
A review of digitized samples in case of discordant results between manual microscopy and analysis of digital samples by the DLS was performed by the researchers (AS, SV, SH, JL) independently of the laboratory technologist who performed the manual microscopy (FK). A total of 5 of 14 (36%) samples positive for A. lumbricoides, 59 of 64 (92%) samples positive for T. trichiura, and 15 of 30 (50%) positive for hookworm according to the DLS but negative according to manual microscopy, were confirmed positive in the digital slides. A re-assessment of the diagnostic accuracy was performed after visual review of the digital slides, with a combined manual microscopy and visual review as the reference standard and is reported in Supporting Table 1. Examples of the visually verified STHs are shown in Fig 5. Thus, in a total of 79 (10%) out of 792 samples classified as negative for a specific STH species by manual microscopy assessment, the DLS diagnosis could be confirmed as correct. With visual review of digital slides as a corrected reference standard, the most prominent change in diagnostic accuracy was the increase in specificity for T. trichiura from 89.7% of 99.1% (S1 Table). The remaining samples classified as positive by the DLS, but negative according to manual microscopy often displayed artefacts with similar morphologic features as the parasite eggs (S1 Fig).
Visually verified soil-transmitted helminth eggs detected by the deep learning system from slides that were originally classified as negative by manual microscopy: A. lumbricoides (top row), T. trichiura (middle row), and hookworm (bottom row) findings. Scalebar 25 μm.
All the DLS false negative samples i.e. three A. lumbricoides, 14 T. trichiura, and 33 hookworm samples that were recorded as positive according to manual microscopy represented light intensity infections as reported in detail below. In the visual assessment of the DLS false negative samples, glycerol disintegration of hookworm eggs had affected 15 of 33 (45%) samples, as partly disintegrated hookworm eggs which had not been detected by the DLS could be observed in the digital images (S2 Fig). DLS false negative samples typically also had out-of-focus areas or dense parts that could explain that a few eggs seen in manual microscopy were missed by the DLS.
Object-level detection of parasites in the samples by analysis with the deep learning system
Following the sample-level analysis of results, we also compared the number of individual parasite eggs in the samples where results were available for both manual and DLS-based quantification of parasites (n = 241). Egg counts according to species and detection method are shown in Table 4. Overall, 93% of A. lumbricoides, 93% of T. trichiura and 97% of hookworm samples in the test set represented light intensity infections according to manual microscopy.
A majority (n = 170, 72%) of the samples showed higher STH egg counts when analysed by the DLS, compared to the STH egg counts recorded by manual microscopy. Especially for T. trichiura, the EPGs according to the DLS were higher than for manual microscopy for both light and moderate intensity infections (S3 Fig). The correlation between the number of detected STH eggs by manual microscopy and DLS was significant (r = 0.63, p = 0.04). When infection intensity was categorized as light, moderate, and heavy infection according to the WHO guidelines, one case of A. lumbricoides (10%), 13 of T. trichiura (10%) and one case of hookworm infection (1%) changed from light to moderate when analysed with the DLS as compared to manual microscopy, whereas three cases of hookworm infection changed from moderate to light (3%) according to the DLS (Table 5).
For the samples classified as false negative by the DLS, the average number of STH eggs according to manual microscopy was 1 for A. lumbricoides (24 EPG), 5 (110 EPG) for T. trichiura, and 5 for hookworm (129 EPG), all representing light intensity infections. When assessing the object-level findings for the samples classified as false positive by the DLS, we also observed a low level of detected STH eggs in the samples (mean = 4, range 1–16), corresponding to 96 EPG (24–384).
Discussion
Analysis of digitally scanned WSIs of Kato-Katz thick smears with the DLS provided high diagnostic accuracy of the most common STHs. Importantly, a substantial number of light intensity infections were missed by manual microscopy but detected by DLS supported digital microscopy, which challenges the role of expert manual microscopy as the recommended method for detection of STHs. Also, STH egg counts were in average significantly higher assessed by the DLS as compared to manual microscopy. Samples were digitized with a portable digital whole slide scanner and uploaded via local data and mobile networks to a cloud server for analysis with the DLS. The DLS was trained with a subset of the samples to detect the most common STHs (T. trichiura, A. lumbricoides and hookworm) in the digital samples. With manual microscopy as the reference standard, a high level of sensitivity (76–92%) and of specificity (90–98%) was observed for the DLS for the detection of STHs. After visual review of samples with discrepant results between the manual and digital methods, the DLS yielded a low proportion of false positive (1–2%) and false negative (0.4–4%) cases. The discrepant results typically originated from samples with low numbers of visible STH eggs and containing artefacts showing similar morphological features as the parasites. Notably, the DLS detected a high number of samples (n = 79) with STHs which had not been detected by manual microscopy. These samples also primarily represented slides with low numbers of visible parasite eggs, which have previously been recognized as challenging to analyse by manual microscopy [12,13].
Overall, for the DLS, the detection of hookworm eggs yielded the highest discrepancy when compared to manual microscopy and the lowest sensitivity, although specificity was high. As the slides had been digitized with a delay following manual microscopy, a possible contributing factor could be the disintegration of hookworm eggs [37]. This is also supported by the fact that partly disintegrated hookworm eggs could be visually identified in several digital slides classified as positive for hookworm by manual microscopy. By ensuring more rapid sample processing and including also non-intact hookworm eggs in the training data, the accuracy for detection of hookworm could likely be improved. Another potentially contributing factor may be the lower number of manual annotations of hookworm eggs available in the training data (n = 552), compared to the other parasites (n > 2000). On an object-level, we observed a clearly higher number (approximately three times higher) of detected A. lumbricoides and T. trichiura parasite eggs by DLS assessment of slides compared to manual microscopy. Even though the DLSs detected generally higher number of parasite eggs in the positive samples, a significant correlation between the manual and digital results were observed. The manual quantification of individual parasites in large sample areas is cumbersome, and our results suggest that the DLS-based analysis could improve the counting process. An additional explanation for higher counts and detection of initially missed eggs of A. lumbricoides and T. trichiura in the digital samples could be a longer clearing time. However, while hookworm eggs may disintegrate over time, A. lumbricoides and T. trichiura egg counts have also been shown to remain relatively stable up to 50 hours after preparation [38]. The potential effect of time from preparation to analysis on diagnostic accuracy of the digital method should be assessed in future studies, for example by randomizing the Kato-Katz thick smears to either manual microscopy or DLS analysis or to prepare duplicate slides which are processed in parallel with the two methods.
The results from our study support findings from earlier work, where deep learning-based algorithms combined with digital microscopy has shown potential for diagnostics of NTDs and STHs [20,23,25]. A major strength of the current study is that true whole-slide images of samples scanned in a primary healthcare setting in an STH endemic setting were analysed, as compared to several previous studies that have only analysed cropped, smaller areas of samples [23,24,26,27]. We found only one previous study where the entire sample area was digitized in a subset of the samples, but the diagnostic accuracy on a sample level was not reported in that study [25]. Although these results have been encouraging, assessment of diagnostic accuracy requires the analysis of whole slide images (i.e., the entire slide) and a sample-level diagnosis. Here, we obtained a large number (n = 1,335) of conventionally prepared Kato-Katz thick smears and digitized and assessed the diagnostic accuracy of the AI-based method based on the entire slides. As the samples were collected, processed, digitized, and uploaded at a basic laboratory in a primary health care setting, the findings demonstrate how the method can be implemented also in a rural, real-world setting, where access to advanced infrastructure for digital microscopy (such as conventional whole slide scanners) is not available. We were not able to find any previous studies that have applied AI-based methods in a similar basic laboratory facility in a primary health care setting. Moreover, the number of slides analysed in this study exceeds previous similar work with a large margin.
WHO has published a target product profile (TPP) for monitoring and evaluation of STH control programmes [11]. The target product is an in vitro/ex vivo laboratory-based (minimum) or point-of-sampling (ideal) test that allows for quantitative detection of analytes specific to STHs in all age groups. We compared our method to the TPP and found that it fulfils most of the WHO stipulated criteria for a laboratory-based test: Our method can be used in regional or national diagnostic testing laboratories by trained laboratory technicians (< 1-week training); is portable (weighs less than 5 kg including a laptop computer and router), is specific (≥ 94%) to each A. lumbricoides, T. trichiura and hookworm and has a sensitivity of at least 60% for each of the three species, if the combined manual microscopy and visual analysis of digital slides is used as the reference standard. However, the diagnostic accuracy of our method as compared to more sensitive and specific STH diagnostic methods (i.e. closer to a ‘ground truth’) such as PCR-based methods or similar remains to be established. If run on battery, the method also has the potential to be used as a point-of-sampling test, where health personnel and community health workers should be able to perform and interpret the test with only a single day of training [11]. Further, the TPP test for point-of-sampling should allow for a throughput of at least seven samples per hour and its cost should not exceed 3 USD per test. Further studies are needed to assess if our method could meet these requirements.
There are limitations of our study that need to be considered when interpreting the results. Firstly, the microscopy assessment of samples here was performed by only a single expert due to practical reasons (such as challenges related to preservation of Kato-Katz thick smears). The visual evaluation of microscopy samples is known to be subjective [39] and, therefore, prone to variations in sensitivity, especially in light intensity infections [13]. Multiple readers per slide would therefore likely improve the reliability of the reference standard. Also, other diagnostic methods, such as DNA-based methods could be used as the reference standard [16]. As the samples here were prepared at a single laboratory, external validation of the results is also needed to assess generalizability to other settings. Furthermore, as the digitization of samples was performed with a delay after the manual analysis of samples, we consider it likely that the disintegration of hookworm eggs or clearing of A. lumbricoides and T. trichiura eggs could have affected the results. Finally, although the total number of samples showing STH infection was relatively high, the distribution of the different STH species varied due to local prevalence, and therefore we obtained variable numbers of samples representing the different species (e.g. the number of samples positive for A. lumbricoides was lower compared to the other species), which could affect the results. Also of note is that 93–97% of all analysed samples in the test set represented light intensity infections, with very few detected STH eggs per sample.
As the performance of this diagnostic method is also affected by several other factors than the analysis of samples, such as sample collection, preparation, digitization, and data upload future studies should also focus on these steps. Here, for example, the analysis of thicker sample areas proved to be challenging both in manual microscopy and digital analysis of samples, and thus emphasizes the importance of high-quality sample preparation. As a subset of digital samples demonstrated certain areas out of focus and with uneven brightness, scanning of multiple focal planes (e.g., z-stacks) and imaging with high dynamic range could likely also contribute to improved diagnostic accuracy.
Also, the current method relies on internet connectivity, since images are transferred to a cloud environment. Sufficient internet connections might not be available in all rural areas. However, in our studies in rural Kenya and Tanzania [27,40], connectivity has been sufficient and upload of data has not been a bottleneck with speeds of 5–10 megabits per second. There are several advantages of data upload to a central repository in a cloud environment, including opportunities to have human experts in the loop for remote verification, to create large-scale databases, continuously monitor the accuracy of the algorithms and quality of samples, perform surveillance of the target infections and to collect more training data. Ideally, the algorithms should also be applicable without internet connectivity and to run on local devices, but also in that case sharing and transferring data when mobile networks are available should be considered an advantage. In a study on deep learning based detection of Schistosoma haematobium in urine samples, an algorithm was developed that was run locally and reached a high diagnostic accuracy on an image (e.g. tile or field-of-view) level [41]. The computational speed was approximately 10 images per minute (sized 1,520 x 2,028), equivalent to 0.5 megapixels per second as compared to approximately 14 megapixels per second with our method (2500% faster). This means that analysis of one of our WSIs with the described local solution would take almost 2 hours, as compared to 15–25 minutes (upload 10–20 minutes + analysis 5 minutes) with our solution.
The total turnaround time of 20–35 minutes per sample (scanning 5–10 minutes, upload 10–20 minutes and analysis 5 minutes) with our method could be considered a limitation, but there are several opportunities to make the current solution faster. These include more efficient hardware and software, either running locally or in a cloud environment. Also, the speed of mobile networks is likely to improve and access to 5G networks could allow 10-fold faster upload to a cloud environment, which would translate into a 1.5–2.5 minute upload time [42]. In the current study, we decided to use the magnification of 20x that is standard among many commercial microscope scanners and that can serve additional diagnostic purposes such as screening for cervical pre-cancerous lesions in pap smears [40]. It might be feasible to identify parasite eggs using a lower magnification, and a 10x magnification could result in a WSI that is one-quarter the size, thereby further speeding up the processing time. However, the diagnostic accuracy of STH detection using a lower magnification and resolution needs to be assessed in future studies.
Overall, the findings of this study support results of previous studies on AI-based detection of STHs, and adds evidence of feasibility of the use of portable whole-slide digital microscopy scanners combined with deep learning-based analysis of samples for the diagnosis of STHs. Although here we only investigated the most common STH species, this technology can be applied to microscopy diagnostics of other similarly diagnosed infectious diseases, such as other NTDs. Further studies should focus on multi-centre, prospective evaluation of the generalisability of the method, and evaluation of other potential diagnostic targets.
Conclusion
Deep learning-based detection of common soil-transmitted helminths in WSIs of Kato-Katz thick smears is feasible and can reach high diagnostic accuracy using a minimal digital microscopy infrastructure approach that can be applied in basic laboratory facilities in primary health care setting of STH endemic countries. Kato-Katz thick smears can be digitized with a portable slide scanner and uploaded to a cloud environment via a basic laptop computer and local data networks for AI-based analysis. This method could be used to facilitate the diagnostics of STHs in light intensity infections and is also likely to be applicable in field settings, and to digital diagnostics of other conditions, such as other NTDs.
Supporting information
S1 Fig. Examples of false positive objects.
Examples of artefacts classified as parasite eggs by the deep learning system causing a false positive result: (a) A. lumbricoides, (b) T. trichiura, and (c) hookworm (c) findings.
https://doi.org/10.1371/journal.pntd.0012041.s001
(TIF)
S2 Fig. Examples of false negative objects.
Examples of desiccated hookworm eggs found in the digital images causing false negative results in the deep learning system (DLS) analysis.
https://doi.org/10.1371/journal.pntd.0012041.s002
(TIF)
S3 Fig. Scatter plots of eggs per gram (EPG) as quantified by manual microscopy and the deep learning system (DLS) in slides classified as positive by both manual microscopy and the DLS.
The x-axis represents manual microscopy results, while the y-axis depicts the DLS results, separately for a) A. lumbricoides, b) T. trichiura, and c) hookworm. Infection intensity levels for each species are denoted by letters on the plots (L for light, M for moderate, and H for heavy infection).
https://doi.org/10.1371/journal.pntd.0012041.s003
(TIF)
S1 Protocol. Standard operation procedure for preparation of Kato-Katz thick smears.
https://doi.org/10.1371/journal.pntd.0012041.s004
(DOCX)
S1 Table. Diagnostic accuracy for soil-transmitted helminths (STHs) after visual review of the digital samples, i.e. with manual microscopy and visual analysis of digital slides as the reference standard.
CI = confidence interval.
https://doi.org/10.1371/journal.pntd.0012041.s005
(DOCX)
Acknowledgments
We thank the Institute for Molecular Medicine Finland (FIMM) Digital Microscopy and Molecular Pathology Unit (University of Helsinki) and Biocenter Finland (Helsinki Institute of Life Science HiLIFE, University of Helsinki) for access to image management and processing infrastructure during the study. We thank Hakan Küçükel (FIMM) for IT support.
References
- 1.
World Health Organization. Soil-transmitted helminth infections 2023 [updated 18.1.2023; cited 2023 19.4.2023]. Available from: https://www.who.int/news-room/fact-sheets/detail/soil-transmitted-helminth-infections.
- 2. Lenk EJ, Redekop WK, Luyendijk M, Rijnsburger AJ, Severens JL. Productivity Loss Related to Neglected Tropical Diseases Eligible for Preventive Chemotherapy: A Systematic Literature Review. PLoS neglected tropical diseases. 2016;10(2):e0004397. pmid:26890487
- 3. Hofstraat K, van Brakel WH. Social stigma towards neglected tropical diseases: a systematic review. Int Health. 2016;8 Suppl 1:i53–70. pmid:26940310.
- 4. Pabalan N, Singian E, Tabangay L, Jarjanazi H, Boivin MJ, Ezeamama AE. Soil-transmitted helminth infection, loss of education and cognitive impairment in school-aged children: A systematic review and meta-analysis. PLoS neglected tropical diseases. 2018;12(1):e0005523. Epub 20180112. pmid:29329288; PubMed Central PMCID: PMC5766095.
- 5. Christian P, Khatry SK, West KP Jr. Antenatal anthelmintic treatment, birthweight, and infant survival in rural Nepal. Lancet. 2004;364(9438):981–3. pmid:15364190.
- 6. Salam RA, Das JK, Bhutta ZA. Effect of mass deworming with antihelminthics for soil-transmitted helminths during pregnancy. Cochrane Database of Systematic Reviews. 2021;(5). pmid:33998661
- 7.
World Health Organization. 2030 targets for soil-transmitted helminthiases control programmes. Geneva: World Health Organization; 2020.
- 8.
World Health Organization. Bench aids for the diagnosis of intestinal parasites. 2nd ed. Geneva: World Health Organization; 2019.
- 9. Bethony J, Brooker S, Albonico M, Geiger SM, Loukas A, Diemert D, et al. Soil-transmitted helminth infections: ascariasis, trichuriasis, and hookworm. Lancet. 2006;367(9521):1521–32. Epub 2006/05/09. pmid:16679166.
- 10. Miswan N, Singham GV, Othman N. Advantages and Limitations of Microscopy and Molecular Detections for Diagnosis of Soil-transmitted Helminths: An Overview. Helminthologia. 2022;59(4):321–40. Epub 20221230. pmid:36875683; PubMed Central PMCID: PMC9979072.
- 11.
World Health Organization. Diagnostic target product profile for monitoring and evaluation of soil-transmitted helminth control programmes. Geneva: World Health Organization; 2021.
- 12. Stuyver LJ, Levecke B. The role of diagnostic technologies to measure progress toward WHO 2030 targets for soil-transmitted helminth control programs. PLoS neglected tropical diseases. 2021;15(6):e0009422. pmid:34081694
- 13. Nikolay B, Brooker SJ, Pullan RL. Sensitivity of diagnostic tests for human soil-transmitted helminth infections: a meta-analysis in the absence of a true gold standard. Int J Parasitol. 2014;44(11):765–74. Epub 2014/06/30. pmid:24992655.
- 14. Kongs A, Marks G, Verle P, Van der Stuyft P. The unreliability of the Kato-Katz technique limits its usefulness for evaluating S. mansoni infections. Trop Med Int Health. 2001;6(3):163–9. Epub 2001/04/12. pmid:11299032.
- 15. Speich B, Knopp S, Mohammed KA, Khamis IS, Rinaldi L, Cringoli G, et al. Comparative cost assessment of the Kato-Katz and FLOTAC techniques for soil-transmitted helminth diagnosis in epidemiological surveys. Parasit Vectors. 2010;3:71. Epub 2010/08/17. 1756-3305-3-71 [pii] pmid:20707931; PubMed Central PMCID: PMC2936391.
- 16. Mbong Ngwese M, Prince Manouana G, Nguema Moure PA, Ramharter M, Esen M, Adégnika AA. Diagnostic Techniques of Soil-Transmitted Helminths: Impact on Control Measures. Trop Med Infect Dis [Internet]. 2020; 5(2). pmid:32516900
- 17. Vaisman A, Linder N, Lundin J, Orchanian-Cheff A, Coulibaly JT, Ephraim RK, et al. Artificial intelligence, diagnostic imaging and neglected tropical diseases: ethical implications. Bull World Health Organ. 2020;98(4):288–9. Epub 2020/04/15. pmid:32284655; PubMed Central PMCID: PMC7133484.
- 18. Bogoch II, Coulibaly JT, Andrews JR, Speich B, Keiser J, Stothard JR, et al. Evaluation of portable microscopic devices for the diagnosis of Schistosoma and soil-transmitted helminth infection. Parasitology. 2014;141(14):1811–8. Epub 2014/04/30. S0031182014000432 [pii] pmid:24776232.
- 19. Bogoch II, Lundin J, Lo NC, Andrews JR. Mobile phone and handheld microscopes for public health applications. Lancet Public Health. 2017;2(8):e355. Epub 2017/12/19. pmid:29253476.
- 20. Saeed MA, Jabbar A. “Smart Diagnosis” of Parasitic Diseases by Use of Smartphones. Journal of Clinical Microbiology. 2018;56(1):e01469–17. pmid:29046408
- 21. Cringoli G, Amadesi A, Maurelli MP, Celano B, Piantadosi G, Bosco A, et al. The Kubic FLOTAC microscope (KFM): a new compact digital microscope for helminth egg counts. Parasitology. 2021;148(4):427–34. Epub 2020/11/20. pmid:33213534
- 22. Kumar S, Arif T, Alotaibi AS, Malik MB, Manhas J. Advances Towards Automatic Detection and Classification of Parasites Microscopic Images Using Deep Convolutional Neural Network: Methods, Models and Research Directions. Arch Comput Methods Eng. 2023;30(3):2013–39. Epub 20221203. pmid:36531561; PubMed Central PMCID: PMC9734923.
- 23. Yang YS, Park DK, Kim HC, Choi MH, Chai JY. Automatic identification of human helminth eggs on microscopic fecal specimens using digital image processing and an artificial neural network. IEEE Trans Biomed Eng. 2001;48(6):718–30. pmid:11396601.
- 24. Yang A, Bakhtari N, Langdon-Embry L, Redwood E, Grandjean Lapierre S, Rakotomanga P, et al. Kankanet: An artificial neural network-based object detection smartphone application and mobile microscope as a point-of-care diagnostic aid for soil-transmitted helminthiases. PLoS neglected tropical diseases. 2019;13(8):e0007577. pmid:31381573
- 25. Ward P, Dahlberg P, Lagatie O, Larsson J, Tynong A, Vlaminck J, et al. Affordable artificial intelligence-based digital pathology for neglected tropical diseases: A proof-of-concept for the detection of soil-transmitted helminths and Schistosoma mansoni eggs in Kato-Katz stool thick smears. PLoS neglected tropical diseases. 2022;16(6):e0010500. pmid:35714140
- 26. Dacal E, Bermejo-Peláez D, Lin L, Álamo E, Cuadrado D, Martínez Á, et al. Mobile microscopy and telemedicine platform assisted by deep learning for the quantification of Trichuris trichiura infection. PLoS neglected tropical diseases. 2021;15(9):e0009677. pmid:34492039
- 27. Holmström O, Linder N, Ngasala B, Mårtensson A, Linder E, Lundin M, et al. Point-of-care mobile digital microscopy and deep learning for the detection of soil-transmitted helminths and Schistosoma haematobium. Global Health Action. 2017;10(sup3):1337325. pmid:28838305
- 28.
Mazumdar M, Zhong X, Ferket B. Diagnostic Trials. In: Piantadosi S, Meinert CL, editors. Principles and Practice of Clinical Trials. Cham: Springer International Publishing; 2020. p. 1–28.
- 29. Cohen JF, Korevaar DA, Altman DG, Bruns DE, Gatsonis CA, Hooft L, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016;6(11):e012799. Epub 20161114. pmid:28137831; PubMed Central PMCID: PMC5128957.
- 30. Martin LK, Beaver PC. Evaluation of Kato thick-smear technique for quantitative diagnosis of helminth infections. The American journal of tropical medicine and hygiene. 1968;17(3):382–91. pmid:5690644.
- 31.
Montresor A, Crompton DWT, Hall A, Bundy DAP, Savioli L, World Health Organization. Division of Control of Tropical Diseases S, et al. Guidelines for the evaluation of soil-transmitted helminthiasis and schistosomiasis at community level: a guide for managers of control programmes / A. Montresor… [et al.]. Geneva: World Health Organization; 1998.
- 32. Konsti J, Lundin M, Linder N, Haglund C, Blomqvist C, Nevanlinna H, et al. Effect of image compression and scaling on automated scoring of immunohistochemical stainings and segmentation of tumor epithelium. Diagn Pathol. 2012;7:29. Epub 20120321. pmid:22436596; PubMed Central PMCID: PMC3375185.
- 33.
Redmon J, Farhadi A, editors. YOLO9000: Better, Faster, Stronger. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2017 21–26 July 2017.
- 34.
He K, Zhang X, Ren S, Sun J, editors. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 2016 27–30 June 2016.
- 35. Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, et al. Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning? IEEE Trans Med Imaging. 2016;35(5):1299–312. Epub 20160307. pmid:26978662.
- 36. Buderer NMF. Statistical Methodology: I. Incorporating the Prevalence of Disease into the Sample Size Calculation for Sensitivity and Specificity. Academic Emergency Medicine. 1996;3(9):895–900. pmid:8870764
- 37. Tarafder MR, Carabin H, Joseph L, Balolong E Jr., Olveda R, McGarvey ST. Estimating the sensitivity and specificity of Kato-Katz stool examination technique for detection of hookworms, Ascaris lumbricoides and Trichuris trichiura infections in humans in the absence of a ’gold standard’. Int J Parasitol. 2010;40(4):399–404. Epub 20090920. pmid:19772859; PubMed Central PMCID: PMC2829363.
- 38. Bosch F, Palmeirim MS, Ali SM, Ame SM, Hattendorf J, Keiser J. Diagnosis of soil-transmitted helminths using the Kato-Katz technique: What is the influence of stirring, storage time and storage temperature on stool sample egg counts? PLoS neglected tropical diseases. 2021;15(1):e0009032. pmid:33481808
- 39. Linder E, Lundin M, Thors C, Lebbad M, Winiecka-Krusnell J, Helin H, et al. Web-based virtual microscopy for parasitology: a novel tool for education and quality assurance. PLoS neglected tropical diseases. 2008;2(10):e315. Epub 2008/10/23. pmid:18941514; PubMed Central PMCID: PMC2565642.
- 40. Holmström O, Linder N, Kaingu H, Mbuuko N, Mbete J, Kinyua F, et al. Point-of-Care Digital Cytology With Artificial Intelligence for Cervical Cancer Screening in a Resource-Limited Setting. JAMA network open. 2021;4(3):e211740-e. pmid:33729503
- 41. Oyibo P, Meulah B, Bengtson M, van Lieshout L, Oyibo W, Diehl J-C, et al. Two-stage automated diagnosis framework for urogenital schistosomiasis in microscopy images from low-resource settings. Journal of Medical Imaging. 2023;10(4):044005. pmid:37554627
- 42. Amuomo N. The Evolution of GSM Technologies into 5G and the Imminent Emergence of Transformative Telemedicine Applications: A Review. East African Journal of Information Technology. 2020;2:8–17.