Study design.

Retrospective, comparative diagnostic study, intended to estimate the sensitivity, specificity, positive and negative predictive values of the tests examined.

Study population.

The study was carried out at the Centre for Tropical Diseases (CTD) of Sacro Cuore Hospital (Negrar—Verona, Italy), a referral center for tropical and parasitic diseases in the country. The study population was retrieved from the CTD patient database and was mostly composed by African recent asylum seekers that were routinely screened for schistosomiasis, regardless if symptomatic or not, in the context of a regional, pilot programme. The study was meant to include all consecutive subjects screened in the 2-year period considered (from March, 2014, when CCA urine dipstick was first made available). The study was carried out at the reference laboratory for parasitic diseases of CTD by experienced professionals.

The results of the routine screening tests (CCA, ELISA and microscopy) were retrieved from the patient database, while the new tests (ICT and WB) were performed for the study purpose on fully anonymized, coded serum samples, routinely stored and cryo-preserved at CTD. The archived specimens were kept frozen at -80°C from the day of the sample collection and the tests were executed within 24 hours of unfreezing. The study covered the two-year period from 1^st March, 2014 to 28^th February, 2016.

Inclusion criteria. Subjects of all ages submitted to routine screening for schistosomiasis with CCA urine dipstick, ELISA serology and microscopy of the stools and urine.

Exclusion criteria. a) Lack of informed consent for usage of biological samples for study purpose; b) Missing result for one of the routine screening tests (CCA urine dipstick, ELISA serology and microscopy,); c) Unavailability of ICT and/or WB result.

Primary endpoints.

Sensitivity, specificity, odds ratio, positive and negative predictive values of the index tests according to the primary reference standard.

Secondary endpoints.

a) Sensitivity, specificity, odds ratio, positive and negative predictive values of the index tests according to CRS and LCA, respectively; b) Concordance between CRS and LCA in identifying true positive and true negative cases.

Test methods.

Primary Reference standard: direct detection of S. mansoni or S. haematobium eggs in stools or urine. Composite Reference Standard. The subjects were classified as: Infected (denominator for sensitivity): positive microscopy OR (negative microscopy AND at least 2 concordant positives of the four index tests). Not infected (denominator for specificity): negative microscopy AND < 2 positives of the four index tests.

Index tests included four commercial tests: Urine CCA dipstick test (NADAL CCA Bilharzia test, Germany), Bordier ELISA (Bordier Affinity Products, Crissier, Switzerland), Schisto II Western Blot IgG (LD-BIO Diagnostics, Lyon, France) and Schistosoma ICT IgG-IgM (LD-BIO Diagnostics, Lyon, France).

A brief description of all the methods follows.

Urine CCA dipstick test.

One drop of urine sample was added to the CCA urine cassette, following the manufacturer's instructions. Once absorbed, one drop of buffer was added and the result was read 20 minutes after. Valid tests were scored as positive or negative depending on whether or not a colored band appeared. The reading was done independently by two observers, and, in case of discordance, by a third observer. In case a result was judged equivocal by two observers (usually due to a very faint band), it was reported as indeterminate.

Urine microscopy.

The remaining urine was shaken and filtered through a 25-mm diameter small meshed filter (12 μm Nucleopore), which was then placed on a labeled slide and examined under a microscope (100x) for Schistosoma eggs.

Stool microscopy.

Three stool samples per patient were collected in formol on consecutive days. Each sample was submitted to formol-ether concentration and examined (100x) [26].

Bordier ELISA.

The test detects Schistosoma IgG antibodies by using antigens from adult of S. mansoni. Briefly, 10 μl of serum were diluted with 2 ml of buffer before adding 100 μl to each ELISA microwell. According to the manufacturer's instructions, the result is positive when the absorbance of the analyzed sample is higher than the absorbance of the weak positive control (provided with the kit). In order to be able to compare results from different runs, we defined as positive samples those with: optical density (OD) of study sample/OD of weak positive serum ≥ 1 (normalized OD). The test was carried out in double and, in case of discordant results, it was further repeated on another blood sample, in order to avoid indeterminate results.

Schisto II Western Blot IgG.

The test was carried out according to the manufacturer's instructions. Briefly, the S. mansoni + S. haematobium antigen strips (supplied with the kit) were incubated with 25 μl of sera diluted in Tris-NaCl sample buffer for 90 minutes. Then, a first washing step with Tris-NaCl washing buffer permitted to remove any unbound sample components. Later, the strips were incubated with an alkaline phosphates—anti human IgG conjugate for 60 minutes. After a second washing step, the whole complex was revealed by the corresponding substrate-chromogenic solution containing nitroblue tetrazolium and 5-bromo-4-chloro-3-indolylphosphate as a dark blue-purple color. The color development was stopped after 60 minutes by washing the strips with distilled water. The strips were dried and glued to paper for reading. The Schistosoma-specific IgG, if present in the sample, appeared as violet colored bands. Valid tests were read as positive depending on the presence of one of the following: 1) band P22-24; 2) a large band P30-34; 3) a thin band P30-34 plus at least one among bands P8, PP9, P10, P12-13, P14-15, P18. The strips were read independently by two observers, and, in case of discordance, by a third observer. In case a result was judged equivocal by two observers (typically for the appearance of a “shadow” rather than a clear band), it was reported as indeterminate.

Schistosoma ICT IgG-IgM.

This immune-chromatographic test (ICT) was carried out according to the manufacturer's instructions. Briefly, 30 μl of serum were added to a cassette, followed by 2 drops of the eluent (supplied with the kit). The result was read after 20 minutes. The tests were read as positive or negative depending on whether or not a colored band had appeared. The reading and the interpretation were carried out the same way as for CCA (see above).

Cost.

The approximate cost per test (in Italy, and without considering the labour costs) is € 7 for ELISA, € 9 for CCA, € 14 for ICT and € 33 for WB.

Handling of indeterminate results.

For the analysis, all indeterminate results were classified as positive. All reference and index tests were executed and read by highly trained, senior staff of the CTD laboratory. Four senior staff members performed and read all the tests, according to the procedures outlined above.

Blinding.

Microscopy, serology and urine CCA dipstick test were carried out at CTD by different laboratory personnel. The staff performing the tests carried out on the frozen specimens (ICT and WB) had no access to the source codes and therefore were blinded as of the results of the previous tests, as well as of the results of either test.

Statistical analysis.

For both the primary and secondary endpoints, the sensitivity of each index test was calculated as the proportion of positive results over all infected subjects according to the primary and composite reference standards, respectively. Uncertainty was quantified using the 95% confidence intervals. Similarly, the specificity was calculated as the proportion of negative results over all not infected subjects according to the primary and composite reference standard, respectively. Uncertainty was quantified the same way as above.

A further analysis was performed using the method of Latent Class Analysis (LCA).

Latent class models (LCMs) combine the results of multiple diagnostic tests through a probabilistic model to obtain estimates of disease prevalence and diagnostic test accuracy in situations where there is no single, accurate reference standard [42].

Diagnostic studies that apply LCMs treat the target disease status as an unmeasured (”latent”) categorical variable with K classes, reflecting the levels of the underlying disease. The manifest variables, the outcomes of T (binary) diagnostic tests, are considered to be imperfect classifiers of the disease status. The LCM describes a statistical model relating the manifest variables to the latent disease status.

When K = 2, it is assumed that the two latent classes correspond to a class of subjects in which the target disease is present and a class of subjects in which the target disease is absent. Parameter estimates obtained from the 2-class LCM are interpreted as estimates of the sensitivity and specificity of each test and the prevalence of the target disease (i.e., the prior probability that the target disease is present).

We assumed the conditional independence among the tests.

The analysis was performed by using LCA, random LCA and extended LCA R’s libraries.

Predictive values (PPV, NPV) were then estimated, based on the prevalence of the infected subjects obtained with the three methods (microscopy, CRS and LCA).

We also assessed the concordance between CRS and LCA in classifying the study subjects. Considering as cases all subjects classified in Class 1 by LCA, and negatives (non-cases) all subjects classified in Class 2 by LCA, we compared the proportion of subjects classified as cases and non-cases by CRS and LCA, respectively.

Cohen’s Kappa measure (with its 95% confidence interval) was used to assess the agreement as follows: K ≤ 0, no agreement; K = 0–0.20, poor agreement; K = 0.21–0.40, fair agreement; K = 0.41–0.60, moderate agreement; K = 0.61–0.80, substantial agreement; and K = 0.81–1.00, nearly perfect agreement [44,45].

Ethical issues.

Samples were anonymously coded, unlinked from any information identifying the source individuals. At the Centre for Tropical Diseases, all subjects submitted to any serologic exam are asked to sign an informed consent for the anonymous storage of a serum specimen for any future research purpose. Parents’ or legal guardians’ consent is obtained in case of individuals of less than 18 years of age. Specimens lacking the consent are immediately discarded. Although the study was retrospective and no action on patients was involved, the study protocol was nevertheless submitted to the Ethics Committee of the Coordinating Site (Comitato Etico Provinciale di Verona e Rovigo) for approval. The latter acknowledged the study protocol and formally authorized the study (protocol n. 33909 of July 13^th, 2016).

Strengths and weaknesses.

We believe that a major strength of our study is the reference standard used. Direct demonstration of eggs in stools and/or urine is of course diagnostic, but this technique is also known to lack sensitivity, as infections with a low parasitic load are easily missed [25–27]. It is clear that any test with a poor sensitivity cannot be used as a gold standard to evaluate the accuracy of alternative, index tests, as discordant results would be impossible to classify. Yet, most diagnostic studies in parasitology, and on schistosomiasis in particular, rely on microscopy as the gold standard [27], and only rarely alternative reference standards are used, although the latter are the only acceptable methods when a gold standard is missing [40–42]. In this study we used, besides microscopy, two of the alternative methods that are recommended when a gold standard is not available, that are, a CRS and the LCA. Both methods provided very similar results, for all the index tests considered, while when microscopy was used as reference, sensitivity of the index tests was similar as with the alternative methods, but specificity was invariably lower (reflecting the likely misclassification of discordant results).

We acknowledge also some limitations of the models used. As far as the CRS is concerned, it may be argued that it is not correct to include an index test among the tests composing the reference standard. Moreover, the latter tests should not be conditionally dependent, while in fact three of them were based on the same principle, that is, antibody detection, and therefore conditional independence cannot be assumed. Besides, antibody detection does not necessarily reflect an active infection.

In our retrospective, real-life study, we aimed at estimating the accuracy of all available tests, and this is the reason why all the index tests were also part of the reference standard. This was the only way of obtaining the same denominator of infected/not infected for all the tests examined. Precisely because we were aware of the limitations of the CRF used, we decided to also use a third, alternative method, namely the LCA.

More in general, the main weakness of our study was the retrospective design. Although most of the tests were carried out for screening purpose and not because of clinical suspicion, some selection bias cannot be ruled out. Moreover not all the tests were always available during the study period, and the subjects lacking any of the tests under study had to be excluded from the analysis. Finally, the estimation of the prevalence in a hospital setting may not be representative of the general population, although in this case, as explained above, most tests were carried out on subjects submitted to a pilot screening, regardless the presence/absence of symptoms.

Both ICT and CCA tests imply the visual reading by the observer of a band appearing within 15 minutes on the device. Theoretically, a classification of positive results (e.g. “strong positives” and “weak positives”) would be possible, based on the band intensity, also considering that the few results originally classified as undetermined obviously concerned faint positives for which an agreement was not reached). Unfortunately the test results were only classified as positive, negative or undetermined, therefore this distinction was not possible and will be certainly taken into account in a prospective study (see below). Automatic readers could possibly improve the use of these and other qualitative diagnostic tools in the future.

Future research.

We are planning an extensive, multi centre, longitudinal diagnostic study in order to assess the best diagnostic and screening strategy for schistosomiasis. Bio-molecular methods (that were not included in this study) will be also assessed. We are also planning to compare the results of the ICT performed after finger prick with those on serum obtained according to the standard procedure. If they are concordant, the use of ICT will be greatly facilitated outside health facilities.