Validation of two multiplex platforms to quantify circulating markers of inflammation and endothelial injury in severe infection

Aleksandra Leligdowicz; Andrea L. Conroy; Michael Hawkes; Kathleen Zhong; Gerald Lebovic; Michael A. Matthay; Kevin C. Kain

doi:10.1371/journal.pone.0175130

Abstract

Biomarkers can prognosticate outcome and enable risk-stratification. In severe infection, focusing on multiple markers reflecting pathophysiological mechanisms of organ injury could enhance management and pathway-directed therapeutics. Limited data exist on the performance of multiplex biomarker platforms. Our goal was to compare endothelial and immune activation biomarkers in severe pediatric infections using two multiplex platforms. Frozen plasma from 410 children presenting to the Jinja Regional Hospital in Uganda with suspected infection was used to measure biomarkers of endothelial (Angiopoietin-2, sFlt-1, sVCAM-1, sICAM-1) and immune (IL-6, IP-10, sTNFR-1, CHI3L1) activation. Two multiplex platforms (Luminex®, Ella^TM) based on monoclonal antibody sandwich immunoassays using biotin-streptavidin conjugate chemistry were selected with reagents from R&D Systems. The two platforms differed in ease and time of completion, number of samples per assay, and dynamic concentration range. Intra-assay variability assessed using a coefficient of variation (CV%) was 2.2–3.4 for Luminex® and 1.2–2.9 for Ella^TM. Correlations for biomarker concentrations within dynamic range of both platforms were best for IL-6 (ρ = 0.96, p<0.0001), IP-10 (ρ = 0.94, p<0.0001) and sFlt-1 (ρ = 0.94, p<0.0001). Agreement between concentrations obtained by both methods assessed by the Bland-Altman test varied, with best agreement for CHI3L1. Our data suggest that biomarkers of endothelial and immune activation can be readily measured with multiplex platforms. Luminex® and Ella^TM produced reliable results with excellent CV% values. The Ella^TM platform was more automated and completed in 75 minutes, potentially compatible with near-patient use. Trends in concentrations obtained by these methods were highly correlated, although absolute values varied, suggesting caution is required when comparing data from different multiplex platforms.

Citation: Leligdowicz A, Conroy AL, Hawkes M, Zhong K, Lebovic G, Matthay MA, et al. (2017) Validation of two multiplex platforms to quantify circulating markers of inflammation and endothelial injury in severe infection. PLoS ONE 12(4): e0175130. https://doi.org/10.1371/journal.pone.0175130

Editor: Viswanathan V. Krishnan, California State University Fresno, UNITED STATES

Received: November 30, 2016; Accepted: March 21, 2017; Published: April 18, 2017

Copyright: © 2017 Leligdowicz et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: This work was supported by the Canadian Institutes of Health Research (CIHR) grants MOP-13721, MOP-115160, MOP-136813, a CIHR Foundation grant (KCK FDN-148439), the Canada Research Chair Program (KCK), a CIHR Banting fellowship (AL), and donations from Kim Kertland and the Tesari Foundation. MAM has received grants from Amgen for experimental studies on acute lung injury and from GlaxoSmithKline for studies of sepsis in patients; consultancy agreements for ARDS or lung diseases with Boehringer-Ingelheim, Bayer Inc, Cerus Therapeutics, Biomarck Pharmaceuticals, Quark Pharmaceuticals, Thesan Pharmaceuticals, Incardia Inc, Biogen, and GlaxoSmithKline; DSMB Chair for clinical trials for Roche-Genentec. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: MAM has received grants from Amgen for experimental studies on acute lung injury and from GlaxoSmithKline for studies of sepsis in patients; consultancy agreements for ARDS or lung diseases with Boehringer-Ingelheim, Bayer Inc, Cerus Therapeutics, Biomarck Pharmaceuticals, Quark Pharmaceuticals, Thesan Pharmaceuticals, Incardia Inc, Biogen, and GlaxoSmithKline; DSMB Chair for clinical trials for Roche-Genentec. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Introduction

The search for novel biological markers to predict response to therapies, prognosticate outcome, or assist in patient enrollment in clinical therapeutic trials is quickly evolving [1]. In the context of life-threatening infection, many biomarkers have been proposed to improve the discriminatory ability to achieve these goals [2, 3]. Emphasizing markers of pathophysiological pathways involved in severe infections and focusing on multiplex platforms with near-patient or point-of-care potential, could accelerate the development of precision medicine tools for life-threatening infections [4, 5].

The third iteration of the international consensus definitions for sepsis and septic shock [6] acknowledged that while multiple candidate biomarkers have been evaluated, robust validation is required prior to incorporating them into a clinical definition of sepsis. However, many challenges exist with the validation of biomarkers, including inconsistency in the biological reagents used, the detection platforms utilized, the combinations and permutations of markers tested, the diversity of patient cohorts from which samples are derived, and the statistical tests used to analyze the results.

Of the many pathophysiologic pathways that may contribute to the high morbidity and mortality of severe infections, endothelial [7, 8] and immune activation [9, 10] have been studied extensively. Key biomarkers of endothelial activation include the Angiopoietin-Tie2 axis [11], the soluble variant of the vascular endothelial growth factor, a receptor known as soluble fms-like tyrosine kinase-1 (sFlt-1) [12, 13], soluble vascular cell adhesion molecule-1 (sVCAM-1) and soluble intercellular adhesion molecule-1 (sICAM-1) [14, 15]. Multiple markers of inflammation have been identified and among these interleukin 6 (IL-6) [16], interferon-gamma-inducible protein-10 (IP-10, CXCL10) [17], chitinase-3-like-1 protein (CHI3L1) [18] and soluble tumor necrosis factor receptor-1 (sTNFR-1) [15, 19] have been correlated with severity of illness and clinical outcome in sepsis as well as other critical care illnesses such as the acute respiratory distress syndrome, usually caused by severe infection [20, 21].

The quantification of plasma proteins still largely relies on the use of enzyme-linked immunosorbent (ELISA)-based assays [22]. In the past decade, the simultaneous detection of multiple distinct proteins was enabled by using highly specific capture and detection monoclonal antibodies [23–26]. The detection antibodies can be conjugated to different indicators, allowing for the quantification of properties such as optical density, electrochemiluminescence, chemiluminescence, or fluorescence intensity. The property of the indicator as well as the instrument used to detect it lead to variation in the sensitivity and dynamic range of the assay, the number of analytes that can be simultaneously analyzed, and reagent cost. Based on its established performance over the past 20 years, we selected the magnetic microsphere-based Luminex® platform [26–29] and compared it to a novel, fully automated microfluidics-based platform, Ella^TM [30–32].

The main goal of this study was to use reagents prepared by a single manufacturer (R&D Systems) to compare concentrations of previously identified biomarkers of endothelial and immune activation in patients with severe infection measured by two different multiplex platforms. Secondly, we address the challenge of using appropriate statistical tests to compare the performance characteristics of two platforms, with an analytic focus on the ease of use, assay dynamic range, intra-assay variability, and agreement between biomarker concentrations computed by each platform. Our findings have important implications for future near-patient and point-of-care biomarker quantification in life-threatening infection.

Materials and methods

Patient sample selection

A previously described prospective cohort of 2,085 consecutive febrile children aged 2 months to 5 years old who presented to the Jinja Regional Referral Hospital in Uganda, 4.7% of whom died during hospital admission, [33] was used to generate a subcohort of patients for this study. Subjects for this nested case cohort design were selected by randomly sampling 18% of the whole cohort and adding all non-sampled deaths that occurred during hospital admission. The subcohort included 410 children, 99 of whom died during hospital admission and 301 who survived to hospital discharge.

Ethical approval was obtained from Uganda National Council for Science and Technology, Makerere University Research Ethics Committee in Uganda, and the University Health Network. Written informed consent for all study participants was provided by the parent or caregiver.

Plasma sample preparation

Up to 1 ml of whole blood was collected by venipuncture and anticoagulated using ethylenediaminetetraacetic acid (EDTA). Blood was centrifuged within 4 hours of sample collection and plasma was frozen at -80°C without freeze-thaw until analyzed. Samples were thawed overnight at 4°C and aliquoted at room temperature immediately prior to assay performance.

Luminex® platform protocol

Reagents for Luminex® assays were custom developed by R&D Systems. Thirteen biomarkers were selected and divided into 2 panels based on relative plasma abundance and assay dynamic range. Panel 1 included the following 5 high-abundance biomarkers tested at a dilution of 1:30: sVCAM-1, sICAM-1, sTNFR-1, CHI3L1, Cystatin C. Panel 2 included the following 8 low-abundance biomarkers tested at a dilution of 1:3: Angiopoeitin-2 (Ang-2), sFlt-1, IL-6, IP-10, Ang-1, IL-8, sTREM-1, Granzyme B. The 5 biomarkers that did not overlap between the platforms are presented in S1 Table. Unfiltered plasma was diluted using diluents supplied by the manufacturer. Each 96-well plate included 7-fold serial dilutions of standards tested in duplicate and 72 patient samples, 8 of which were tested in duplicate. A total of 6 batches of Luminex Panel 1 and 2 were necessary to complete analysis of 410 samples. Assays were performed according to manufacturer’s magnetic Luminex® screening assay protocol [34]. Briefly, a microparticle cocktail, diluted plasma, and biomarker standards were added to a 96-well plate. Following a 2-hour incubation, plates were washed and a biotin antibody cocktail was added. After a 1-hour incubation, plates were washed and streptavidin-Phycoerythrin (PE) was added for 30-min, followed by a final wash and resuspension in wash buffer. All incubations were done at room temperature on a microplate shaker at 800 rpm. Plates were read immediately on the MAGPIX® instrument and raw data were analyzed using the xPONENT® software. Values outside the lower limit of quantification were assigned a value of 1/3 of the lower limit of the standard curve.

Ella^TM platform protocol

Reagents for the Simple Plex^TM Ella microfluidic platform (Protein Simple, CA, USA) were custom developed. Eight biomarkers that overlapped with the Luminex® platform were selected and divided into 2 panels based on relative plasma abundance and assay dynamic range. Panel 1 included the following 4 high-abundance biomarkers tested at a dilution of 1:100: sVCAM-1, sICAM-1, sTNFR1, CHI3L1. Panel 2 included the following 4 low-abundance biomarkers tested at a dilution of 1:10: Ang-2, sFlt-1, IL-6, IP-10. Plasma samples were analyzed on the same day and after a single thaw, the same as the corresponding samples analyzed using the Luminex® platform. Unfiltered plasma samples were diluted using diluents supplied by the manufacturer and assays were performed according to manufacturer’s protocol. Briefly, 50 μl of diluted plasma was added to the appropriate cartridge, followed by placement in to the Ella instrument, requiring no further user intervention. Each cartridge included a built-in lot-specific standard curve and samples were run as internal triplicates. This was accomplished by the presence of three nanorods present inside each individual channel corresponding to a single biomarker and each nanorod was coated with biomarker-specific capture monoclonal antibodies [30]. Detection monoclonal antibodies and streptavidin-DyLight650 conjugate as well as all washing steps were automatically performed by the instrument. Data were manually screened from each run for clogged nanorods or >20% discrepancy in readings between one of the three nanorods. Sixteen samples were accommodated by each cartridge, 5–6 cartridges were performed per day, and all assays were completed within 6 days. Raw data were analyzed using the SimplePlex Explorer software.

Statistical analysis

Intra-assay variability was calculated using coefficient of variation (CV%) values [35] with the following formula: [standard deviation (σ)/mean (μ)]*100. The CV% values for the Luminex® platform were calculated from concertation values of duplicate samples from subjects who were part of the main cohort. 156 samples were included, spread across 20 plates. The CV% values for the Ella^TM platform were calculated from internal triplicate values of samples from all patients who were part of the subcohort. 410 samples were included, spread across 26 cartridges.

Platform comparisons were performed for values within dynamic range of both assays using three statistical tests [36, 37]. Spearman’s rank correlation was used to compute relationships of biomarker concentrations obtained by each platform. The paired Wilcoxon signed rank sum test was used to compare the median values obtained by each platform. The Bland-Altman method [38] was used to measure agreement between of Log_e-transformed biomarker concentrations obtained by each platform. The variance of differences in concentration ([Log_e Luminex]-[Log_e Ella]) across the mean of concentration values (([Log_e Luminex]+[Log_e Ella])/2) was calculated using the Pitman's test [39]. Linear mixed effects models were used to calculate the 95% confidence intervals and p-values for the difference between Log_e-transformed biomarker concentrations (bias), correcting for random Luminex batch effects for values within dynamic range of both assays.

Statistical analyses were performed using STATA v14.1 (StataCorp 2015). Graphical presentation was done using Excel (Microsoft 2016) and Prism v5.0a (GraphPad 2007).

Results

Biomarker assay performance

The Luminex® and Ella^TM platforms differed in ease of assay performance, number of samples analyzed per run, and time to perform each assay. Both platforms required optimization of plasma dilutions to ensure biomarker concentrations fell within the dynamic range of the assay.

The time to complete a Luminex assay was approximately 5 hours, accommodating up to 80 samples and up to 49 analytes per plate. The assay required manual preparation of serial dilutions of analyte standards, addition of pre-diluted samples, micro-particles, biotin-labeled antibodies, streptavidin substrate, three triplicate wash steps, and loading the plate into a MAGPIX instrument.

The time to complete an Ella assay was 1.25 hours, accommodating up to 16 samples and up to 4 analytes per plate. The Ella^TM platform requires loading the pre-diluted samples into a cartridge and inserting it into the instrument. No further user interface is required since the Ella instrument is fully automated, performs all assays in triplicate and uses internal microfluidics for completion of all immunoassay wash and detection steps.

Biomarker assay limits of quantification and dynamic range

The Luminex® and Ella^TM platforms differed in lower limit of quantification (assay sensitivity) and biomarker dynamic range (Table 1). For samples measured using the Luminex® platform at an appropriate plasma dilution, 63.7–99.5% were within assay dynamic range. Majority of values outside of dynamic range were below the lower limit of quantification, suggesting lower assay sensitivity. For samples measured using the Ella platform at an appropriate plasma dilution, 94.6–100% were within assay dynamic range. The Ella^TM dynamic range was superior compared to Luminex® platform using reagents from R&D System especially, especially at the lower limits of quantification, and the only biomarker with values below the lower limit of assay quantification was IL-6. sICAM-1 values were not included in analyses as the sample dilution was inappropriate for reliable concentration quantification of values above the upper limit of the standard curve.

Download:

Table 1. Luminex® and Ella^TM multiplex platforms: dynamic range and limits of quantification for biomarkers of severe infection.

https://doi.org/10.1371/journal.pone.0175130.t001

Intra-assay variability

Intra-assay performance was calculated using coefficient of variation (CV%) values, to determine the precision of the concentration values obtained by both platforms [35]. The results are shown in Fig 1.

Download:

Fig 1. Intra-assay variability for biomarker concentrations determined using the Luminex® and Ella^TM platforms.

Coefficient of variance (CV%) values for 156 sample assayed in duplicate by the Luminex® platform (shaded circles) and for 406–410 samples assayed by internal triplicate by the Ella^TM platform (open triangles). Analysis for sICAM-1 was excluded due to inappropriate dilution range. Graphs depict point estimates of the mean CV% and the corresponding 95% confidence intervals.

https://doi.org/10.1371/journal.pone.0175130.g001

For the Luminex® platform, 156 random samples were assayed in duplicate. The CV% for the 8 biomarkers varied between 2.2–3.4. For the Ella^TM platform, all samples were run as internal triplicates and 2.7–12.3% of biomarkers were analyzed in duplicate due to an occlusion in one of three glass nanorods. For the 7 biomarkers analyzed at appropriate dilutions, the CV% varied between 1.2–2.9.

Inter-assay variability was determined by quantifying the same healthy control sample on each day of analysis (total of 6 days). The same sample was frozen in multiple aliquots and each aliquot was thawed once on the day of the assay. The quantified biomarkers for the healthy control sample were at the lower end or below the limit of quantification for both multiplex platforms. Therefore, inter-assay variability was greater than if a sample with biomarker concentration within the mid-zone of the standard curve was selected (S2 Table).

Comparisons of concentration correlations

Biomarker concentrations within dynamic range of both assays were compared using the Spearman correlation and paired Wilcoxon signed rank sum test. The strength of association for biomarker concentrations obtained using both platforms was excellent. For biomarkers quantified at appropriate dilutions, the strength of association varied between Spearman’s rho correlation coefficient (ρ) values of 0.79–0.97 (p-values <0.0001 for all analytes, Fig 2). The absolute median values for individual biomarkers quantified using the Luminex® and Ella^TM platforms varied (Table 2). For biomarker concentrations within the dynamic range of both multiplex platforms, there was no difference between the median CHI3L1 concentrations (p value = 0.99) but other biomarker concentrations varied significantly between the two multiplex platforms.

Download:

Fig 2. Relationship between raw untransformed concentration values obtained by the Luminex® compared to the Ella^TM platform.

Spearman’s rho correlation coefficient (ρ) and p-values obtained using Spearman rank correlation for values (n) within assay dynamic range of both platforms.

https://doi.org/10.1371/journal.pone.0175130.g002

Download:

Table 2. Correlation between Luminex® and Ella^TM platform concentrations in pg/mL.

https://doi.org/10.1371/journal.pone.0175130.t002

Agreement between platforms

The Bland-Altman method [38] was used to assess agreement between Log_e-transformed biomarker concentrations generated using the Luminex® and Ella^TM platforms. This graphical tool allows for depicting systematic bias between two methods of measurement and significant outlier values across the range of assay values.

As shown in Fig 3, the Log_e-transformed concentrations of the 7 biomarkers showed varying patterns of agreement. These results were consistent with significant differences in median values demonstrated by paired Wilcoxon signed rank sum test shown in Table 2. A positive bias was indicative of higher concentration values measured by the Luminex® platform, while a negative bias higher concentrations obtained by the Ella^TM platform, A value close to zero meant that there was no difference between the two platforms for concentrations within the dynamic range of the two platforms. A significant bias towards one of the platforms was identified for all biomarkers, with the exception of CHI3L1 (Table 3). Using the Pitman's test of difference in variance, the bias for most biomarkers was consistent across the range of concentrations of both platforms, (p-values displayed in Fig 3).

Download:

Fig 3. Bland-Altman plots comparing agreement between biomarker concentrations determined using the Luminex® and Ella^TM platforms.

Upper and lower limits of agreement (dotted black lines) correspond to 2 standard deviations (SD) away from the mean difference (dashed black line). P-values represent Pitman's test of difference in variance, with a non-significant value indicating no variance across the range of mean values determined by both platforms. Analyses were performed on Log_e-transformed biomarker concentrations that were within assay dynamic range of both platforms (summarized in Table 2).

https://doi.org/10.1371/journal.pone.0175130.g003

Download:

Table 3. Bland-Altman bias representing difference in Log_e-transformed biomarker concentrations computed by Luminex® compared to the Ella^TM platform for values within dynamic range of both assays.

https://doi.org/10.1371/journal.pone.0175130.t003

Discussion

In the past decade, the use of multiplex immunoassays has made the quantification of multiple analytes obtained from clinical samples more feasible. However the variety of potential platforms and reagents has created challenges in selecting optimal assays for large-scale projects that demand consistent results and ultimately has hindered the use of these platforms in clinical research and their translation into clinical practice [25, 27, 40]. The lack of available data comparing multiplex platforms using consistent reagents makes it difficult to compare previous studies examining host biomarkers of severe infections. In this study, we evaluated two different multiplex platforms (Luminex® and Ella^TM) using reagents from a single manufacturer (R&D Systems) to test over 400 well-annotated plasma samples from febrile children presenting with suspected severe infections. We found that the correlations between values for biomarkers of endothelial and immune activation tested within assay dynamic range were excellent. Moreover, using two different statistical methods, we identified consistent differences between absolute biomarker concentrations obtained using the two different multiplex platforms.

The selection of a multiplex immunoassay platform for a specific clinical project is based on many factors. When considering the optimal platform, we evaluated the availability of reagents for analytes of interest, the dilution factors to enable inclusion of multiple biomarkers in a single assay, the ease of assay performance, the time required to complete the assay with potential for translation to near-patient-care settings, as well as cost. Based on these criteria, the Luminex® [27] and Ella^TM [30] platforms were selected. Both platforms were reliable, with CV% for both platforms less than 5%, which is below the acceptable Food and Drug Administration (FDA) recommendations [41]. The Ella^TM platform has the advantage of being fully automated with internal triplicate sample testing, decreasing the sample and reagent quantity, and time required for assay performance. Although the per analyte cost was 5.8-fold greater for the Ella relative to the Luminex platform, the expense related to staff time for assay performance and data analysis could decrease this difference.

All of the biomarkers included in this study have been associated with severity of illness in severe infection [7–19]. Therefore, results of this study provide valuable data to guide future research in predictive and prognostic markers for life-threatening infections. In addition, several biomarkers evaluated here are noteworthy for their roles in infection-induced endothelial injury and as therapeutic targets for future clinical trials [42–44]. Validating a platform with minimal user interface and fast turn-around times would enable its use at near-patient-care to facilitate patient recruitment into novel clinical therapeutic trials.

Of note, there were significant differences observed in absolute biomarker concentrations between these two platforms. The bias towards higher concentrations with either the Luminex® or Ella^TM platforms, however, was uniform across the concentration ranges for most biomarkers as identified by the non-significant Pitman test p-values. This suggests that values of the biomarker concentrations were not different between the platforms at a different concentration range, making the relative values between the two tests comparable. Also worth noting is that although the absolute biomarker concentration values were statistically significantly different, the visual representation of the Log_e-transformed biomarker concentrations displayed on Bland-Altman plots suggest that these concentrations were in a similar pg/mL range. The differences in absolute concentrations could be attributed to differences in the capture and/or detection monoclonal antibody specificity and/or affinity as described in S3 Table. Alternatively, competition between multiple monoclonal antibodies for antigen detection in the multiplex Luminex® assay could have been overcome with the Ella^TM platform multiplex design which uses independent microfluidic channels for each analyte. Finally, although both methods used reagents from R&D Systems and fluorescence was used as the indicator, the fluorochrome conjugated to streptavidin differed (Phycoerythrin (PE) for Luminex® and DyLight650 conjugate for Ella^TM), affecting how the fluorochrome was excited, and what emission wavelength was detected by each platform, possibly contributing to a difference in concentration ranges on both lower and upper limits of quantification.

Selecting a platform to analyze large number of samples in a short time frame is best performed with an assay that can be done in multi-well format, enabling high throughput. Of the two platforms we tested, the Luminex® platform was better suited to this purpose. However, when near-patient or point-of-care data is required, with fast turn-around and with the capacity to test fewer samples, the Ella^TM platform offers the required characteristics. Our data provide evidence that concentrations obtained by both platforms are highly correlated. Therefore, it is possible to use either platform for clinical research examining a relationship between biomarker level and clinical end-point. However, given the statistically significant differences in absolute values, direct comparison of concentration values between platforms is problematic. These observations have implications for future point-of-care novel biomarker research as direct comparisons of data will likely only be possible if assays are performed using the same methodology with reagents obtained from the same manufacturer.

The strengths of this study include inclusion of biomarkers of current clinical interest in the field of critical illness and life-threatening infections. The large sample size of more than 400 patients provides this study with higher analytical power. Additionally, the selection of the Luminex® microsphere-based system, an established multiplex platform [27], as well as the novel Ella^TM microfluidics-based system [30] allowed the validation a novel platform with the potential to be used in future studies requiring real-time results. One of the clearly demonstrated strengths of the Ella^TM platform was a superior dynamic range compared to Luminex using reagents from R&D Systems.

This study had some limitations. First, frozen versus fresh plasma was used. Future studies are planned to validate biomarker concentrations quantified in fresh blood samples. Second, we could not analyze data obtained from the Ella^TM platform for sICAM-1 due to inappropriate sample dilution. This limitation highlights the limitation of multiplex assays which require all analytes tested to be at dilution suitable for the assay dynamic range. Also, up to 34% of concentration values quantified by the Luminex platform were below the limit of quantification (CHI3L1, sFlt-1, and IL-6). This limited the ability to compare the complete sample set, likely diminishing the accuracy of the comparison between the two platforms. This also limits the generalizability of the results to more critically ill patients, with higher biomarker levels.

Future clinical studies evaluating the potential of plasma markers as predictive and prognostic tools will require reliable, robust, easy to use multiplex platforms, providing data in real time at the point-of-care. Ultimately this approach could enable precision medicine via rapid triage of febrile patients and stratification for pathway-directed therapeutics for severe infections that may be applicable in critically ill patients identified in an emergency department or an intensive care unit.

In conclusion, plasma biomarkers of endothelial and immune activation can be reliably quantified using multiplex immunoassay-based platforms. The dynamic range of biomarker concentrations is greater for the Ella^TM platform. The intra-assay variability is excellent for both multiplex platforms. The trends in concentrations obtained by the Luminex and Ella instruments are highly correlated but the absolute value of analyte concentrations vary between these platforms. Novel multiplex assays that produce rapid and reliable results, with minimal user interface and enable near-patient application, could transform prediction of response to therapies, prognostication of clinical outcome, and risk-stratification for enrollment in clinical trials.

Supporting information

S1 Table. Dynamic range for biomarkers included in the Luminex® platform that did not overlap with Ella^TM multiplex platform.

https://doi.org/10.1371/journal.pone.0175130.s001

(DOCX)

S2 Table. Inter-assay variability for a single healthy control quantified across 6 Luminex® assay plates and 6 Ella^TM cartridges.

https://doi.org/10.1371/journal.pone.0175130.s002

(DOCX)

S3 Table. Comparison of reagents included in the Luminex® and Ella^TM platform.

https://doi.org/10.1371/journal.pone.0175130.s003

(DOCX)

Acknowledgments

We would like to thank the children and their care givers from the Jinja Regional Referral Hospital for participation in the prospective observational study to investigate the relationship between biomarkers of endothelial and immune activation and mortality. We also thank the biostatistics faculty members of the Training in Clinical Research program at the University of California, San Francisco, for their guidance in statistical analysis selection. This work was supported by the Canadian Institutes of Health Research (CIHR) grants MOP-13721, MOP-115160, MOP-136813, a CIHR Foundation grant FDN-148439 (K.C.K.), the Canada Research Chair Program (K.C.K), a CIHR Banting fellowship (A.L.), and donations from Kim Kertland and the Tesari Foundation. M.A.M. has received grants from Amgen for experimental studies on acute lung injury and from GlaxoSmithKline for studies of sepsis in patients; consultancy agreements for ARDS or lung diseases with Boehringer-Ingelheim, Bayer Inc, Cerus Therapeutics, Biomarck Pharmaceuticals, Quark Pharmaceuticals, Thesan Pharmaceuticals, Incardia Inc, Biogen, and GlaxoSmithKline; DSMB Chair for clinical trials for Roche-Genentec. This does not alter our adherence to PLOS ONE policies on sharing data and materials.

Author Contributions

Conceptualization: AL ALC MH KCK.
Data curation: AL ALC MH GL KZ MAM KCK.
Formal analysis: AL ALC MH GL MAM KCK.
Funding acquisition: KCK.
Investigation: AL ALC MH KZ.
Methodology: AL ALC MH GL KCK.
Project administration: AL ALC MH KCK.
Resources: AL ALC MH KZ KCK.
Software: AL GL.
Supervision: AL ALC MH KCK.
Validation: AL ALC MH GL MAM KCK.
Visualization: AL ALC MH MAM KCK.
Writing – original draft: AL ALC MH GL MAM KCK.
Writing – review & editing: AL ALC MH GL MAM KCK.

References

1. Pene F, Courtine E, Cariou A, Mira JP. Toward theragnostics. Crit Care Med. 2009;37(1 Suppl):S50–8.
- View Article
- Google Scholar
2. Kojic D, Siegler BH, Uhle F, Lichtenstern C, Nawroth PP, Weigand MA, et al. Are there new approaches for diagnosis, therapy guidance and outcome prediction of sepsis? World J Exp Med. 2015;5(2):50–63. PubMed Central PMCID: PMCPMC4436940. pmid:25992320
- View Article
- PubMed/NCBI
- Google Scholar
3. Reinhart K, Bauer M, Riedemann NC, Hartog CS. New approaches to sepsis: molecular diagnostics and biomarkers. Clin Microbiol Rev. 2012;25(4):609–34. PubMed Central PMCID: PMCPMC3485751. pmid:23034322
- View Article
- PubMed/NCBI
- Google Scholar
4. Cohen J, Vincent JL, Adhikari NK, Machado FR, Angus DC, Calandra T, et al. Sepsis: a roadmap for future research. The Lancet infectious diseases. 2015;15(5):581–614. pmid:25932591
- View Article
- PubMed/NCBI
- Google Scholar
5. Matthay MA, Liu KD. New Strategies for Effective Therapeutics in Critically Ill Patients. JAMA. 2016;315(8):747–8. pmid:26903328
- View Article
- PubMed/NCBI
- Google Scholar
6. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801–10. pmid:26903338
- View Article
- PubMed/NCBI
- Google Scholar
7. Xing K, Murthy S, Liles WC, Singh JM. Clinical utility of biomarkers of endothelial activation in sepsis—a systematic review. Crit Care. 2012;16(1):R7. PubMed Central PMCID: PMCPMC3396237. pmid:22248019
- View Article
- PubMed/NCBI
- Google Scholar
8. Lee WL, Slutsky AS. Sepsis and endothelial permeability. N Engl J Med. 2010;363(7):689–91. Epub 2010/09/08. pmid:20818861
- View Article
- PubMed/NCBI
- Google Scholar
9. Hotchkiss RS, Monneret G, Payen D. Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol. 2013;13(12):862–74. PubMed Central PMCID: PMCPMC4077177. pmid:24232462
- View Article
- PubMed/NCBI
- Google Scholar
10. Delano MJ, Ward PA. Sepsis-induced immune dysfunction: can immune therapies reduce mortality? J Clin Invest. 2016;126(1):23–31. PubMed Central PMCID: PMCPMC4701539. pmid:26727230
- View Article
- PubMed/NCBI
- Google Scholar
11. Ricciuto DR, dos Santos CC, Hawkes M, Toltl LJ, Conroy AL, Rajwans N, et al. Angiopoietin-1 and angiopoietin-2 as clinically informative prognostic biomarkers of morbidity and mortality in severe sepsis. Crit Care Med. 2011;39(4):702–10. pmid:21242795
- View Article
- PubMed/NCBI
- Google Scholar
12. Shapiro NI, Yano K, Okada H, Fischer C, Howell M, Spokes KC, et al. A prospective, observational study of soluble FLT-1 and vascular endothelial growth factor in sepsis. Shock. 2008;29(4):452–7. pmid:18598002
- View Article
- PubMed/NCBI
- Google Scholar
13. Skibsted S, Jones AE, Puskarich MA, Arnold R, Sherwin R, Trzeciak S, et al. Biomarkers of endothelial cell activation in early sepsis. Shock. 2013;39(5):427–32. PubMed Central PMCID: PMCPMC3670087. pmid:23524845
- View Article
- PubMed/NCBI
- Google Scholar
14. de Pablo R, Monserrat J, Reyes E, Diaz D, Rodriguez-Zapata M, de la Hera A, et al. Circulating sICAM-1 and sE-Selectin as biomarker of infection and prognosis in patients with systemic inflammatory response syndrome. Eur J Intern Med. 2013;24(2):132–8. pmid:23352000
- View Article
- PubMed/NCBI
- Google Scholar
15. Mikacenic C, Hahn WO, Price BL, Harju-Baker S, Katz R, Kain KC, et al. Biomarkers of Endothelial Activation Are Associated with Poor Outcome in Critical Illness. PloS one. 2015;10(10):e0141251. PubMed Central PMCID: PMCPMC4619633. pmid:26492036
- View Article
- PubMed/NCBI
- Google Scholar
16. Hack CE, De Groot ER, Felt-Bersma RJ, Nuijens JH, Strack Van Schijndel RJ, Eerenberg-Belmer AJ, et al. Increased plasma levels of interleukin-6 in sepsis. Blood. 1989;74(5):1704–10. pmid:2790194
- View Article
- PubMed/NCBI
- Google Scholar
17. Ng PC, Li K, Chui KM, Leung TF, Wong RP, Chu WC, et al. IP-10 is an early diagnostic marker for identification of late-onset bacterial infection in preterm infants. Pediatr Res. 2007;61(1):93–8. pmid:17211148
- View Article
- PubMed/NCBI
- Google Scholar
18. Kornblit B, Hellemann D, Munthe-Fog L, Bonde J, Strom JJ, Madsen HO, et al. Plasma YKL-40 and CHI3L1 in systemic inflammation and sepsis-experience from two prospective cohorts. Immunobiology. 2013;218(10):1227–34. pmid:23706599
- View Article
- PubMed/NCBI
- Google Scholar
19. de Pablo R, Monserrat J, Reyes E, Diaz-Martin D, Rodriguez Zapata M, Carballo F, et al. Mortality in patients with septic shock correlates with anti-inflammatory but not proinflammatory immunomodulatory molecules. Journal of intensive care medicine. 2011;26(2):125–32. pmid:21464065
- View Article
- PubMed/NCBI
- Google Scholar
20. Calfee CS, Delucchi K, Parsons PE, Thompson BT, Ware LB, Matthay MA, et al. Subphenotypes in acute respiratory distress syndrome: latent class analysis of data from two randomised controlled trials. Lancet Respir Med. 2014;2(8):611–20. PubMed Central PMCID: PMCPMC4154544. pmid:24853585
- View Article
- PubMed/NCBI
- Google Scholar
21. Matthay MA, Ware LB, Zimmerman GA. The acute respiratory distress syndrome. J Clin Invest. 2012;122(8):2731–40. PubMed Central PMCID: PMCPMC3408735. pmid:22850883
- View Article
- PubMed/NCBI
- Google Scholar
22. Engvall E, Perlmann P. Enzyme-linked immunosorbent assay (ELISA). Quantitative assay of immunoglobulin G. Immunochemistry. 1971;8(9):871–4. pmid:5135623
- View Article
- PubMed/NCBI
- Google Scholar
23. Nolan JP, Mandy F. Multiplexed and microparticle-based analyses: quantitative tools for the large-scale analysis of biological systems. Cytometry A. 2006;69(5):318–25. PubMed Central PMCID: PMCPMC2200865. pmid:16604537
- View Article
- PubMed/NCBI
- Google Scholar
24. Elshal MF, McCoy JP. Multiplex bead array assays: performance evaluation and comparison of sensitivity to ELISA. Methods. 2006;38(4):317–23. PubMed Central PMCID: PMCPMC1534009. pmid:16481199
- View Article
- PubMed/NCBI
- Google Scholar
25. Tighe PJ, Ryder RR, Todd I, Fairclough LC. ELISA in the multiplex era: potentials and pitfalls. Proteomics Clin Appl. 2015;9(3–4):406–22. pmid:25644123
- View Article
- PubMed/NCBI
- Google Scholar
26. Khan SS, Smith MS, Reda D, Suffredini AF, McCoy JP Jr. Multiplex bead array assays for detection of soluble cytokines: comparisons of sensitivity and quantitative values among kits from multiple manufacturers. Cytometry B Clin Cytom. 2004;61(1):35–9. pmid:15351980
- View Article
- PubMed/NCBI
- Google Scholar
27. Fulton RJ, McDade RL, Smith PL, Kienker LJ, Kettman JR, Jr. Advanced multiplexed analysis with the FlowMetrix system. Clin Chem. 1997;43(9):1749–56. pmid:9299971
- View Article
- PubMed/NCBI
- Google Scholar
28. Baker HN, Murphy R, Lopez E, Garcia C. Conversion of a capture ELISA to a Luminex xMAP assay using a multiplex antibody screening method. J Vis Exp. 2012;(65). PubMed Central PMCID: PMCPMC3471270.
- View Article
- Google Scholar
29. Carrick DM, Chaturvedi AK, Shiels MS, Divi RL, Filipski KK, Hebert EF, et al. Using Immune Marker Panels to Evaluate the Role of Inflammation in Cancer: Summary of an NCI-sponsored Workshop. Cancer Epidemiol Biomarkers Prev. 2015. PubMed Central PMCID: PMCPMC4690808.
- View Article
- Google Scholar
30. ProteinSimple. Ella [21.September.2016]. Available from: http://www.proteinsimple.com/ella.html.
31. Aldo P, Marusov G, Svancara D, David J, Mor G. Simple Plex^TM: A Novel Multi-Analyte, Automated Microfluidic Immunoassay Platform for the Detection of Human and Mouse Cytokines and Chemokines. Am J Reprod Immunol. 2016;75(6):678–93. PubMed Central PMCID: PMCPMC5084752. pmid:27170460
- View Article
- PubMed/NCBI
- Google Scholar
32. Cao J, Seegmiller J, Hanson NQ, Zaun C, Li D. A microfluidic multiplex proteomic immunoassay device for translational research. Clin Proteomics. 2015;12:28. PubMed Central PMCID: PMCPMC4676148. pmid:26692826
- View Article
- PubMed/NCBI
- Google Scholar
33. Conroy AL, Hawkes M, Hayford K, Namasopo S, Opoka RO, John CC, et al. Prospective validation of pediatric disease severity scores to predict mortality in Ugandan children presenting with malaria and non-malaria febrile illness. Crit Care. 2015;19:47. PubMed Central PMCID: PMCPMC4339236. pmid:25879892
- View Article
- PubMed/NCBI
- Google Scholar
34. R&DSystems. Magnetic Luminex® Assay Human Premixed Multi-Analyte Kit [3.October.2016]. Available from: https://resources.rndsystems.com/pdfs/datasheets/lxsahm.pdf.
35. Reed GF, Lynn F, Meade BD. Use of coefficient of variation in assessing variability of quantitative assays. Clin Diagn Lab Immunol. 2002;9(6):1235–9. PubMed Central PMCID: PMCPMC130103. pmid:12414755
- View Article
- PubMed/NCBI
- Google Scholar
36. Zaki R, Bulgiba A, Ismail R, Ismail NA. Statistical methods used to test for agreement of medical instruments measuring continuous variables in method comparison studies: a systematic review. PloS one. 2012;7(5):e37908. PubMed Central PMCID: PMCPMC3360667. pmid:22662248
- View Article
- PubMed/NCBI
- Google Scholar
37. Luiz RR, Szklo M. More than one statistical strategy to assess agreement of quantitative measurements may usefully be reported. J Clin Epidemiol. 2005;58(3):215–6. pmid:15718108
- View Article
- PubMed/NCBI
- Google Scholar
38. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10. pmid:2868172
- View Article
- PubMed/NCBI
- Google Scholar
39. Pitman EJG. A note on normal correlation. Biometrika. 1939;31(1–2):9–12.
- View Article
- Google Scholar
40. Ellington AA, Kullo IJ, Bailey KR, Klee GG. Antibody-based protein multiplex platforms: technical and operational challenges. Clin Chem. 2010;56(2):186–93. PubMed Central PMCID: PMCPMC2901849. pmid:19959625
- View Article
- PubMed/NCBI
- Google Scholar
41. FDA. Guidance for Industry Bioanalytical Method Validation 2013. Available from: http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm368107.pdf.
42. Higgins SJ, Purcell LA, Silver KL, Tran V, Crowley V, Hawkes M, et al. Dysregulation of angiopoietin-1 plays a mechanistic role in the pathogenesis of cerebral malaria. Science translational medicine. 2016;8(358):358ra128. pmid:27683553
- View Article
- PubMed/NCBI
- Google Scholar
43. van der Heijden M, van Nieuw Amerongen GP, Chedamni S, van Hinsbergh VW, Johan Groeneveld AB. The angiopoietin-Tie2 system as a therapeutic target in sepsis and acute lung injury. Expert Opin Ther Targets. 2009;13(1):39–53. pmid:19063705
- View Article
- PubMed/NCBI
- Google Scholar
44. Tsao PN, Chan FT, Wei SC, Hsieh WS, Chou HC, Su YN, et al. Soluble vascular endothelial growth factor receptor-1 protects mice in sepsis. Crit Care Med. 2007;35(8):1955–60. pmid:17568329
- View Article
- PubMed/NCBI
- Google Scholar

[ref1] 1. Pene F, Courtine E, Cariou A, Mira JP. Toward theragnostics. Crit Care Med. 2009;37(1 Suppl):S50–8.
View Article
Google Scholar

[2] View Article

[3] Google Scholar

[ref2] 2. Kojic D, Siegler BH, Uhle F, Lichtenstern C, Nawroth PP, Weigand MA, et al. Are there new approaches for diagnosis, therapy guidance and outcome prediction of sepsis? World J Exp Med. 2015;5(2):50–63. PubMed Central PMCID: PMCPMC4436940. pmid:25992320
View Article
PubMed/NCBI
Google Scholar

[5] View Article

[6] PubMed/NCBI

[7] Google Scholar

[ref3] 3. Reinhart K, Bauer M, Riedemann NC, Hartog CS. New approaches to sepsis: molecular diagnostics and biomarkers. Clin Microbiol Rev. 2012;25(4):609–34. PubMed Central PMCID: PMCPMC3485751. pmid:23034322
View Article
PubMed/NCBI
Google Scholar

[9] View Article

[10] PubMed/NCBI

[11] Google Scholar

[ref4] 4. Cohen J, Vincent JL, Adhikari NK, Machado FR, Angus DC, Calandra T, et al. Sepsis: a roadmap for future research. The Lancet infectious diseases. 2015;15(5):581–614. pmid:25932591
View Article
PubMed/NCBI
Google Scholar

[13] View Article

[14] PubMed/NCBI

[15] Google Scholar

[ref5] 5. Matthay MA, Liu KD. New Strategies for Effective Therapeutics in Critically Ill Patients. JAMA. 2016;315(8):747–8. pmid:26903328
View Article
PubMed/NCBI
Google Scholar

[17] View Article

[18] PubMed/NCBI

[19] Google Scholar

[ref6] 6. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801–10. pmid:26903338
View Article
PubMed/NCBI
Google Scholar

[21] View Article

[22] PubMed/NCBI

[23] Google Scholar

[ref7] 7. Xing K, Murthy S, Liles WC, Singh JM. Clinical utility of biomarkers of endothelial activation in sepsis—a systematic review. Crit Care. 2012;16(1):R7. PubMed Central PMCID: PMCPMC3396237. pmid:22248019
View Article
PubMed/NCBI
Google Scholar

[25] View Article

[26] PubMed/NCBI

[27] Google Scholar

[ref8] 8. Lee WL, Slutsky AS. Sepsis and endothelial permeability. N Engl J Med. 2010;363(7):689–91. Epub 2010/09/08. pmid:20818861
View Article
PubMed/NCBI
Google Scholar

[29] View Article

[30] PubMed/NCBI

[31] Google Scholar

[ref9] 9. Hotchkiss RS, Monneret G, Payen D. Sepsis-induced immunosuppression: from cellular dysfunctions to immunotherapy. Nat Rev Immunol. 2013;13(12):862–74. PubMed Central PMCID: PMCPMC4077177. pmid:24232462
View Article
PubMed/NCBI
Google Scholar

[33] View Article

[34] PubMed/NCBI

[35] Google Scholar

[ref10] 10. Delano MJ, Ward PA. Sepsis-induced immune dysfunction: can immune therapies reduce mortality? J Clin Invest. 2016;126(1):23–31. PubMed Central PMCID: PMCPMC4701539. pmid:26727230
View Article
PubMed/NCBI
Google Scholar

[37] View Article

[38] PubMed/NCBI

[39] Google Scholar

[ref11] 11. Ricciuto DR, dos Santos CC, Hawkes M, Toltl LJ, Conroy AL, Rajwans N, et al. Angiopoietin-1 and angiopoietin-2 as clinically informative prognostic biomarkers of morbidity and mortality in severe sepsis. Crit Care Med. 2011;39(4):702–10. pmid:21242795
View Article
PubMed/NCBI
Google Scholar

[41] View Article

[42] PubMed/NCBI

[43] Google Scholar

[ref12] 12. Shapiro NI, Yano K, Okada H, Fischer C, Howell M, Spokes KC, et al. A prospective, observational study of soluble FLT-1 and vascular endothelial growth factor in sepsis. Shock. 2008;29(4):452–7. pmid:18598002
View Article
PubMed/NCBI
Google Scholar

[45] View Article

[46] PubMed/NCBI

[47] Google Scholar

[ref13] 13. Skibsted S, Jones AE, Puskarich MA, Arnold R, Sherwin R, Trzeciak S, et al. Biomarkers of endothelial cell activation in early sepsis. Shock. 2013;39(5):427–32. PubMed Central PMCID: PMCPMC3670087. pmid:23524845
View Article
PubMed/NCBI
Google Scholar

[49] View Article

[50] PubMed/NCBI

[51] Google Scholar

[ref14] 14. de Pablo R, Monserrat J, Reyes E, Diaz D, Rodriguez-Zapata M, de la Hera A, et al. Circulating sICAM-1 and sE-Selectin as biomarker of infection and prognosis in patients with systemic inflammatory response syndrome. Eur J Intern Med. 2013;24(2):132–8. pmid:23352000
View Article
PubMed/NCBI
Google Scholar

[53] View Article

[54] PubMed/NCBI

[55] Google Scholar

[ref15] 15. Mikacenic C, Hahn WO, Price BL, Harju-Baker S, Katz R, Kain KC, et al. Biomarkers of Endothelial Activation Are Associated with Poor Outcome in Critical Illness. PloS one. 2015;10(10):e0141251. PubMed Central PMCID: PMCPMC4619633. pmid:26492036
View Article
PubMed/NCBI
Google Scholar

[57] View Article

[58] PubMed/NCBI

[59] Google Scholar

[ref16] 16. Hack CE, De Groot ER, Felt-Bersma RJ, Nuijens JH, Strack Van Schijndel RJ, Eerenberg-Belmer AJ, et al. Increased plasma levels of interleukin-6 in sepsis. Blood. 1989;74(5):1704–10. pmid:2790194
View Article
PubMed/NCBI
Google Scholar

[61] View Article

[62] PubMed/NCBI

[63] Google Scholar

[ref17] 17. Ng PC, Li K, Chui KM, Leung TF, Wong RP, Chu WC, et al. IP-10 is an early diagnostic marker for identification of late-onset bacterial infection in preterm infants. Pediatr Res. 2007;61(1):93–8. pmid:17211148
View Article
PubMed/NCBI
Google Scholar

[65] View Article

[66] PubMed/NCBI

[67] Google Scholar

[ref18] 18. Kornblit B, Hellemann D, Munthe-Fog L, Bonde J, Strom JJ, Madsen HO, et al. Plasma YKL-40 and CHI3L1 in systemic inflammation and sepsis-experience from two prospective cohorts. Immunobiology. 2013;218(10):1227–34. pmid:23706599
View Article
PubMed/NCBI
Google Scholar

[69] View Article

[70] PubMed/NCBI

[71] Google Scholar

[ref19] 19. de Pablo R, Monserrat J, Reyes E, Diaz-Martin D, Rodriguez Zapata M, Carballo F, et al. Mortality in patients with septic shock correlates with anti-inflammatory but not proinflammatory immunomodulatory molecules. Journal of intensive care medicine. 2011;26(2):125–32. pmid:21464065
View Article
PubMed/NCBI
Google Scholar

[73] View Article

[74] PubMed/NCBI

[75] Google Scholar

[ref20] 20. Calfee CS, Delucchi K, Parsons PE, Thompson BT, Ware LB, Matthay MA, et al. Subphenotypes in acute respiratory distress syndrome: latent class analysis of data from two randomised controlled trials. Lancet Respir Med. 2014;2(8):611–20. PubMed Central PMCID: PMCPMC4154544. pmid:24853585
View Article
PubMed/NCBI
Google Scholar

[77] View Article

[78] PubMed/NCBI

[79] Google Scholar

[ref21] 21. Matthay MA, Ware LB, Zimmerman GA. The acute respiratory distress syndrome. J Clin Invest. 2012;122(8):2731–40. PubMed Central PMCID: PMCPMC3408735. pmid:22850883
View Article
PubMed/NCBI
Google Scholar

[81] View Article

[82] PubMed/NCBI

[83] Google Scholar

[ref22] 22. Engvall E, Perlmann P. Enzyme-linked immunosorbent assay (ELISA). Quantitative assay of immunoglobulin G. Immunochemistry. 1971;8(9):871–4. pmid:5135623
View Article
PubMed/NCBI
Google Scholar

[85] View Article

[86] PubMed/NCBI

[87] Google Scholar

[ref23] 23. Nolan JP, Mandy F. Multiplexed and microparticle-based analyses: quantitative tools for the large-scale analysis of biological systems. Cytometry A. 2006;69(5):318–25. PubMed Central PMCID: PMCPMC2200865. pmid:16604537
View Article
PubMed/NCBI
Google Scholar

[89] View Article

[90] PubMed/NCBI

[91] Google Scholar

[ref24] 24. Elshal MF, McCoy JP. Multiplex bead array assays: performance evaluation and comparison of sensitivity to ELISA. Methods. 2006;38(4):317–23. PubMed Central PMCID: PMCPMC1534009. pmid:16481199
View Article
PubMed/NCBI
Google Scholar

[93] View Article

[94] PubMed/NCBI

[95] Google Scholar

[ref25] 25. Tighe PJ, Ryder RR, Todd I, Fairclough LC. ELISA in the multiplex era: potentials and pitfalls. Proteomics Clin Appl. 2015;9(3–4):406–22. pmid:25644123
View Article
PubMed/NCBI
Google Scholar

[97] View Article

[98] PubMed/NCBI

[99] Google Scholar

[ref26] 26. Khan SS, Smith MS, Reda D, Suffredini AF, McCoy JP Jr. Multiplex bead array assays for detection of soluble cytokines: comparisons of sensitivity and quantitative values among kits from multiple manufacturers. Cytometry B Clin Cytom. 2004;61(1):35–9. pmid:15351980
View Article
PubMed/NCBI
Google Scholar

[101] View Article

[102] PubMed/NCBI

[103] Google Scholar

[ref27] 27. Fulton RJ, McDade RL, Smith PL, Kienker LJ, Kettman JR, Jr. Advanced multiplexed analysis with the FlowMetrix system. Clin Chem. 1997;43(9):1749–56. pmid:9299971
View Article
PubMed/NCBI
Google Scholar

[105] View Article

[106] PubMed/NCBI

[107] Google Scholar

[ref28] 28. Baker HN, Murphy R, Lopez E, Garcia C. Conversion of a capture ELISA to a Luminex xMAP assay using a multiplex antibody screening method. J Vis Exp. 2012;(65). PubMed Central PMCID: PMCPMC3471270.
View Article
Google Scholar

[109] View Article

[110] Google Scholar

[ref29] 29. Carrick DM, Chaturvedi AK, Shiels MS, Divi RL, Filipski KK, Hebert EF, et al. Using Immune Marker Panels to Evaluate the Role of Inflammation in Cancer: Summary of an NCI-sponsored Workshop. Cancer Epidemiol Biomarkers Prev. 2015. PubMed Central PMCID: PMCPMC4690808.
View Article
Google Scholar

[112] View Article

[113] Google Scholar

[ref30] 30. ProteinSimple. Ella [21.September.2016]. Available from: http://www.proteinsimple.com/ella.html.

[ref31] 31. Aldo P, Marusov G, Svancara D, David J, Mor G. Simple Plex^TM: A Novel Multi-Analyte, Automated Microfluidic Immunoassay Platform for the Detection of Human and Mouse Cytokines and Chemokines. Am J Reprod Immunol. 2016;75(6):678–93. PubMed Central PMCID: PMCPMC5084752. pmid:27170460
View Article
PubMed/NCBI
Google Scholar

[116] View Article

[117] PubMed/NCBI

[118] Google Scholar

[ref32] 32. Cao J, Seegmiller J, Hanson NQ, Zaun C, Li D. A microfluidic multiplex proteomic immunoassay device for translational research. Clin Proteomics. 2015;12:28. PubMed Central PMCID: PMCPMC4676148. pmid:26692826
View Article
PubMed/NCBI
Google Scholar

[120] View Article

[121] PubMed/NCBI

[122] Google Scholar

[ref33] 33. Conroy AL, Hawkes M, Hayford K, Namasopo S, Opoka RO, John CC, et al. Prospective validation of pediatric disease severity scores to predict mortality in Ugandan children presenting with malaria and non-malaria febrile illness. Crit Care. 2015;19:47. PubMed Central PMCID: PMCPMC4339236. pmid:25879892
View Article
PubMed/NCBI
Google Scholar

[124] View Article

[125] PubMed/NCBI

[126] Google Scholar

[ref34] 34. R&DSystems. Magnetic Luminex® Assay Human Premixed Multi-Analyte Kit [3.October.2016]. Available from: https://resources.rndsystems.com/pdfs/datasheets/lxsahm.pdf.

[ref35] 35. Reed GF, Lynn F, Meade BD. Use of coefficient of variation in assessing variability of quantitative assays. Clin Diagn Lab Immunol. 2002;9(6):1235–9. PubMed Central PMCID: PMCPMC130103. pmid:12414755
View Article
PubMed/NCBI
Google Scholar

[129] View Article

[130] PubMed/NCBI

[131] Google Scholar

[ref36] 36. Zaki R, Bulgiba A, Ismail R, Ismail NA. Statistical methods used to test for agreement of medical instruments measuring continuous variables in method comparison studies: a systematic review. PloS one. 2012;7(5):e37908. PubMed Central PMCID: PMCPMC3360667. pmid:22662248
View Article
PubMed/NCBI
Google Scholar

[133] View Article

[134] PubMed/NCBI

[135] Google Scholar

[ref37] 37. Luiz RR, Szklo M. More than one statistical strategy to assess agreement of quantitative measurements may usefully be reported. J Clin Epidemiol. 2005;58(3):215–6. pmid:15718108
View Article
PubMed/NCBI
Google Scholar

[137] View Article

[138] PubMed/NCBI

[139] Google Scholar

[ref38] 38. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1(8476):307–10. pmid:2868172
View Article
PubMed/NCBI
Google Scholar

[141] View Article

[142] PubMed/NCBI

[143] Google Scholar

[ref39] 39. Pitman EJG. A note on normal correlation. Biometrika. 1939;31(1–2):9–12.
View Article
Google Scholar

[145] View Article

[146] Google Scholar

[ref40] 40. Ellington AA, Kullo IJ, Bailey KR, Klee GG. Antibody-based protein multiplex platforms: technical and operational challenges. Clin Chem. 2010;56(2):186–93. PubMed Central PMCID: PMCPMC2901849. pmid:19959625
View Article
PubMed/NCBI
Google Scholar

[148] View Article

[149] PubMed/NCBI

[150] Google Scholar

[ref41] 41. FDA. Guidance for Industry Bioanalytical Method Validation 2013. Available from: http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm368107.pdf.

[ref42] 42. Higgins SJ, Purcell LA, Silver KL, Tran V, Crowley V, Hawkes M, et al. Dysregulation of angiopoietin-1 plays a mechanistic role in the pathogenesis of cerebral malaria. Science translational medicine. 2016;8(358):358ra128. pmid:27683553
View Article
PubMed/NCBI
Google Scholar

[153] View Article

[154] PubMed/NCBI

[155] Google Scholar

[ref43] 43. van der Heijden M, van Nieuw Amerongen GP, Chedamni S, van Hinsbergh VW, Johan Groeneveld AB. The angiopoietin-Tie2 system as a therapeutic target in sepsis and acute lung injury. Expert Opin Ther Targets. 2009;13(1):39–53. pmid:19063705
View Article
PubMed/NCBI
Google Scholar

[157] View Article

[158] PubMed/NCBI

[159] Google Scholar

[ref44] 44. Tsao PN, Chan FT, Wei SC, Hsieh WS, Chou HC, Su YN, et al. Soluble vascular endothelial growth factor receptor-1 protects mice in sepsis. Crit Care Med. 2007;35(8):1955–60. pmid:17568329
View Article
PubMed/NCBI
Google Scholar

[161] View Article

[162] PubMed/NCBI

[163] Google Scholar

Figures

Abstract

Introduction

Materials and methods

Patient sample selection

Plasma sample preparation

Luminex® platform protocol

EllaTM platform protocol

Statistical analysis

Results

Biomarker assay performance

Biomarker assay limits of quantification and dynamic range

Intra-assay variability

Comparisons of concentration correlations

Agreement between platforms

Discussion

Supporting information

S1 Table. Dynamic range for biomarkers included in the Luminex® platform that did not overlap with EllaTM multiplex platform.

S2 Table. Inter-assay variability for a single healthy control quantified across 6 Luminex® assay plates and 6 EllaTM cartridges.

S3 Table. Comparison of reagents included in the Luminex® and EllaTM platform.

Acknowledgments

Author Contributions

References

Ella^TM platform protocol

S1 Table. Dynamic range for biomarkers included in the Luminex® platform that did not overlap with Ella^TM multiplex platform.

S2 Table. Inter-assay variability for a single healthy control quantified across 6 Luminex® assay plates and 6 Ella^TM cartridges.

S3 Table. Comparison of reagents included in the Luminex® and Ella^TM platform.