Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Open Access

Peer-reviewed

Research Article

Assay Reproducibility in Clinical Studies of Plasma miRNA

  • Jonathan Rice,

    Affiliation Price Institute of Surgical Research, Hiram C. Polk Jr., M.D. Department of Surgery, University of Louisville School of Medicine, Louisville, KY, United States of America

    ⨯
  • Henry Roberts,

    Affiliation Price Institute of Surgical Research, Hiram C. Polk Jr., M.D. Department of Surgery, University of Louisville School of Medicine, Louisville, KY, United States of America

    ⨯
  • James Burton,

    Affiliation Price Institute of Surgical Research, Hiram C. Polk Jr., M.D. Department of Surgery, University of Louisville School of Medicine, Louisville, KY, United States of America

    ⨯
  • Jianmin Pan,

    Affiliation Department of Bioinformatics and Biostatistics, University of Louisville School of Public Health and Information Sciences, Louisville, KY, United States of America

    ⨯
  • Vanessa States,

    Affiliation Price Institute of Surgical Research, Hiram C. Polk Jr., M.D. Department of Surgery, University of Louisville School of Medicine, Louisville, KY, United States of America

    ⨯
  • Shesh N. Rai,

    Affiliation Department of Bioinformatics and Biostatistics, University of Louisville School of Public Health and Information Sciences, Louisville, KY, United States of America

    ⨯
  • Susan Galandiuk

    s0gala01@louisville.edu

    Affiliation Price Institute of Surgical Research, Hiram C. Polk Jr., M.D. Department of Surgery, University of Louisville School of Medicine, Louisville, KY, United States of America

    ⨯

Abstract

There are increasing reports of plasma miRNAs as biomarkers of human disease but few standards in methodologic reporting, leading to inconsistent data. We systematically reviewed plasma miRNA studies published between July 2013-June 2014 to assess methodology. Six parameters were investigated: time to plasma extraction, methods of RNA extraction, type of miRNA, quantification, cycle threshold (Ct) setting, and methods of statistical analysis. We compared these data with a proposed standard methodologic technique. Beginning with initial screening for 380 miRNAs using microfluidic array technology and validation in an additional cohort of patients, we compared 11 miRNAs that exhibited differential expression between 16 patients with benign colorectal neoplasms (advanced adenomas) and 16 patients without any neoplasm (controls). Plasma was isolated immediately, 12, 24, 48, or 72 h following phlebotomy. miRNA was extracted using two different techniques (Trizol LS with pre-amplification or modified miRNeasy). We performed Taqman-based RT-PCR assays for the 11 miRNAs with subsequent analyses using a variable Ct setting or a fixed Ct set at 0.01, 0.03, 0.05, or 0.5. Assays were performed in duplicate by two different operators. RNU6 was the internal reference. Systematic review yielded 74 manuscripts meeting inclusion criteria. One manuscript (1.4%) documented all 6 methodological parameters, while < 5% of studies listed Ct setting. In our proposed standard technique, plasma extraction ≤12 h provided consistent ΔCt. miRNeasy extraction yielded higher miRNA concentrations and fewer non-expressed miRNAs compared to Trizol LS (1/704 miRNAs [0.14%] vs 109/704 miRNAs [15%], not expressed, respectively). A fixed Ct bar setting of 0.03 yielded the most reproducible data, provided that <10% miRNA were non-expressed. There was no significant intra-operator variability. There was significant inter-operator variation using Trizol LS extraction, while this was negligible using modified miRNeasy. For standardized reporting, we recommend plasma extraction ≤ 12 h, using modified miRNeasy extraction and utilizing a 0.03 Ct.

Citation: Rice J, Roberts H, Burton J, Pan J, States V, Rai SN, et al. (2015) Assay Reproducibility in Clinical Studies of Plasma miRNA. PLoS ONE 10(4): e0121948. https://doi.org/10.1371/journal.pone.0121948

Academic Editor: Jörg D. Hoheisel, Deutsches Krebsforschungszentrum, GERMANY

Received: August 11, 2014; Accepted: February 17, 2015; Published: April 8, 2015

Copyright: © 2015 Rice et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: This publication is made possible in part by the John W. and Caroline Price Family Trust as well as gifts from Donald and Irene Dizney and the Alagia Family Foundation. Dr. Rai is partly supported by the Wendell Cherry Chair Clinical Trial Research Fund. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist. The authors also have no commercial interest or support from any manufacturer or industry to disclose.

Introduction

MicroRNAs are small 19–23 nucleotide noncoding ribonucleic acids (RNA) that bind to complementary sequences on the 3' untranslated region of target messenger RNAs (mRNA) [1]. Consequently, microRNAs (miRNA) post-transcriptionally regulate mRNA expression and are essential in numerous molecular regulatory pathways [2]. miRNA expression profiles have been shown to be unique to both the source material (i.e. plasma, tissue, etc.) and the disease process being investigated. miRNA profiles have, therefore, emerged as prospective biomarkers for cancer and many other human diseases [3–7].

This has led to a rapid proliferation of miRNA research. Unfortunately, many studies have been conducted without attention to standardization of methods or reproducibility of results, particularly with respect to studies of plasma miRNA. In many reports, it is difficult to deduce the actual methods used for analysis. This has led to the use of different extraction protocols, and various methods of quantification and statistical analysis, which, in turn, are a source of variability (Table 1). In part, due to this lack of standardization, many different miRNAs have been reported to be associated with a given disease process [5]. There is ongoing controversy over the optimal analytic methods for studies of miRNA in plasma[8].

thumbnail
Download:
Table 1. Multiple sources of variability in microRNA data and literature search [9–84].

https://doi.org/10.1371/journal.pone.0121948.t001

Since the discovery of miRNAs, their detection in blood has received much attention due to the ease of access and ready availability of peripheral blood as compared to tissue [5]. Initially, we performed a systematic review of publications focusing on plasma miRNA in order to ascertain what methods and reporting criteria were currently being utilized. We then we used a panel of 11 selected miRNA to study the effect of 5 of the variables shown in Table 1 on data obtained in plasma miRNA studies, namely the effect of:

  • Time to plasma extraction
  • Method of RNA extraction
  • Cycle threshold bar setting
  • Intra-operator variability
  • Inter-operator variability

Materials and Methods

Systematic Review

In order to determine the consistency and current status of methods reporting of clinical studies of plasma miRNA, we retrieved original manuscripts published from July 1, 2013, until June 30, 2014. We utilized a single search engine (PubMed) without language restriction using the following search words: plasma, microRNA, and human. We excluded review articles, case reports, or non-English language articles. Remaining articles were then obtained for review. These were then graded as to how many of the following criteria were clearly documented in the Materials and Methods sections: 1) time of plasma extraction, 2) method of RNA extraction, 3) type of miRNA used (total vs exosomal), 4) method of quantification (external vs internal reference), 5) cycle threshold bar setting, and 6) methods of statistical analysis (items denoted by * in Table 1).

microRNA Selection

An eleven miRNA panel with RNU6 as an internal reference was used in this study. These miRNAs were selected based upon unpublished data from our laboratory utilizing 380 miRNA arrays (Applied Biosystems, Carlsbad, California) to determine miRNA expression in the plasma of 20 colorectal cancer (CRC) patients, and 10 patients each with colorectal advanced adenoma (CAA)(adenomatous polyps> 0.6 cm in maximal diameter), breast cancer (BC), lung cancer (LC), pancreatic cancer (PC), and 10 controls.

In order to determine sample size, the most important aspect is adjustment of the significance level (alpha). For screening studies, we use Jung’s procedure to adjust the alpha. Using the method of Jung to find about 5% of features to be significant at a false detection rate (FDR) of 5%, the adjusted alpha will be 0.0038. With any two groups, a minimum of n1 = 10 and n2 = 10 using a two sample t test, we can detect at least 2.7 fold means (which we have observed in our preliminary data sets) using the common standard deviation at significance level of 0.0038 and power of 80%.

With respect to choice of number of miRNA in our panel, it was our expectation that no more than 10% of miRNA would be differentially expressed between cases and controls after adjusting the p values for multiple comparisons. Of these, in turn, one would not expect more that 0.5 to 3% of miRNA to be able to accurately identify cases and controls. Ten miRNA and one reference (housekeeping gene) miRNA were, therefore, chosen (approximately 3%). Statistical analysis using ANOVA identified 11 significantly dysregulated miRNAs specific for colorectal neoplasia (Table 2). Multiple test control was based on controlling the false discovery rate (FDR) at 10%. A logistic regression model was established using the top up-regulated miRNAs and used for predicting the adenoma and control groups for the validation data. The sensitivity and specificity for this prediction were calculated. The receiver operator characteristic (ROC) curves with AUC values were microRNA-rated using current versions of SAS [85] and R [86–88] (Fig 1a & 1b). These data are not the focus of this report; however, these selected miRNA were utilized for the present study to evaluate the effect of time to plasma extraction, the effect of multiple samples drawn from an individual over time, the method of RNA extraction, cycle threshold bar setting, and intra- as well as inter-operator variability.

thumbnail
Download:
Table 2. Results of ANOVA analysis for initial microRNA selection.

https://doi.org/10.1371/journal.pone.0121948.t002

thumbnail
Download:
Fig 1.

A) Receiver operator characteristic (ROC) curve for miR-523,miR-218,miR-142-3p,miR 27a,miR-21. Colorectal cancer (n = 20) vs. Breast cancer +Pancreatic cancer + Lung cancer (n = 30 [10 each group]). B) ROC Curve for miR-523, miR-218, miR-142-3p,miR-27a,miR-376c,miR-374. Colorectal cancer (n = 20) + Colorectal adenoma (n = 10) vs. Breast cancer +Pancreatic cancer + Lung cancer (n = 30 [10 each group]).

https://doi.org/10.1371/journal.pone.0121948.g001

Patient Population

The University of Louisville Institutional Review Board reviewed and approved this study. Written informed consent was obtained from all subjects who were treated at a single university-based colorectal surgery practice. The patient population consisted of 16 patients with colorectal advanced adenomas, and 16 patients without colorectal neoplasia (controls). The patient groups were age-, race-, and gender-matched. Prior to patient treatment, 6 mL of peripheral whole blood was obtained in EDTA tubes (Becton-Dickinson, Franklin Lakes, NJ) via venipuncture from the adenoma group and from individuals in the “control” group at the time of routine screening colonoscopy. The latter group of individuals (n = 16) had no colonic neoplasia or inflammatory bowel disease. Blood was stored at 4°C until plasma isolation. Plasma was isolated within 24 h of venipuncture, unless noted otherwise (see section below “Time to Plasma Extraction”). Patient demographics are displayed in Table 3 and did not differ between patient groups.

thumbnail
Download:
Table 3. Patient demographics.

https://doi.org/10.1371/journal.pone.0121948.t003

Time to Plasma Extraction

Five 6 mL aliquots of peripheral blood from 6 controls were obtained and stored at 4°C until extraction. Plasma was extracted at different time points: immediately (within 30 minutes after phlebotomy) and then at 12, 24, 48, and 72 h post-phlebotomy. Whole blood was centrifuged at 600 relative centrifugal force (rcf) for 15 minutes in order to isolate the plasma, which was then stored at -80°C for later use. Once plasma was isolated, it underwent downstream processing via the modified phenol/guanidine-based lysis and silica membrane-based extraction technique (miRNeasy, Qiagen, Venlo, Limburg).

Effect of Repeated Sample Acquisition

In order to determine whether there was a difference between samples when plasma miRNA were analyzed in samples drawn at different times from the same individual, 6 mL peripheral blood was drawn from each of 12 healthy subjects without any neoplasia or inflammatory condition at 6:30 AM and 12 hours later at 6:30 PM on the same day. Plasma was extracted within 30 minutes of phlebotomy, and stored at -80°C for later use. Downstream processing was performed using the modified phenol/guanidine-based lysis and silica membrane-based extraction technique (miRNeasy, Qiagen, Venlo, Limburg) and methodology described below.

Method of RNA Extraction

Total RNA was extracted from 250 μL plasma samples using either the Trizol LS reagent protocol (Ambion, Austin, Texas), which was modified by addition of an extended overnight drying period, or by a modified miRNeasy (Qiagen, Venlo, Limburg) extraction technique with yeast carrier [89, 90]. Total RNA purity was assessed using a Nanodrop 2000 spectrophotometer (Thermo Scientific, Middlesex, MA).

Pre-Amplification

When using the TRIzol LS total RNA extraction technique, pre-amplification was necessary, as the majority of samples yielded quantities of total RNA < 500 ng. In our experience, this is the minimum amount needed for adequate expression and amplification in qPCR. The Applied Biosystems protocol for producing custom reverse transcription and pre-amplification pools with TaqMan miRNA assays was followed [91].

Reverse Transcription

For reverse transcription, 10 ng of RNA were converted into cDNA utilizing the TaqMan miRNA reverse transcription kit with our custom 5X microRNA-specific stem-loop primer pool. The reverse transcription product was then added to the pre-amplification master mix and our custom 20X miRNA-specific stem-loop primer pool.

MicroRNA Quantification

A Step-One Plus RT-PCR System (Life Technologies, Carlsbad, California), with default fast thermal cycling conditions, was used for RT-PCR. Individual Taqman probes (20x) were used during RT-PCR to bind to a complementary sequence in the target cDNA. These values were, in turn, normalized to the expression of the endogenous miRNA RNU6 as internal reference to calculate ΔCt values.

Setting of Cycle Threshold

In order to compare cycle threshold (Ct) values across plates, both fixed and variable thresholds were utilized. Since different miRNAs on the same plate may have different linear phases, a fixed threshold may not intersect across the linear phase. For this reason, the variable threshold is the default setting. Cycle threshold is illustrated in Fig 2, which shows the phases of the PCR curves: (1) the baseline, (2) the exponential phase, (3) the linear phase, and 4) the plateau phase. The RQ manager from Applied Biosystems may use a different threshold (variable) within the same plate if one selects the option “automatic Ct”. Different thresholds will be chosen for different miRNAs according to the linear phases. However, these Ct values cannot be compared directly between different plates. The threshold needs to be considered in the analysis and adjustment of the Ct values is needed. In addition to this variable threshold setting, we examined the effect of fixed threshold settings of 0.01, 0.03, 0.05, or 0.5. Ct values, threshold, and 40 cycles fluorescence intensities are miRNA-rated from RQ manager 1.2, Applied Biosystems.

thumbnail
Download:
Fig 2. The phases and components of a PCR curves: (1) baseline, (2) exponential phase, (3) linear phase, (4) plateau phase, and (5) cycle threshold.

https://doi.org/10.1371/journal.pone.0121948.g002

Intra-operator Variability

Assays for individual miRNA for each patient sample were run in duplicate to permit assessment of intra-operator variability (comparison of duplicate samples).

Inter-operator Variability

In order to assess inter-operator variation following total RNA extraction, two experienced operators separately performed subsequent sample processing, including reverse transcription, preamplification (for samples extracted using TRIzol LS), and qPCR. Each operator used the same thermocycler (Mastercycler, Eppendorf, Hamburg, Germany) and Step-One Plus RT-PCR System, but at different times on different days. The cycle threshold (Ct) values were exported with the Ct bar set at 0.03.

Statistical Methods

A portion of the statistical analysis plan is already discussed in the methods under the section “miRNA selection.” Given the clinical information, in addition to gene expression data, we use a full statistical model-based approach, such as ANOVA or ANCOVA to analyze the data. Another alternative is the use of a simple approach for comparing means or medians using a two-sample t test based on a parametric approach or a Wilcoxon rank sum non-parametric test [92, 93].

For data with a variable threshold, we considered the threshold as a covariate in the ANOVA model or used it in the normalization. The ANOVA model was fitted as follows. where ▽Ct = Ct—CR for a fixed threshold or a variable threshold with a covariate log2(Tt/TR), and ▽Ct = Ct—CR—log2(Tt/TR) for variable threshold using threshold in the normalization. ▽Ct = Ct—CR for a fixed threshold or a variable threshold with a covariate log2 (Tt / TR), and ▽Ct = Ct—CR—log2 (Tt / TR) for variable threshold using threshold in the normalization. Tt is the threshold intensity of the miRNA, CR and TR are the Ct values and the threshold intensity of reference gene U6, respectively. The error term Ɛ in the model is due to multiple factors (subject variability, operator/machine variability). We considered six ANOVA models, with the operator as a covariate or not, with interaction, Group×Operator, or not, with Operator = 1 or 2 only, and with the average of Operator = 1 and 2. Differences were considered significant for p-values < 0.05. To study the effect of threshold intensity, we applied a variable threshold and a wide range (0.01, 0.03, 0.05, or 0.5) of threshold settings. If the fixed threshold setting is used, the penultimate correction is not required. These data results are discussed in detail elsewhere (Rai et al., unpublished data, 2014).

Another critical approach is to build a prediction model to identify groups of individuals (such as cases and controls). Unlike high-throughput data analysis with tens of thousands of biomarker (genes), which involves hierarchical modeling, principal component analysis, heat map, and other methods, with a lesser number of biomarkers (miRNAs), we apply an ANOVA/ANCOVA model for model fitting and a logistic model for prediction [94, 95]. The classification method based on a logistic model is simple and is extensively used [88, 96, 97]. All statistical analyses were performed using SAS 9.3 and R [85] [86–88].

Results

Literature Review

Of the 220 retrieved abstracts, 130 were excluded because they were reviews, case reports, or non-English language manuscripts. Sixteen publications were unobtainable through our library, leaving 74 manuscripts available for review. A PRISMA flow diagram is shown in Fig 3 and data shown in Table 4 [98]. Although nearly one-third of studies did not mention time to plasma, in the vast majority (nearly 58%), plasma extraction was completed ≤2h after phlebotomy. Most authors used either Trizol, miRNeasy, or mirVana protocols for RNA extraction, and nearly all publications used total miRNA rather than exosomal miRNA. Equal proportions of manuscripts used internal and external references (43% and 42%, respectively). The most commonly utilized internal reference miRNAs were miR-16 (12 publications) and RNU-6 (7 publications). Seventy-one of the 74 reviewed publications did not describe the setting of the cycle threshold bar. All but 2 of the 74 (97%) papers stated at least some aspect of their statistical methods, with many using multiple, different testing methods as shown in Table 4. Among reviewed manuscripts, only 1 (1.4%) listed all 6 assessed criteria [9–84].

thumbnail
Download:
Fig 3. A PRISMA flow diagram illustrating the search strategy used[98].

https://doi.org/10.1371/journal.pone.0121948.g003

thumbnail
Download:
Table 4. Systematic review: 74 plasma miRNA publications July 1, 2013—June 30, 2014.

https://doi.org/10.1371/journal.pone.0121948.t004

Time to Plasma Extraction

Our proposed standard technique provided a comparison. When performing plasma extraction from whole blood at different time points, we found that 6 miRNAs showed no differences in ΔCT values. In contrast, 5 miRNAs expressed a statistically significant difference in ΔCT values between immediate plasma extraction and extraction ≥ 24 h for 1 miRNA (miR-122), between immediate extraction and extraction after ≥48 h for 2 miRNAs (miR 485-3p, miR-21), and for 2 miRNAs with extraction at 72 h (miR 523, miR 218) (Table 5, raw data S1 Table). Based upon these data using the representative miRNA we have analyzed, plasma extraction less than 12h after phlebotomy appears to provide equivalent results to immediate (within 30 minutes of phlebotomy) extraction.

thumbnail
Download:
Table 5. Plasma extraction at multiple time points.

https://doi.org/10.1371/journal.pone.0121948.t005

Effect of Repeated Sample Acquisition

Repeated phlebotomy of the same individuals at 12-hour intervals showed no differences in miRNA expression values (Table 6, raw data S2 Table).

thumbnail
Download:
Table 6. Effect of Repeated Sample Acquisition.

https://doi.org/10.1371/journal.pone.0121948.t006

Setting of Cycle Threshold

Setting the cycle threshold bar at a fixed value of 0.03 was preferable to using the default “variable” threshold setting, provided that the number of missing values was less than 10%. Using the modified phenol/guanidine-based lysis and silica membrane-based extraction technique (mirNeasy), only 0.14% of values were missing (see Results‘ section “Method of RNA Extraction” below). With this method of extraction, therefore, utilization of a fixed threshold of 0.03 yields reproducible results without the need for normalization required with the use of a variable threshold. The statistical modeling and in depth description of this choice of threshold are discussed elsewhere (Rai et al., unpublished data, 2014).

Intra-Operator Variability

There was no significant variability between duplicate samples performed by a single operator (Table 7, raw data S3 Table).

thumbnail
Download:
Table 7. Intra-Operator Variability (Duplicates) for the Trizol LS RNA Extraction and Preamplicification Protocol.

https://doi.org/10.1371/journal.pone.0121948.t007

Inter-Operator Variability

Inter-operator variability was assessed using two different miRNA extraction techniques. Inter-operator variability using the Trizol purification and pre-amplification showed one miRNA miR-21 to be significantly different between operators. In addition, a large number of samples were noted to have no miRNA expression for 6 of the 11 evaluated miRNA (Table 8, raw data S3 Table). The large number of samples with no miRNA expression again highlighted the importance of selecting the best extraction technique, since this was not seen with the other method (see also data Table 9). Inter-operator variability using the Qiagen miRNeasy technique yielded 2 miRNAs (miR-485-3p and miR-21) with statistically significantly different ΔCT values (p-values comparing the ΔCT of the two different operators, 0.0191 and 0.0500, respectively) (Table 10, raw data S4 Table). It was believed that these differences in miRNA expression data between operators were within the range of experimental error. Indeed, when these experiments were repeated, these differences were not observed (data not shown). In order to determine whether any inter-observer differences would have a significant impact upon observed group differences we utilized an ANOVA model to examine variations in both miR-21 and RNU6. The group effect remained significant, even when there was an effect of inter-operator variability. These data are presented in S5 Table). The coefficient of variation for RNU6 was 9.1398, and 8.1211 for miR-21. In addition, we determined the mean and standard deviation of 64 measurements of miR-21 performed in plasma samples of patients with adenoma (mean 26.3 ± 1.8, median 25.9) as well as 64 measurements of miR-21 performed in plasma samples of controls (mean 27.6 ± 3.2, median 28.4).

thumbnail
Download:
Table 8. Differences in ΔCT between operators with Trizol LS RNA extraction and pre-amplification (inter-operator variability).

https://doi.org/10.1371/journal.pone.0121948.t008

thumbnail
Download:
Table 9. Inter-operator variability with two different extraction methods.

https://doi.org/10.1371/journal.pone.0121948.t009

thumbnail
Download:
Table 10. Differences in ΔCT between operators with Qiagen miRNeasy with RNA Carrier and No Pre-amplification (Inter-Operator Variability).

https://doi.org/10.1371/journal.pone.0121948.t010

Method of RNA Extraction

The Qiagen miRNeasy extraction without pre-amplification resulted in a higher yield of RNA than Trizol LS extraction; therefore, pre-amplification was not necessary and could be omitted as a source of possible variation (Table 11). The average ΔCT was lower with mirNeasy, with fewer missing values than with Trizol purification and pre-amplification. Qiagen miRNeasy extraction without pre-amplification resulted in only 1of 704 (0.14%) miRNAs not expressing; however, utilizing the Trizol LS extraction with pre-amplification, 109 of 704 (15%) of miRNAs investigated were not expressed (Table 9). A heat map showing miRNA expression for patient groups is shown in Fig 4.

thumbnail
Download:
Fig 4. Heat map showing expression of 11 miRNA in plasma of 16 patients with colorectal adenoma prior to treatment and in plasma of 16 controls.

Color gradation refers to delta Ct values with RNU6 as reference. Negative values represent over-expression of the target miRNA in comparison to the reference miRNA (RNU6).

https://doi.org/10.1371/journal.pone.0121948.g004

thumbnail
Download:
Table 11. Total RNA concentration and purity for Trizol LS and Qiagen miRNeasy with RNA yeast carrier.

https://doi.org/10.1371/journal.pone.0121948.t011

Discussion

With the growing field of miRNA research and the potential use of miRNAs as biomarkers for cancer and other human diseases, the lack of standardization and reproducibility among studies is becoming more apparent. Such variations in methodology can lead to large variability in results reported from one research group to another. In this study, 5 different sources of variation in experimental technique were examined with respect to their effect on miRNA expression data: 1) time of plasma extraction, 2) method of RNA extraction, 3) cycle threshold setting, 4) intra-operator variability and 5) inter-operator variability. Our brief literature review of a 1-year time frame, was not meant to be an exhaustive review, but rather to provide a “snap-shot” of current practices of performing experiments and analyzing data. Our literature review has shown, that many publications focusing on plasma miRNA do not carefully describe their experimental methods, that there is wide variation in performing such studies and analyzing resulting data.

There has been discussion as to whether plasma or serum is a better for study of circulating miRNA. While some report comparable data others report differences between the two sources [99–101]. We chose to focus on plasma, due to the concerns that miRNAs might be released from blood cells into the serum during the coagulation process as suggested by Wang et al [100].

Our data regarding the timing of plasma extraction suggest that plasma needs to be isolated rapidly, within hours after phlebotomy. With blood samples that were stored at 4°C, even with a modest sample size, we observed statistically significant different ΔCTs for select miRNA beginning as early as plasma extraction at 24 h. This is in agreement with prior reports linking hemolysis to altered miRNA expression [102]. Due to the limited sample size and small number of miRNA investigated, we suggest that plasma be isolated within 12 h following phlebotomy in order to avoid falsely elevated or reduced miRNA expression levels.

The three most widely used methods of total RNA extraction from plasma or serum are guanidinium thiocyanate-phenol chloroform-based methods (e.g., Trizol LS, Life Technologies, Carlsbad, CA), phenol with glass fiber filter-based (e.g. miRVANA, Ambion, Life Technologies), and phenol/guanidine with silica membrane based purification (e.g. miRNeasy, Qiagen, Venlo, The Netherlands). Moret et al. [90] compared all three methods and found that a modified phenol/guanidine lysis with silica membrane-based RNA extraction method yielded enhanced quantity, purity, and performance on assays. Although we did not assess the glass-fiber filter-based method here, our data agree with those of Moret et al. [90] and suggest that the Qiagen miRNeasy with yeast carrier isolation method yields a very reproducible result. The high concentration of 353 +/- 85 ng/μL total RNA obtained with this technique as compared to Trizol LS, which resulted in concentration of 19 +/- 12 ng/μL, allows for pre-amplification to be omitted prior to qPCR, omitting yet another source of data variability. In using this technique, we were able to reduce the number of “missing” samples (miRNA that did not express) from 15% to 0.14%.

Analysis of various fixed as well as of variable threshold settings indicated that a fixed Ct setting of 0.03 produced the most reproducible data provided that < 10% of data were missing. This has the significant advantage of not requiring the additional statistical adjustment that is required when a variable threshold is utilized (Rai et al., unpublished data, 2014).

We demonstrated a lack of significant intra-operator variability. In view of this, triplicates are not necessary, and if there are few missing values (<10%), one could even question the need to perform duplicates given the very low ΔΔCT. Inter-operator variation was also low. Using the phenol/guanidine-based lysis and silica membrane-based purification technique resulted in a narrowing of ΔΔCT values between operators as compared to guanidinium thiocyanate-phenol chloroform purification. In fact, when these experiments were repeated, the differences in miRNA expression between operators for miR-21 and miR-485-3p were not seen, leading us to conclude that this was within the range of experimental error and not a significant issue. The technique of robotic automation could potentially be used to reduce such an experimental error; we did not, however, have access to such technology.

Conclusions

Trizol LS extraction with pre-amplification results in unacceptable inter-operator variability and should not be utilized when analyzing plasma miRNA. A modified miRNeasy extraction method yields negligible inter-operator variation and the lowest number of missing values. For standardization, we recommend utilizing 0.03 as the cycle threshold bar. No significant intra-operator variability was observed. As such, miRNA studies that are restricted to duplicates rather than triplicates result in greater accuracy and cost savings.

While some inter-operator variation was noted, this was least with the modified phenol/guanidine-based lysis and silica membrane-based RNA purification technique. In addition to the sources of variability noted above, there are many others (Table 1). Similar to the QUADAS tool for the quality assessment of diagnostic accuracy studies, we suggest that studies of plasma miRNA should contain the following information: time to plasma extraction, method of RNA extraction, type of miRNA used (total vs exosomal), setting of cycle threshold, the type of quantification used, and details of the statistical analysis [103]. Based upon our studies, rapid plasma extraction is essential. A modified phenol/guanidine-based lysis and silica membrane-based purification RNA extraction is preferred due to the extremely low rate of missing values and high RNA yield, allowing the investigator to avoid using pre-amplification, another source of variability. If there is a low number of missing values (<10%), a fixed threshold setting of 0.03 provides the most reliable, consistent data without the need for data normalization. With this setting, there is no significant intra-operator variability, i.e. replicates can be restricted to duplicates, rather than triplicates, resulting in cost and labor savings. The use of plasma miRNA as biomarkers of human disease is evolving and expanding. Data reproducibility is essential prior to clinical application. Standardization of analytic methods and reporting is necessary to permit accurate data comparison and validation.

Supporting Information

S1 Table. Plasma extraction at multiple time points: within 30 minutes after phlebotomy, and at 12, 24, 48, and 72 h post-phlebotomy.

https://doi.org/10.1371/journal.pone.0121948.s001

(XLSX)

S2 Table. Effect of Repeated Sample Acquisition: samples drawn from each of 7 healthy subjects at 6.30 AM and 12 hours later at 6.30 PM on the same day.

https://doi.org/10.1371/journal.pone.0121948.s002

(XLS)

S3 Table. Intra-Operator Variability and Inter-Operator Variability: using the Trizol purification and pre-amplification.

https://doi.org/10.1371/journal.pone.0121948.s003

(XLS)

S4 Table. Inter-Operator Variability: using the Qiagen miRNeasy technique.

https://doi.org/10.1371/journal.pone.0121948.s004

(XLS)

S5 Table. Variability of miR21 and the housekeeping RNU6.

https://doi.org/10.1371/journal.pone.0121948.s005

(XLSX)

Author Contributions

Conceived and designed the experiments: JR HR SG. Performed the experiments: JR HR JB VS. Analyzed the data: SNR JP JR JB HR VS SG. Contributed reagents/materials/analysis tools: JR HR SG. Wrote the paper: JR SNR SG. Statistical analysis: SNR JP.

REFERENCES

  1. 1. Fire A, Xu S, Montgomery MK, Kostas SA, Driver SE, Mello CC. Potent and specific genetic interference by double-stranded RNA in Caenorhabditis elegans. Nature. 1998;391(6669):806–11. pmid:9486653.
  2. 2. Julia Winter SJ, Keller S, Gregory RI, Diederichs S. Many roads to maturity: microRNA biogenesis pathways and their regulation. Nature Cell Biology. 2009;11:228–34. pmid:19255566
  3. 3. Stefanie Sassen EM, Caldas C. MicroRNA-implications for cancer. Virchows Arch. 2008;452:1–10. Epub 11-27-2007. pmid:18040713
  4. 4. Aaron J. Schetter HO, Harris CC. The Role of microRNAs in Colorectal Cancer. Cancer J. 2012;18(3):244–52. Epub 05-01-2013. pmid:22647361
  5. 5. Dong Y, Wu WK, Wu CW, Sung JJ, Yu J, Ng SS. MicroRNA dysregulation in colorectal cancer: a clinical perspective. British journal of cancer. 2011;104(6):893–8. pmid:21364594; PubMed Central PMCID: PMC3065287.
  6. 6. Cho WC. Great potential of miRNAs as predictive and prognostic markers for cancer. Expert review of molecular diagnostics. 2012;12(4):315–8. pmid:22616694.
  7. 7. Cho WC. Circulating MicroRNAs as Minimally Invasive Biomarkers for Cancer Theragnosis and Prognosis. Frontiers in genetics. 2011;2:7. pmid:22303306; PubMed Central PMCID: PMC3268566.
  8. 8. Ichikawa M, Akiyama H. A combination of extraction reagent and DNA microarray that allows for the detection of global miRNA profiles from serum/plasma. Methods in molecular biology. 2013;1024:247–53. pmid:23719957.
  9. 9. Cheng L, Sharples RA, Scicluna BJ, Hill AF. Exosomes provide a protective and enriched source of miRNA for biomarker profiling compared to intracellular and cell-free blood. Journal of extracellular vesicles. 2014;3(23743). pmid:24683445; PubMed Central PMCID: PMC3968297.
  10. 10. Ganepola GA, Rutledge JR, Suman P, Yiengpruksawan A, Chang DH. Novel blood-based microRNA biomarker panel for early diagnosis of pancreatic cancer. World journal of gastrointestinal oncology. 2014;6(1):22–33. pmid:24578785; PubMed Central PMCID: PMC3936193.
  11. 11. Bhatnagar S, Chertkow H, Schipper HM, Yuan Z, Shetty V, Jenkins S, et al. Increased microRNA-34c abundance in Alzheimer's disease circulating blood plasma. Frontiers in molecular neuroscience. 2014;7:2. pmid:24550773; PubMed Central PMCID: PMC3912349.
  12. 12. Westermann AM, Schmidt D, Holdenrieder S, Moritz R, Semjonow A, Schmidt M, et al. Serum microRNAs as biomarkers in patients undergoing prostate biopsy: results from a prospective multi-center study. Anticancer research. 2014;34(2):665–9. pmid:24510997.
  13. 13. Tsukamoto O, Miura K, Mishima H, Abe S, Kaneuchi M, Higashijima A, et al. Identification of endometrioid endometrial carcinoma-associated microRNAs in tissue and plasma. Gynecologic oncology. 2014;132(3):715–21. pmid:24491411.
  14. 14. Zanutto S, Pizzamiglio S, Ghilotti M, Bertan C, Ravagnani F, Perrone F, et al. Circulating miR-378 in plasma: a reliable, haemolysis-independent biomarker for colorectal cancer. British journal of cancer. 2014;110(4):1001–7. pmid:24423916; PubMed Central PMCID: PMC3929896.
  15. 15. Sozzi G, Boeri M, Rossi M, Verri C, Suatoni P, Bravi F, et al. Clinical utility of a plasma-based miRNA signature classifier within computed tomography lung cancer screening: a correlative MILD trial study. Journal of clinical oncology: official journal of the American Society of Clinical Oncology. 2014;32(8):768–73. pmid:24419137.
  16. 16. Yu H, Jiang L, Sun C, Li Guo L, Lin M, Huang J, et al. Decreased circulating miR-375: a potential biomarker for patients with non-small-cell lung cancer. Gene. 2014;534(1):60–5. pmid:24404590.
  17. 17. Teixeira AL, Ferreira M, Silva J, Gomes M, Dias F, Santos JI, et al. Higher circulating expression levels of miR-221 associated with poor overall survival in renal cell carcinoma patients. Tumour biology: the journal of the International Society for Oncodevelopmental Biology and Medicine. 2014;35(5):4057–66. pmid:24379138.
  18. 18. Achberger S, Aldrich W, Tubbs R, Crabb JW, Singh AD, Triozzi PL. Circulating immune cell and microRNA in patients with uveal melanoma developing metastatic disease. Molecular immunology. 2014;58(2):182–6. pmid:24370793.
  19. 19. Shapira I, Oswald M, Lovecchio J, Khalili H, Menzin A, Whyte J, et al. Circulating biomarkers for detection of ovarian cancer and predicting cancer outcomes. British journal of cancer. 2014;110(4):976–83. pmid:24366298; PubMed Central PMCID: PMC3929876.
  20. 20. Szabo DR, Luconi M, Szabo PM, Toth M, Szucs N, Horanyi J, et al. Analysis of circulating microRNAs in adrenocortical tumors. Laboratory investigation; a journal of technical methods and pathology. 2014;94(3):331–9. pmid:24336071.
  21. 21. Lu J, Xu X, Liu X, Peng Y, Zhang B, Wang L, et al. Predictive value of miR-9 as a potential biomarker for nasopharyngeal carcinoma metastasis. British journal of cancer. 2014;110(2):392–8. pmid:24327016; PubMed Central PMCID: PMC3899774.
  22. 22. Quackenbush JF, Cassidy PB, Pfeffer LM, Boucher KM, Hawkes JE, Pfeffer SR, et al. Isolation of circulating microRNAs from microvesicles found in human plasma. Methods in molecular biology. 2014;1102:641–53. pmid:24259003.
  23. 23. Li H, Ge Q, Guo L, Lu Z. Maternal plasma miRNAs expression in preeclamptic pregnancies. BioMed research international. 2013;2013:970265. pmid:24195082; PubMed Central PMCID: PMC3781840.
  24. 24. Roncarati R, Viviani Anselmi C, Losi MA, Papa L, Cavarretta E, Da Costa Martins P, et al. Circulating miR-29a, among other up-regulated microRNAs, is the only biomarker for both hypertrophy and fibrosis in patients with hypertrophic cardiomyopathy. Journal of the American College of Cardiology. 2014;63(9):920–7. pmid:24161319.
  25. 25. Aushev VN, Zborovskaya IB, Laktionov KK, Girard N, Cros MP, Herceg Z, et al. Comparisons of microRNA patterns in plasma before and after tumor removal reveal new biomarkers of lung squamous cell carcinoma. PloS one. 2013;8(10):e78649. pmid:24130905; PubMed Central PMCID: PMC3793941.
  26. 26. Kishimoto T, Eguchi H, Nagano H, Kobayashi S, Akita H, Hama N, et al. Plasma miR-21 is a novel diagnostic biomarker for biliary tract cancer. Cancer science. 2013;104(12):1626–31. pmid:24118467.
  27. 27. Tian Q, Jia J, Ling S, Liu Y, Yang S, Shao Z. A causal role for circulating miR-34b in osteosarcoma. European journal of surgical oncology: the journal of the European Society of Surgical Oncology and the British Association of Surgical Oncology. 2014;40(1):67–72. pmid:24063968.
  28. 28. Xiao B, Wang Y, Li W, Baker M, Guo J, Corbet K, et al. Plasma microRNA signature as a noninvasive biomarker for acute graft-versus-host disease. Blood. 2013;122(19):3365–75. pmid:24041574; PubMed Central PMCID: PMC3821726.
  29. 29. Rong Y, Bao W, Shan Z, Liu J, Yu X, Xia S, et al. Increased microRNA-146a levels in plasma of patients with newly diagnosed type 2 diabetes mellitus. PloS one. 2013;8(9):e73272. pmid:24023848; PubMed Central PMCID: PMC3759444.
  30. 30. Garcia R, Villar AV, Cobo M, Llano M, Martin-Duran R, Hurle MA, et al. Circulating levels of miR-133a predict the regression potential of left ventricular hypertrophy after valve replacement surgery in patients with aortic stenosis. Journal of the American Heart Association. 2013;2(4):e000211. pmid:23948643; PubMed Central PMCID: PMC3828793.
  31. 31. Cheng HH, Mitchell PS, Kroh EM, Dowell AE, Chery L, Siddiqui J, et al. Circulating microRNA profiling identifies a subset of metastatic prostate cancer patients with evidence of cancer-associated hypoxia. PloS one. 2013;8(7):e69239. pmid:23935962; PubMed Central PMCID: PMC3728349.
  32. 32. Kumar P, Dezso Z, MacKenzie C, Oestreicher J, Agoulnik S, Byrne M, et al. Circulating miRNA biomarkers for Alzheimer's disease. PloS one. 2013;8(7):e69807. pmid:23922807; PubMed Central PMCID: PMC3726785.
  33. 33. Benson EA, Skaar TC. Incubation of whole blood at room temperature does not alter the plasma concentrations of microRNA-16 and -223. Drug metabolism and disposition: the biological fate of chemicals. 2013;41(10):1778–81. pmid:23886700; PubMed Central PMCID: PMC3781369.
  34. 34. Murata K, Furu M, Yoshitomi H, Ishikawa M, Shibuya H, Hashimoto M, et al. Comprehensive microRNA analysis identifies miR-24 and miR-125a-5p as plasma biomarkers for rheumatoid arthritis. PloS one. 2013;8(7):e69118. pmid:23874885; PubMed Central PMCID: PMC3715465.
  35. 35. Zheng W, Di Y, Liu Y, Huang G, Zheng Y, Zhang Y, et al. Development and application of a novel reverse transcription real-time PCR method for miR-499 quantification. Clinical biochemistry. 2013;46(15):1566–71. pmid:23872321.
  36. 36. Leuenberger N, Schumacher YO, Pradervand S, Sander T, Saugy M, Pottgiesser T. Circulating microRNAs as biomarkers for detection of autologous blood transfusion. PloS one. 2013;8(6):e66309. pmid:23840438; PubMed Central PMCID: PMC3688786.
  37. 37. Cheng HH, Yi HS, Kim Y, Kroh EM, Chien JW, Eaton KD, et al. Plasma processing conditions substantially influence circulating microRNA biomarker levels. PloS one. 2013;8(6):e64795. pmid:23762257; PubMed Central PMCID: PMC3676411.
  38. 38. Marfella R, Di Filippo C, Potenza N, Sardu C, Rizzo MR, Siniscalchi M, et al. Circulating microRNA changes in heart failure patients treated with cardiac resynchronization therapy: responders vs. non-responders. European journal of heart failure. 2013;15(11):1277–88. pmid:23736534.
  39. 39. Du J, Cao X, Zou L, Chen Y, Guo J, Chen Z, et al. MicroRNA-21 and risk of severe acute kidney injury and poor outcomes after adult cardiac surgery. PloS one. 2013;8(5):e63390. pmid:23717419; PubMed Central PMCID: PMC3662667.
  40. 40. Ellis KL, Cameron VA, Troughton RW, Frampton CM, Ellmers LJ, Richards AM. Circulating microRNAs as candidate markers to distinguish heart failure in breathless patients. European journal of heart failure. 2013;15(10):1138–47. pmid:23696613.
  41. 41. Gourzones C, Ferrand FR, Amiel C, Verillaud B, Barat A, Guerin M, et al. Consistent high concentration of the viral microRNA BART17 in plasma samples from nasopharyngeal carcinoma patients—evidence of non-exosomal transport. Virology journal. 2013;10:119. pmid:23590857; PubMed Central PMCID: PMC3685608.
  42. 42. Shrivastava S, Petrone J, Steele R, Lauer GM, Di Bisceglie AM, Ray RB. Up-regulation of circulating miR-20a is correlated with hepatitis C virus-mediated liver disease progression. Hepatology. 2013;58(3):863–71. pmid:23390075; PubMed Central PMCID: PMC3664107.
  43. 43. Gorur A, Balci Fidanci S, Dogruer Unal N, Ayaz L, Akbayir S, Yildirim Yaroglu H, et al. Determination of plasma microRNA for early detection of gastric cancer. Molecular biology reports. 2013;40(3):2091–6. pmid:23212612.
  44. 44. Olivieri F, Antonicelli R, Lorenzi M, D'Alessandra Y, Lazzarini R, Santini G, et al. Diagnostic potential of circulating miR-499-5p in elderly patients with acute non ST-elevation myocardial infarction. International journal of cardiology. 2013;167(2):531–6. pmid:22330002.
  45. 45. Khoo SK, Petillo D, Kang UJ, Resau JH, Berryhill B, Linder J, et al. Plasma-based circulating MicroRNA biomarkers for Parkinson's disease. Journal of Parkinson's disease. 2012;2(4):321–31. pmid:23938262.
  46. 46. Ayaz L, Gorur A, Yaroglu HY, Ozcan C, Tamer L. Differential expression of microRNAs in plasma of patients with laryngeal squamous cell carcinoma: potential early-detection markers for laryngeal squamous cell carcinoma. Journal of cancer research and clinical oncology. 2013;139(9):1499–506. pmid:23817697.
  47. 47. Cai H, Yuan Y, Hao YF, Guo TK, Wei X, Zhang YM. Plasma microRNAs serve as novel potential biomarkers for early detection of gastric cancer. Medical oncology. 2013;30(1):452. pmid:23307259.
  48. 48. De Rosa S, Fichtlscherer S, Lehmann R, Assmus B, Dimmeler S, Zeiher AM. Transcoronary concentration gradients of circulating microRNAs. Circulation. 2011;124(18):1936–44. pmid:21969012.
  49. 49. Fan KL, Zhang HF, Shen J, Zhang Q, Li XL. Circulating microRNAs levels in Chinese heart failure patients caused by dilated cardiomyopathy. Indian heart journal. 2013;65(1):12–6. pmid:23438607; PubMed Central PMCID: PMC3860780.
  50. 50. Fayyad-Kazan H, Bitar N, Najar M, Lewalle P, Fayyad-Kazan M, Badran R, et al. Circulating miR-150 and miR-342 in plasma are novel potential biomarkers for acute myeloid leukemia. Journal of translational medicine. 2013;11:31. pmid:23391324; PubMed Central PMCID: PMC3579719.
  51. 51. Freedman JE, Ercan B, Morin KM, Liu CT, Tamer L, Ayaz L, et al. The distribution of circulating microRNA and their relation to coronary disease. F1000Research. 2012;1:50. pmid:24358814; PubMed Central PMCID: PMC3752638.
  52. 52. Gandhi R, Healy B, Gholipour T, Egorova S, Musallam A, Hussain MS, et al. Circulating microRNAs as biomarkers for disease staging in multiple sclerosis. Annals of neurology. 2013;73(6):729–40. pmid:23494648.
  53. 53. Ge Q, Li H, Yang Q, Lu J, Tu J, Bai Y, et al. Sequencing circulating miRNA in maternal plasma with modified library preparation. Clinica chimica acta; international journal of clinical chemistry. 2011;412(21–22):1989–94. pmid:21806976.
  54. 54. Gidlof O, Smith JG, Miyazu K, Gilje P, Spencer A, Blomquist S, et al. Circulating cardio-enriched microRNAs are associated with long-term prognosis following myocardial infarction. BMC cardiovascular disorders. 2013;13:12. pmid:23448306; PubMed Central PMCID: PMC3598930.
  55. 55. Giraldez MD, Lozano JJ, Ramirez G, Hijona E, Bujanda L, Castells A, et al. Circulating microRNAs as biomarkers of colorectal cancer: results from a genome-wide profiling and validation study. Clin Gastroenterol Hepatol. 2013;11(6):681–8 e3. pmid:23267864.
  56. 56. Hirajima S, Komatsu S, Ichikawa D, Takeshita H, Konishi H, Shiozaki A, et al. Clinical impact of circulating miR-18a in plasma of patients with oesophageal squamous cell carcinoma. British journal of cancer. 2013;108(9):1822–9. pmid:23579215; PubMed Central PMCID: PMC3658511.
  57. 57. Huang JJ, Yu J, Li JY, Liu YT, Zhong RQ. Circulating microRNA expression is associated with genetic subtype and survival of multiple myeloma. Medical oncology. 2012;29(4):2402–8. pmid:22447484.
  58. 58. Huang X, Yuan T, Tschannen M, Sun Z, Jacob H, Du M, et al. Characterization of human plasma-derived exosomal RNAs by deep sequencing. BMC genomics. 2013;14:319. pmid:23663360; PubMed Central PMCID: PMC3653748.
  59. 59. Jia SZ, Yang Y, Lang J, Sun P, Leng J. Plasma miR-17-5p, miR-20a and miR-22 are down-regulated in women with endometriosis. Human reproduction. 2013;28(2):322–30. pmid:23203215; PubMed Central PMCID: PMC3733164.
  60. 60. Kanaan Z, Roberts H, Eichenberger MR, Billeter A, Ocheretner G, Pan J, et al. A Plasma MicroRNA Panel for Detection of Colorectal Adenomas: A Step Toward More Precise Screening for Colorectal Cancer. Ann Surg. 2013;258(3):400–8. pmid:24022433.
  61. 61. Kawano Y, Iwata S, Kawada J, Gotoh K, Suzuki M, Torii Y, et al. Plasma viral microRNA profiles reveal potential biomarkers for chronic active Epstein-Barr virus infection. The Journal of infectious diseases. 2013;208(5):771–9. pmid:23687223.
  62. 62. Kin K, Miyagawa S, Fukushima S, Shirakawa Y, Torikai K, Shimamura K, et al. Tissue- and plasma-specific MicroRNA signatures for atherosclerotic abdominal aortic aneurysm. Journal of the American Heart Association. 2012;1(5):e000745. pmid:23316282; PubMed Central PMCID: PMC3541608.
  63. 63. Kumar S, Keerthana R, Pazhanimuthu A, Perumal P. Overexpression of circulating miRNA-21 and miRNA-146a in plasma samples of breast cancer patients. Indian journal of biochemistry & biophysics. 2013;50(3):210–4. pmid:23898484.
  64. 64. Laterza OF, Scott MG, Garrett-Engele PW, Korenblat KM, Lockwood CM. Circulating miR-122 as a potential biomarker of liver disease. Biomarkers in medicine. 2013;7(2):205–10. pmid:23547815.
  65. 65. Leidner RS, Li L, Thompson CL. Dampening enthusiasm for circulating microRNA in breast cancer. PloS one. 2013;8(3):e57841. pmid:23472110; PubMed Central PMCID: PMC3589476.
  66. 66. Li C, Li JF, Cai Q, Qiu QQ, Yan M, Liu BY, et al. MiRNA-199a-3p: A potential circulating diagnostic biomarker for early gastric cancer. Journal of surgical oncology. 2013;108(2):89–92. pmid:23733518.
  67. 67. Li YQ, Zhang MF, Wen HY, Hu CL, Liu R, Wei HY, et al. Comparing the diagnostic values of circulating microRNAs and cardiac troponin T in patients with acute myocardial infarction. Clinics. 2013;68(1):75–80. pmid:23420161; PubMed Central PMCID: PMC3552456.
  68. 68. Luo X, Stock C, Burwinkel B, Brenner H. Identification and evaluation of plasma MicroRNAs for early detection of colorectal cancer. PloS one. 2013;8(5):e62880. pmid:23690963; PubMed Central PMCID: PMC3653912.
  69. 69. Moldovan L, Batte K, Wang Y, Wisler J, Piper M. Analyzing the circulating microRNAs in exosomes/extracellular vesicles from serum or plasma by qRT-PCR. Methods in molecular biology. 2013;1024:129–45. pmid:23719947; PubMed Central PMCID: PMC3923604.
  70. 70. Nair N, Kumar S, Gongora E, Gupta S. Circulating miRNA as novel markers for diastolic dysfunction. Molecular and cellular biochemistry. 2013;376(1–2):33–40. pmid:23247724.
  71. 71. Ng EK, Li R, Shin VY, Jin HC, Leung CP, Ma ES, et al. Circulating microRNAs as specific biomarkers for breast cancer detection. PloS one. 2013;8(1):e53141. pmid:23301032; PubMed Central PMCID: PMC3536802.
  72. 72. Ouyang L, Liu P, Yang S, Ye S, Xu W, Liu X. A three-plasma miRNA signature serves as novel biomarkers for osteosarcoma. Medical oncology. 2013;30(1):340. pmid:23269581.
  73. 73. Parikh VN, Chan SY. Analysis of microRNA niches: techniques to measure extracellular microRNA and intracellular microRNA in situ. Methods in molecular biology. 2013;1024:157–72. pmid:23719949; PubMed Central PMCID: PMC3753217.
  74. 74. Prats-Puig A, Ortega FJ, Mercader JM, Moreno-Navarrete JM, Moreno M, Bonet N, et al. Changes in circulating microRNAs are associated with childhood obesity. The Journal of clinical endocrinology and metabolism. 2013;98(10):E1655-60. pmid:23928666.
  75. 75. Sanfiorenzo C, Ilie MI, Belaid A, Barlesi F, Mouroux J, Marquette CH, et al. Two panels of plasma microRNAs as non-invasive biomarkers for prediction of recurrence in resectable NSCLC. PloS one. 2013;8(1):e54596. pmid:23342174; PubMed Central PMCID: PMC3546982.
  76. 76. Shimizu C, Kim J, Stepanowsky P, Trinh C, Lau HD, Akers JC, et al. Differential expression of miR-145 in children with Kawasaki disease. PloS one. 2013;8(3):e58159. pmid:23483985; PubMed Central PMCID: PMC3590129.
  77. 77. Soeda S, Ohyashiki JH, Ohtsuki K, Umezu T, Setoguchi Y, Ohyashiki K. Clinical relevance of plasma miR-106b levels in patients with chronic obstructive pulmonary disease. International journal of molecular medicine. 2013;31(3):533–9. pmid:23338559.
  78. 78. Suryawanshi S, Vlad AM, Lin HM, Mantia-Smaldone G, Laskey R, Lee M, et al. Plasma microRNAs as novel biomarkers for endometriosis and endometriosis-associated ovarian cancer. Clinical cancer research: an official journal of the American Association for Cancer Research. 2013;19(5):1213–24. pmid:23362326; PubMed Central PMCID: PMC3596045.
  79. 79. Takahashi K, Yokota S, Tatsumi N, Fukami T, Yokoi T, Nakajima M. Cigarette smoking substantially alters plasma microRNA profiles in healthy subjects. Toxicology and applied pharmacology. 2013;272(1):154–60. pmid:23726802.
  80. 80. Wang E, Nie Y, Zhao Q, Wang W, Huang J, Liao Z, et al. Circulating miRNAs reflect early myocardial injury and recovery after heart transplantation. Journal of cardiothoracic surgery. 2013;8:165. pmid:23816326; PubMed Central PMCID: PMC3716980.
  81. 81. Winther TN, Bang-Berthelsen CH, Heiberg IL, Pociot F, Hogh B. Differential plasma microRNA profiles in HBeAg positive and HBeAg negative children with chronic hepatitis B. PloS one. 2013;8(3):e58236. pmid:23469271; PubMed Central PMCID: PMC3587589.
  82. 82. Yamamoto M, Singh A, Sava F, Pui M, Tebbutt SJ, Carlsten C. MicroRNA expression in response to controlled exposure to diesel exhaust: attenuation by the antioxidant N-acetylcysteine in a randomized crossover study. Environmental health perspectives. 2013;121(6):670–5. pmid:23584289; PubMed Central PMCID: PMC3672916.
  83. 83. Zeng RC, Zhang W, Yan XQ, Ye ZQ, Chen ED, Huang DP, et al. Down-regulation of miRNA-30a in human plasma is a novel marker for breast cancer. Medical oncology. 2013;30(1):477. pmid:23389917.
  84. 84. Zhao J, Lei T, Xu C, Li H, Ma W, Yang Y, et al. MicroRNA-187, down-regulated in clear cell renal cell carcinoma and associated with lower survival, inhibits cell growth and migration though targeting B7-H3. Biochemical and biophysical research communications. 2013;438(2):439–44. pmid:23916610.
  85. 85. Product Documentation: SAS Institute Inc.; 2014 [cited 2014 August 11, 2014]. Available from: http://support.sas.com/documentation/93/.
    • 86. Schoen RE, Pinsky PF, Weissfeld JL, Yokochi LA, Church T, Laiyemo AO, et al. Colorectal-cancer incidence and mortality with screening flexible sigmoidoscopy. The New England journal of medicine. 2012;366(25):2345–57. pmid:22612596; PubMed Central PMCID: PMC3641846.
    • 87. Rai SN RH, Yuan X, Pan J, Hamid T, Prabhu SD. Statistical Analysis of Repeated MicroRNA High Throughput Data with Application to Human Heart Failure: A Methodology Review. Open Access Medical Statistics, In Press. 2012. pmid:24738042
      • 88. Gonen M, editor Receiver Operating Characteristics (ROC) Curves. Thirty-first Annual SAS Users Group International Conference; 2006 Proceedings of the Thirty-first Annual SAS Users Group International Conference, Paper 210–31. Cary, NC: SAS Institute Inc..
        • 89. Technologies L. TRIzol LS Reagent http://tools.lifetechnologies.com/content/sfs/manuals/trizol_ls_reagent.pdf: Life Technologies; 2010 [cited 2014].
          • 90. Moret I, Sanchez-Izquierdo D, Iborra M, Tortosa L, Navarro-Puche A, Nos P, et al. Assessing an improved protocol for plasma microRNA extraction. PloS one. 2013;8(12):e82753. pmid:24376572; PubMed Central PMCID: PMC3871541.
          • 91. Technologies L. Protocol for Creating Custom RT and Preamplification Pools using TaqMan MicroRNA Assays http://tools.lifetechnologies.com/content/sfs/manuals/cms_094060.pdf: 2013; 2013 [cited 2014].
            • 92. Fleiss JL. Design and Analysis of Clinical Experiments. New York: John Wiley and Sons; 1986.
              • 93. Dmitrienko A MG, Chuang-Stein C and Offen W. Analysis of Clinical Trials Using SAS: A practical Guide. Cary, NC, USA: SAS Institute Inc.; 2005.
                • 94. Cambon AC BK, Brock GN, Cooper NGF, Wu D, and Rai SN. Classification of Clinical Outcomes Using High-throughput Informatics: Part 1- Nonparametric Method Reviews. Model Assisted Statistics and Applications. 2014. Epub In Press.
                  • 95. Cambon AC BK, Brock GN, Cooper NGF, Wu D, and Rai SN. Classification of Clinical Outcomes Using High-throughput Informatics: Part 2- Parametric Method Reviews. Model Assisted Statistics and Applications. 2014. Epub In Press.
                    • 96. Pepe MS, Etzioni R, Feng Z, Potter JD, Thompson ML, Thornquist M, et al. Phases of biomarker development for early detection of cancer. Journal of the National Cancer Institute. 2001;93(14):1054–61. pmid:11459866.
                    • 97. Pepe MS. The Statistical Evaluation of Medical Tests for Classification and Prediction: Oxford; 2003.
                      • 98. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS medicine. 2009;6(7):e1000097. pmid:19621072; PubMed Central PMCID: PMC2707599.
                      • 99. Mitchell PS, Parkin RK, Kroh EM, Fritz BR, Wyman SK, Pogosova-Agadjanyan EL, et al. Circulating microRNAs as stable blood-based markers for cancer detection. Proceedings of the National Academy of Sciences of the United States of America. 2008;105(30):10513–8. pmid:18663219; PubMed Central PMCID: PMC2492472.
                      • 100. Wang K, Yuan Y, Cho JH, McClarty S, Baxter D, Galas DJ. Comparing the MicroRNA spectrum between serum and plasma. PloS one. 2012;7(7):e41561. pmid:22859996; PubMed Central PMCID: PMC3409228.
                      • 101. Wedge DC, Allwood JW, Dunn W, Vaughan AA, Simpson K, Brown M, et al. Is serum or plasma more appropriate for intersubject comparisons in metabolomic studies? An assessment in patients with small-cell lung cancer. Analytical chemistry. 2011;83(17):6689–97. pmid:21766834.
                      • 102. Kirschner MB, Kao SC, Edelman JJ, Armstrong NJ, Vallely MP, van Zandwijk N, et al. Haemolysis during sample preparation alters microRNA content of plasma. PloS one. 2011;6(9):e24145. pmid:21909417; PubMed Central PMCID: PMC3164711.
                      • 103. Penny Whiting AWR, Reitsma JB, Bossuyt PMM, Kleijnen J. The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Medical Research Methodology. 2003;3(25). Epub 11-10-2003. pmid:14606960