Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

From Gigabyte to Kilobyte: A Bioinformatics Protocol for Mining Large RNA-Seq Transcriptomics Data

  • Jilong Li,

    Affiliations Computer Science Department, University of Missouri, Columbia, Missouri, United States of America, MU Botanical Center, University of Missouri, Columbia, Missouri, United States of America

  • Jie Hou,

    Affiliation Computer Science Department, University of Missouri, Columbia, Missouri, United States of America

  • Lin Sun,

    Affiliation Division of Biological Sciences, University of Missouri, Columbia, Missouri, United States of America

  • Jordan Maximillian Wilkins,

    Affiliation Department of Biochemistry, University of Missouri, Columbia, Missouri, United States of America

  • Yuan Lu,

    Affiliations MU Botanical Center, University of Missouri, Columbia, Missouri, United States of America, Department of Biochemistry, University of Missouri, Columbia, Missouri, United States of America

  • Chad E. Niederhuth,

    Affiliation Division of Biological Sciences, University of Missouri, Columbia, Missouri, United States of America

  • Benjamin Ryan Merideth,

    Affiliation Department of Biochemistry, University of Missouri, Columbia, Missouri, United States of America

  • Thomas P. Mawhinney,

    Affiliation Department of Biochemistry, University of Missouri, Columbia, Missouri, United States of America

  • Valeri V. Mossine,

    Affiliations MU Botanical Center, University of Missouri, Columbia, Missouri, United States of America, Department of Biochemistry, University of Missouri, Columbia, Missouri, United States of America

  • C. Michael Greenlief,

    Affiliations MU Botanical Center, University of Missouri, Columbia, Missouri, United States of America, Department of Chemistry, University of Missouri, Columbia, Missouri, United States of America

  • John C. Walker,

    Affiliation Division of Biological Sciences, University of Missouri, Columbia, Missouri, United States of America

  • William R. Folk,

    Affiliations MU Botanical Center, University of Missouri, Columbia, Missouri, United States of America, Department of Biochemistry, University of Missouri, Columbia, Missouri, United States of America

  • Mark Hannink,

    Affiliations MU Botanical Center, University of Missouri, Columbia, Missouri, United States of America, Department of Biochemistry, University of Missouri, Columbia, Missouri, United States of America

  • Dennis B. Lubahn,

    Affiliations MU Botanical Center, University of Missouri, Columbia, Missouri, United States of America, Department of Biochemistry, University of Missouri, Columbia, Missouri, United States of America

  • James A. Birchler,

    Affiliation Division of Biological Sciences, University of Missouri, Columbia, Missouri, United States of America

  •  [ ... ],
  • Jianlin Cheng

    chengji@missouri.edu

    Affiliations Computer Science Department, University of Missouri, Columbia, Missouri, United States of America, MU Botanical Center, University of Missouri, Columbia, Missouri, United States of America, Informatics Institute, University of Missouri, Columbia, Missouri, United States of America, C. Bond Life Science Center, University of Missouri, Columbia, Missouri, United States of America

  • [ view all ]
  • [ view less ]

Abstract

RNA-Seq techniques generate hundreds of millions of short RNA reads using next-generation sequencing (NGS). These RNA reads can be mapped to reference genomes to investigate changes of gene expression but improved procedures for mining large RNA-Seq datasets to extract valuable biological knowledge are needed. RNAMiner—a multi-level bioinformatics protocol and pipeline—has been developed for such datasets. It includes five steps: Mapping RNA-Seq reads to a reference genome, calculating gene expression values, identifying differentially expressed genes, predicting gene functions, and constructing gene regulatory networks. To demonstrate its utility, we applied RNAMiner to datasets generated from Human, Mouse, Arabidopsis thaliana, and Drosophila melanogaster cells, and successfully identified differentially expressed genes, clustered them into cohesive functional groups, and constructed novel gene regulatory networks. The RNAMiner web service is available at http://calla.rnet.missouri.edu/rnaminer/index.html.

Introduction

Transcriptome analysis is essential for determining the relationship between the information encoded in a genome, its expression, and phenotypic variation [1,2]. Next-generation sequencing (NGS) of RNAs (RNA-Seq) has emerged as a powerful approach for transcriptome analysis [3,4] that has many advantages over microarray technologies [5,6,7].

A RNA-Seq experiment typically generates hundreds of millions of short reads that are mapped to reference genomes and counted as a measure of expression [5]. Mining the gigabytes or even terabytes of RNA-Seq raw data is an essential, but challenging step in the analysis.

In order to address these challenges, RNAMiner has been developed to convert gigabytes of raw RNA-Seq data into kilobytes of valuable biological knowledge through a five-step data mining and knowledge discovery process. RNAMiner integrates both public tools (e.g., TopHat2 [8], Bowtie2 [9], Cufflinks [10], HTSeq [11], edgeR [12], and DESeq2 [13]) with our in-house tools (MULTICOM-MAP [14,15,16]) to preprocess data and identify differentially expressed genes in the first three steps. In the last two steps, RNAMiner uses our in-house tools MULTICOM-PDCN [17,18] and MULTICOM-GNET [19,20] to predict both functions and gene regulatory networks of differentially expressed genes, respectively.

As proof of principle, we have applied the RNAMiner protocol to RNA-Seq data generated from Human, Mouse, Arabidopsis thaliana, and Drosophila melanogaster cells. The data mining process successfully produced valuable biological knowledge such as differentially expressed genes, cohesive functional gene groups, and novel hypothetical gene regulatory networks by reducing the size of the initial data set over a thousand-fold.

Methods

Some RNA-Seq data analysis pipelines (e.g. Galaxy [21], KBase [22], iPlant [23]) provide users with a convenient and free platform for RNA-Seq data analysis by combing public tools, such as TopHat [24], Bowtie [25], Cufflinks [10], Cuffmerge [10], and Cuffdiff [10]. As with these pipelines, RNAMiner combines these public tools such as TopHat2 [8], Bowtie2 [9], Cufflinks [10], Cuffdiff [10], and it is free. However, there are several differences between RNAMiner and other pipelines. First, RNAMiner integrates more tools, such as HTSeq [11], edgeR [12], DESeq2 [13], and our in-house MULTICOM-MAP [14,15,16], to calculate gene expression values and identify differentially expressed genes. These tools can generate more accurate consensus results. For example, RNAMiner uses Cuffdiff, edgeR, and DESeq2 to identify differentially expressed genes based on TopHat mapping results and gene expression values calculated by HTSeq and MULTICOM-MAP. RNAMiner generates up to five distinct lists and one consensus list of differentially expressed genes, which usually produces more accurate results. Second, RNAMiner predicts functions of differentially expressed genes and builds gene regulatory networks by integrating our in-house tools MULTICOM-PDCN [17,18] and MULTICOM-GNET [19,20]. These analyses provide more biological information. Other pipelines (e.g. Galaxy and iPlant) do not provide these analyses. Another software package—KBase—contains a service to predict gene functions, but the service only provides GO annotation for plant genomes. Third, without requirements for user registration and selection of many parameters, RNAMiner is easier to use than other pipelines. Compared to running each tool separately, users can easily run all these tools integrated in RNAMiner at one time and download results generated by all the tools at the RNAMiner web site.

The five data analysis steps of the RNAMiner protocol (Fig 1) are described individually in sub-sections below. Tables 1 and 2 list the versions and the parameters of all the public tools used in RNAMiner and describe the meanings of the parameters.

thumbnail
Fig 1. The RNAMiner protocol for big transcriptomics data analysis.

Five blue boxes denote five data analysis steps, i.e. mapping RNA-Seq reads to a reference genome, calculating gene expression values, identifying differentially expressed genes, predicting gene functions, and constructing gene regulatory networks. The tools used in each step are listed inside each box. The external input information is represented by brown boxes and the final output information is represented by green boxes. The information flow between these components is denoted by arrows.

https://doi.org/10.1371/journal.pone.0125000.g001

thumbnail
Table 1. The versions of the public tools used in RNAMiner.

https://doi.org/10.1371/journal.pone.0125000.t001

thumbnail
Table 2. The parameters of the public tools used in RNAMiner, the parameter values, and the descriptions.

https://doi.org/10.1371/journal.pone.0125000.t002

1. Mapping RNA-Seq reads to a reference genome

We use two public tools, TopHat2 [8] and Bowtie2 [9], to map RNA-Seq reads to reference genomes in the UCSC genome browser [26] in conjunction with the RefSeq genome reference annotations [27]. The workflow of mapping RNA-Seq reads to a reference genome and calculating gene expression values is illustrated in Fig 2. It is worth noting that, since the RefSeq genome reference annotations contain information about some non-coding small RNAs, the reads of the non-coding RNAs are mapped and counted in addition to regular protein coding mRNAs. MULTICOM-MAP [14,15,16] is used to remove reads mapped to multiple locations in a reference genome from the mapping data in BAM/SAM format [28] generated by TopHat2 and Bowtie2. Only reads mapped to a unique location on the genome are retained to calculate the read counts of the genes. We use MULTICOM-MAP to analyze the mapping results to obtain baseline information, such as the total number of reads, the number of reads mapped to a unique location, and the number of reads mapped to multiple locations. This mapping process can generally reduce the size of datasets by several orders of magnitude.

thumbnail
Fig 2. The workflow of mapping RNA-Seq reads to a reference genome and calculating gene expression values.

The blue boxes denote the tools (TopHat2, Bowtie2, MULTICOM-MAP, HTSeq, Cufflinks) used in the steps of mapping RNA-Seq reads to a reference genome and calculating gene expression values. The external input information is represented by brown boxes and the output information is represented by green boxes. The information flow between these components is denoted by arrows.

https://doi.org/10.1371/journal.pone.0125000.g002

2. Calculating gene expression values

For RNAMiner, MULTICOM-MAP [14,15,16] and two public tools: HTSeq [11] and Cufflinks [10] are used to calculate gene expression values according to the genome mapping output and the RefSeq genome reference annotation [27]. MULTICOM-MAP and HTSeq produce raw read counts, while Cufflinks generates normalized values in terms of FPKM, i.e., fragments per kilobase of exon model per million mapped fragments. The normalized gene expression values generated by Cufflinks are used to identify differentially expressed genes in the next step. The read counts generated by MULTICOM-MAP and HTSeq are fed separately into two R Bioconductor packages, edgeR [12] and DESeq2 [13], to identify differentially expressed genes. The normalized gene expression values (RPKM, reads per kilobase of exon model per million mapped reads) of MULTICOM-MAP are used to construct gene regulatory networks in the last step. Cufflinks, MULTICOM-MAP, and HTSeq are largely complementary and mostly differ in how they handle the reads mapped to common exons of multiple isoforms of a gene. Cufflinks distributes the count of such reads to each isoform proportionally according the estimated probability that the reads were derived from the isoform. In contrast, MULTICOM-MAP distributes the total count of such reads to each isoform, while HTSeq discards the reads without counting them for any isoform. This analysis step generates the overall expression profile of most genes in a transcriptome and can reduce the size of data from Step 1 by ~one thousand-fold, from gigabytes to several megabytes.

3. Identifying differentially expressed genes

We use Cuffdiff [10] and two R Bioconductor packages, edgeR [12] and DESeq2 [13] to identify differentially expressed genes separately (see Fig 3 for the workflow). EdgeR and DESeq2 identify differentially expressed genes based on the raw read counts calculated by MULTICOM-MAP and HTSeq, resulting in four lists of differentially expressed genes (i.e., edgeR+MULTICOM-MAP, edgeR+HTSeq, DESeq2+MULTICOM-MAP, and DESeq2+HTSeq). In contrast Cuffdiff identifies differentially expressed genes directly from the genome mapping outputs containing only reads mapped to a unique location on the genome, resulting in one list of differentially expressed genes. Cuffdiff, edgeR and DESeq2 further adjust p-values by multiple testing using Benjamini and Hochberg's approach, which controls the false discovery rate (FDR) [10,12,13]. Usually, the cut-off of p-value (or q-value) is set to 0.05. Based on the five lists of differentially expressed genes generated by Cuffdiff, edgeR+MULTICOM-MAP, DESeq2+MULTICOM-MAP, edgeR+HTSeq, and DESeq2+HTSeq, a consensus list of differentially expressed genes is generated as the final output which usually comes from the overlap of at least three lists of differentially expressed genes. This step generates valuable information that may play an important role in the biological experiment. For example, the significantly differentially expressed genes identified by RNAMiner could be the targets for new biological experiments. This analysis step can generally reduce the size of data of the previous step by a couple orders of magnitude, condensing the data set size to several hundred kilobytes.

thumbnail
Fig 3. The workflow of identifying differentially expressed genes.

The blue boxes denote the tools (edgeR, DESeq2, Cuffdiff) used in the step of identifying differentially expressed genes. The external input information is represented by brown boxes and the output information is represented by green boxes. The information flow between these components is denoted by arrows.

https://doi.org/10.1371/journal.pone.0125000.g003

4. Predicting gene functions

We use MULTICOM-PDCN [17,18], a protein function prediction method ranked among the top methods in the 2011–2012 Critical Assessment of Function Annotation (CAFA) [29], to predict functions of differentially expressed genes (see Fig 4 for the workflow). MULTICOM-PDCN integrates sequence-profile and profile-profile alignment methods (PSI-BLAST [30] and HHSearch [31]) with protein function databases such as the Gene Ontology database [32], the Swiss-Prot database [33], and the Pfam database [34], to predict functions of proteins in Gene Ontology [32] terms in three categories: biological process, molecular function, and cellular component. MULTICOM-PDCN also provides some statistical information about the predicted functions, such as the number of differentially expressed genes predicted in each function. We then use the Cochran-Mantel-Haenszel test implemented by R program mantelhaen.test [35,36] to check if predicted function terms are good for Fisher’s exact test to identify the significantly enriched GO function terms. A p-value from the MH test lower than 0.05 suggests the two nominal variables (e.g., two function terms) are conditionally independent in each stratum [35,36]. We then calculate a p-value of enrichment for each predicted function using R function fisher.test [35,36,37,38,39,40,41,42] and sort the predicted functions by their p-value in ascending order, from the most significant ones to the least significant ones. The list of the most significantly enriched functions can provide an overview of the biological processes differentially perturbed in two biological conditions. Although the physical size of the data and knowledge generated in this step is comparable to the size of the data in the previous step, the differentially expressed genes can be organized in three functional perspectives: biological process, molecular function, and cellular component.

thumbnail
Fig 4. The workflow of predicting gene functions.

The blue box denotes the tool used in the step of predicting gene functions. The tools of PSI-BLAST and HHSearch used in MULTICOM-PDCN are listed inside the blue box. The external input information is represented by brown boxes and the output information is represented by green boxes. The information flow between these components is denoted by arrows.

https://doi.org/10.1371/journal.pone.0125000.g004

5. Constructing gene regulatory networks

We use MULTICOM-GNET [19,20] to construct gene regulatory networks based on differentially expressed genes and transcription factors in a genome (see Fig 5 for the workflow). MULTICOM-GNET firstly clusters differentially expressed genes with similar expression patterns into functional clusters using the K-means clustering algorithm. Secondly, it builds a binary decision tree to represent potential regulatory relationships between several selected transcription factors (TFs) and the genes in each cluster. Thirdly, it re-assigns differentially expressed genes into clusters whose gene regulatory tree best explained the expression patterns of the genes. The last two steps are repeated until the maximized likelihood of the gene expression data is reached [19,20]. We also use a R network analysis and visualization package “igraph” [43] to visualize gene regulatory networks by linking the regulatory relationships between and within all the gene regulatory modules predicted by MULTICOM-GNET together. The regulatory network construction step generates a comprehensive understanding of underlying mechanisms controlling the expression of a transcriptome and can significantly reduce the size of data. The hundreds of kilobytes of the biological network data provide a system view of the cellular systems, which can be more readily utilized to generate valuable hypotheses for biological experiments.

thumbnail
Fig 5. The workflow of constructing gene regulatory networks.

The blue boxes denote the methods used by MULTICOM-GNET in constructing gene regulatory networks. The external input information is represented by brown boxes and the output information is represented by green boxes. The information flow between these components is denoted by arrows.

https://doi.org/10.1371/journal.pone.0125000.g005

For replicates from RNA-Seq experiments, RNAMiner maps reads of the replicates to reference genomes and calculates gene expression values separately. The gene expression values of the replicates of two samples are combined into a profile (i.e. a vector of the expression values of a gene in each replicate of each condition), which is input into edgeR and DESeq2 to identify differentially expressed genes. Additionally, the TopHat mapping results of the replicates of two samples are input into Cuffdiff to identify differentially expressed genes. EdgeR [12], DESeq2 [13], and Cuffdiff [10] handle the replicates by modeling the variance (dispersion) in counts across the replicates as a function of the mean count of the replicates. EdgeR [12] estimates the variance by conditional maximum likelihood conditioned on the total count for the gene. DESeq2 [13] uses a flexible and mean-dependent local regression to estimate the variance between the replicates by pooling genes with similar expression levels to enhance the variance estimation. Cuffdiff [10] estimates the variance based on a negative binomial model and uses t-test to calculate the test statistics. Cuffdiff can make a model on each condition with replicates, or use a global model for all conditions together.

Before calling a tool to do data analysis, RNAMiner checks whether the data is appropriate to the tool. For example, MULTICOM-GNET [19,20] is not applied if no transcription factors exist in differentially expressed genes because MULTICOM-GNET needs at least one transcription factor to build gene regulatory networks. Another example is, for some special datasets, overexpression of some treatments in some regions of the genome in one condition leads to very large read counts of some genes in this condition, and dramatic differences of gene expressions between two conditions. This violates the assumption of edgeR’s normalization method [12] that the majority of the genes should have similar expression levels. Therefore, calculating a normalization factor across all loci is difficult. RNAMiner will check this assumption and will not call edgeR if it is violated.

Evaluation and Discussion

We tested the RNAMiner protocol on six sets of RNA-Seq data generated from Human, Mouse, Arabidopsis thaliana and Drosophila melanogaster cells in order to evaluate its effectiveness. The details such as organisms, biological conditions, and experimental settings about the six sets of RNA-Seq data were reported in Table 3. The results of each of the five analysis steps are described and discussed as follows.

thumbnail
Table 3. The organisms, conditions, replicate numbers, and descriptions of the six sets of RNA-Seq data.

https://doi.org/10.1371/journal.pone.0125000.t003

1. Results of mapping RNA-Seq reads to a reference genome

RNAMiner used TopHat2 [8] and Bowtie [44] to map RNA-Seq reads in the first and second data sets to the Mouse reference genome (mm9) in the UCSC genome browser [26] in conjunction with the RefSeq genome reference annotation (mm9) [27], map RNA-Seq reads in the third and fourth data sets to the Drosophila melanogaster reference genome (dm3) in the UCSC genome browser [26] in conjunction with the RefSeq genome reference annotation (dm3) [27], map RNA-Seq reads in the fifth data set to the Arabidopsis thaliana reference genome (ftp://ftp.arabidopsis.org/home/tair/Sequences/whole_ chromosomes/) in conjunction with the Arabidopsis thaliana genome reference annotation (ftp://ftp.arabidopsis.org/home/tair/Genes/TAIR10_genome_release/TAIR10_gff3/), and used TopHat2 [8] and Bowtie2 [9] to map RNA-Seq reads in the sixth data set to the Homo sapiens reference genomes (hg19) in the UCSC genome browser [26] in conjunction with the RefSeq genome reference annotation (hg19) [27]. Tables 49 show the mapping statistics of six sets of RNA-Seq data. Overall, more than 70% of reads were mapped to the genome successfully. Particularly, a very high mapping rate (~97%) was reached on the sixth data set. These mapping success rates were within the reasonable range, suggesting the good quality of the data and the correctness of the mapping process. This reads mapping process reduced the size of data by several orders of magnitude.

thumbnail
Table 4. Mapping statistics of the first set of RNA-Seq data of mouse.

https://doi.org/10.1371/journal.pone.0125000.t004

thumbnail
Table 5. Mapping statistics of the second set of RNA-Seq data of mouse.

https://doi.org/10.1371/journal.pone.0125000.t005

thumbnail
Table 6. Mapping statistics of the third set of RNA-Seq data of Drosophila melanogaster.

https://doi.org/10.1371/journal.pone.0125000.t006

thumbnail
Table 7. Mapping statistics of the fourth set of RNA-Seq data of Drosophila melanogaster.

https://doi.org/10.1371/journal.pone.0125000.t007

thumbnail
Table 8. Mapping statistics of the fifth set of RNA-Seq data of Arabidopsis thaliana.

https://doi.org/10.1371/journal.pone.0125000.t008

thumbnail
Table 9. Mapping statistics of the sixth set of RNA-Seq data of human.

https://doi.org/10.1371/journal.pone.0125000.t009

2. Gene expression values calculated from the reads mapping data

RNAMiner removed reads that mapped to multiple locations on a reference genome from the mapping data. The gene expression values were calculated by Cufflinks [10], MULTICOM-MAP [14,15,16], and HTSeq [11] on the remaining RNA-Seq reads mapped to unique locations on the genome. Compiling reads mappings into gene expression values generates an overall profile of the expression levels of most genes in a transcriptome, which can reduce the size of dataset by about one thousand-fold (i.e., from gigabytes to megabytes) in our experiments. The compilation process transforms the raw data into meaningful expression profiles of genes. For example, three gene expression plots for comparisons between Control and each treatment in mutant mouse in the first data set are shown in Fig 6, and two gene expression plots for comparisons between 2A and 2B, between 2A and 3D in the second dataset are shown in Fig 7. In these figures, gene expression values calculated by MULTICOM-MAP were used, and the range of these values was constrained to [0, 100] while keeping the original ratios in order to make these figures readable. Usually, the points beyond the diagonal are candidates of differentially expressed genes.

thumbnail
Fig 6. Three gene expression plots in the first data set.

These plots are for comparisons between Control and each treatment in mutant mouse. The x-axis represents Control and the y-axis represents CDDO treatment in A, Sutherlandia treatment in B, and Elderberry treatment in C. We used gene expression values calculated by MULTICOM-MAP to make the plots. The range of these values was constrained to [0, 100] while keeping the original ratios in order to make these figures readable.

https://doi.org/10.1371/journal.pone.0125000.g006

thumbnail
Fig 7. Two gene expression plots in the second data set.

These plots are for comparisons between 2A and 2B, between 2A and 3D. The x-axis represents 2A (Control, no FruHis, no Lycopene) and the y-axis represents 2B (FruHis = 1, no Lycopene) in A and 3D (FruHis = 16, with Lycopene) in B. We used gene expression values calculated by MULTICOM-MAP to make the plots. The range of these values was constrained to [0, 100] while keeping the original ratios in order to make these figures readable.

https://doi.org/10.1371/journal.pone.0125000.g007

MULTICOM-MAP and HTSeq were used to calculate the raw read counts in the third and fourth sets of data. The counts were normalized by dividing them by the total number of uniquely mapped reads in the replicate. The normalized count of a gene was an indicator of the relative expression level of the gene in the replicate. The normalized counts of a gene in multiple replicates of a sample were further averaged and used as the measure of the relative expression level of the gene in the sample. Fig 8 shows one gene expression plot for the comparison between CF (Control Female) and CM (Control Male) in the third data set. In this figure, gene expression values were calculated by MULTICOM-MAP, and the values were transformed by log2 in order to make the figure readable. Two gene expression plots for the comparison between Col (Wild-Type) and hae-3 hsl2-3 (mutant) in the fifth data set are illustrated in Fig 9, and two gene expression plots for the comparison between 2pc3 and 1Sfesrrb in the sixth data set are illustrated in Fig 10. The left plot in each figure was generated from all the genes, and the right one was generated from differentially expressed genes. The gene expression values were calculated by MULTICOM-MAP and normalized by log2. According to the two plots in Figs 9 and 10, the distribution of expression values of differentially expressed genes is quite different than that of the rest of the genes.

thumbnail
Fig 8. One gene expression plot in the third data set.

The plot is for the comparison between CF and CM. The x-axis represents CF and the y-axis represents CM. We used gene expression values calculated by MULTICOM-MAP to make the plot. The raw counts were transformed by log2 in order to make the figure readable.

https://doi.org/10.1371/journal.pone.0125000.g008

thumbnail
Fig 9. Two gene expression plots in the fifth data set.

These plots are for the comparison between Col and hae-3 hsl2-3. The x-axis represents Col (Wild-Type) and the y-axis represents hae-3 hsl2-3 (mutant). The left plot visualizes the expression values of all the genes, and the right one displays the expression values of differentially expressed genes. The gene expression values were calculated by MULTICOM-MAP. The raw counts were transformed by log2.

https://doi.org/10.1371/journal.pone.0125000.g009

thumbnail
Fig 10. Two gene expression plots in the sixth data set.

These plots are for the comparison between 2pc3 and 1Sfesrrb. The x-axis represents 2pc3 and the y-axis represents 1Sfesrrb. The left plot visualizes the expression values of all the genes, and the right one displays the expression values of differentially expressed genes. The gene expression values were calculated by MULTICOM-MAP. The raw counts were transformed by log2.

https://doi.org/10.1371/journal.pone.0125000.g010

3. Differentially expressed genes identified from the RNA-Seq data

RNAMiner identified differentially expressed genes between control and each treatment using Cuffdiff [10], edgeR [12], and DESeq [13]. The threshold of p-value was set to 0.05 to select differentially expressed genes. For example, the number of differentially expressed genes for each comparison and their overlaps in both mutant mouse and wild-type mouse in the first data set are shown in Fig 11. The number of differentially expressed genes for each comparison in the second data set is shown in Fig 12. These differentially expressed genes were derived from the overlaps of three sets of differentially expressed genes separately identified by Cuffdiff, MULTICOM-MAP+edgeR, and MULTICOM-MAP+DESeq. As shown in Fig 12, the number of differentially expressed genes increased with the increase of FruHis concentration in the absence or presence of 4 μM lycopene.

thumbnail
Fig 11. The number of differentially expressed genes in the first data set.

These numbers were calculated for different pairs of comparisons between Control and each treatment, and their overlaps in both mutant mouse and wild-type mouse cells. The differentially expressed genes in each comparison were derived from the overlaps of three sets of differentially expressed genes generated by Cuffdiff, MULTICOM-MAP+edgeR, and MULTICOM-MAP+DESeq.

https://doi.org/10.1371/journal.pone.0125000.g011

thumbnail
Fig 12. The number of differentially expressed genes for each pairwise comparison in the second data set.

The differentially expressed genes in each comparison were derived from the overlaps of three sets of differentially expressed genes generated by Cuffdiff, MULTICOM-MAP+edgeR, and MULTICOM-MAP+DESeq.

https://doi.org/10.1371/journal.pone.0125000.g012

The number of differentially expressed genes for two comparisons between Col (Wild-Type) and hae-3 hsl2-3 (mutant), between Col_qtrim (Wild-Type) and hae-3 hsl2-3_qtrim (mutant), and their overlaps in the fifth data set are shown in Fig 13. These differentially expressed genes were derived from the overlaps of three sets of differentially expressed genes generated separately by Cuffdiff, MULTICOM-MAP+edgeR, MULTICOM-MAP+DESeq. We also identified differentially expressed genes for two comparisons: between 2pc3 and 1Sfesrrb, between 4ctrl and 3DY131 in the sixth data set using edgeR based on read counts calculated by MULTICOM-MAP. EdgeR identified 6,210 differentially expressed genes for the comparison between 2pc3 and 1Sfesrrb, and 590 differentially expressed genes for the comparison between 4ctrl and 3DY131. On the RNAMiner web service, users can select different p-value (or q-value) thresholds to select a specific number of differentially expressed genes according to their needs. In addition to generating the testable biological hypotheses (e.g. gene targets for experimental testing), differential gene expression analysis generally reduces the size of data by about two folds, shifting point of interest from almost all the genes in a genome to a small portion of genes most relevant to the biological experiment.

thumbnail
Fig 13. The number of differentially expressed genes in the fifth data set.

These numbers were calculated for two comparisons between Col and hae-3 hsl2-3, between Col_qtrim and hae-3 hsl2-3_qtrim, and their overlap. The differentially expressed genes in each comparison were derived from the overlaps of three sets of differentially expressed genes generated by Cuffdiff, MULTICOM-MAP+edgeR, MULTICOM-MAP+DESeq.

https://doi.org/10.1371/journal.pone.0125000.g013

4. Predicted functions of differentially expressed genes

RNAMiner predicted functions of differentially expressed genes using MULTICOM-PDCN. The predicted function terms were ranked by their significance of enrichment among the differentially expressed genes. For example, Fig 14 shows the top 10 most significantly enriched biological process functions for the comparison between Control and CDDO in both mutant mouse and wild-type mouse in the first data set. The two comparisons have 5 common biological processes in the top 10 biological processes. The top 10 biological process functions for two comparisons between 2A and 2F (FruHis = 16 without Lycopene), between 2A and 3D (FruHis = 16 with Lycopene) in the second data set are shown in Fig 15. The two comparisons have 3 common biological processes in the top 10 biological processes. The top 10 biological process functions for the comparison between Col (Wild-Type) and hae-3 hsl2-3 (mutant) in the fifth data set are reported in Fig 16, and the two comparisons share 8 common biological processes in the top 10 biological processes. In these figures, the number besides each column is p-value of the enrichment of each predicted function.

thumbnail
Fig 14. Top 10 biological process functions in the first data set.

These functions were predicted for the comparison between Control and CDDO in both mutant mouse and wild-type mouse cells. Red bars denote the p-values of the top 10 predicted functions for wild-type mouse and blue bars denote the p-values of the top 10 predicted functions for mutant mouse. The number besides each bar is the significance of enrichment (p-value) of the predicted function. The p-value was calculated by Fisher’s exact test. The function names of wild-type mouse and mutant mouse are listed on the left separated by “/”. The two comparisons have five common biological processes among the top 10 biological processes.

https://doi.org/10.1371/journal.pone.0125000.g014

thumbnail
Fig 15. Top 10 biological process functions in the second data set.

These functions were predicted for two comparisons between 2A (Control, no FruHis, no Lycopene) and 2F (FruHis = 16 without Lycopene), between 2A and 3D (FruHis = 16 with Lycopene). Red bars denote the p-values of the top 10 predicted functions for the comparison between 2A and 3D and blue bars denote the p-values of the top 10 predicted functions for the comparison between 2A and 2F. The number besides each bar is the significance of enrichment (p-value) of the predicted function. The p-value was calculated by Fisher’s exact test. The function names of the two comparisons between 2A and 3D, between 2A and 2F are listed on the left separated by “/”. The two comparisons have three common biological processes among the top 10 biological processes.

https://doi.org/10.1371/journal.pone.0125000.g015

thumbnail
Fig 16. Top 10 biological process functions in the fifth data set.

These functions were predicted for two comparisons between Col and hae-3 hsl2-3, between Col_qtrim and hae-3 hsl2-3_qtrim. Red bars denote the p-values of the top 10 predicted functions for the comparison between Col_qtrim and hae-3 hsl2-3_qtrim and blue bars denote the p-values of the top 10 predicted functions for the comparison between Col and hae-3 hsl2-3. The number besides each bar is the significance of enrichment (p-value) of the predicted function. The p-value was calculated by Fisher’s exact test. The function names of the two comparisons between Col_qtrim and hae-3 hsl2-3_qtrim, between Col and hae-3 hsl2-3 are listed on the left separated by “/”. The two comparisons have 8 common biological processes among the top 10 biological processes.

https://doi.org/10.1371/journal.pone.0125000.g016

Although the step of gene function analysis does not substantially reduce the size of data physically, it can logically summarize hundreds of differentially expressed genes into a small number (i.e., tens) of biological processes activated or deactivated in the biological experiment which sheds light into the potential biological mechanism relevant to the experiment.

5. Constructed gene regulatory networks

RNAMiner used MULTICOM-GNET [19,20] to construct gene regulatory networks based on differentially expressed genes and transcription factors. For example, a repression gene regulatory module with expression correlation 0.85 in mutant mouse in the first data set is illustrated in Fig 17. This module was comprised of 21 differentially expressed genes. Three transcription factors: Tgfb1i1, Htatip2, and Jun, were predicted to collaboratively regulate this group of genes. An activation gene regulatory module for the comparison between Col (Wild-Type) and hae-3 hsl2-3 (mutant) with expression correlation 0.85 in the fifth data set is shown in Fig 18. This module was comprised of 35 differentially expressed genes. Four transcription factors, AT3G59580, AT1G56650, AT1G28050, and AT1G52890, were predicted to collaboratively regulate this group of genes.

thumbnail
Fig 17. One repression gene regulatory module in mutant mouse cells in the first data set.

The expression correlation score of the module was 0.85. The decision tree on the middle top illustrates how three putative transcription factors (Tgfb1i1, Htatip2, Jun) may collaboratively regulate the cluster of co-expressed genes in the middle bottom, where each row denotes a gene listed in the bottom left box and each column denotes one of four biological conditions (i.e. Control, CDDO, Sutherlandia, and Elderberry). The levels of gene expression values were represented by different colors ranging from lowest (green) to highest (red). The expression of the genes in the cluster under each condition is predicted to be regulated according to the expression levels of transcription factors listed on top of the condition. For example, under Sutherlandia treatment, the relatively low expression of Tgfbli1 and the medium expression of Jun caused the repression of the group of genes.

https://doi.org/10.1371/journal.pone.0125000.g017

thumbnail
Fig 18. One activation gene regulatory module in the fifth data set.

The gene regulatory module was constructed for the comparison between Col (Wild-Type) and hae-3 hsl2-3 (mutant), and the expression correlation score of the module was 0.85. The decision tree on the middle top illustrates how four putative transcription factors (AT3G59580, AT1G28050, AT1G52890, AT1G56650) may collaboratively regulate the cluster of co-expressed genes in the middle bottom, where each row denotes a gene listed in the bottom left box and each column denotes one of six biological replicates of two samples (i.e. Col and hae-3 hsl2-3). The levels of gene expression values were represented by different colors ranging from lowest (green) to highest (red). The expression of the genes in the cluster under each sample is predicted to be regulated according to the expression levels of transcription factors listed on top of the condition. For example, under the first replicate of Col, the low expression of AT3G59580 and the low expression of AT1G28050 caused the repression of the group of genes.

https://doi.org/10.1371/journal.pone.0125000.g018

RNAMiner also used a R package “igraph” [43] to visualize gene regulatory networks by linking the regulatory relationships between and within all the gene regulatory modules predicted by MULTICOM-GNET together. Fig 19 shows a gene regulatory network representing the regulatory relationships of top 10 gene regulatory modules ranked by expression correlation scores on the first data set. There are 14 transcription factors (red nodes), 338 genes (blue nodes), and 1,280 edges (regulatory relationships) in the network.

thumbnail
Fig 19. A visualized global gene regulatory network on the first dataset.

The network includes all the gene regulatory relationships between and within top 10 gene regulatory modules ranked by expression correlation scores on the first dataset. Blue nodes represent target genes, and red nodes represent transcription factors which regulate the target genes. Each edge represents a regulatory relationship between a transcription factor and a gene.

https://doi.org/10.1371/journal.pone.0125000.g019

The step of gene regulatory network reconstruction condenses hundreds of differentially expressed genes and their expression data into dozens of valuable gene regulatory modules, which may reveal the underlying biological mechanism controlling the expression in the biological experiment. The network modules not only provide the human comprehensible interpretation of the gene expression levels, but also the important transcription factors and their target genes that are very valuable for generating hypotheses for new biological experiments.

Use of the RNAMiner Web Service

The RNAMiner web service (Fig 20) is available at http://calla.rnet.missouri.edu/rnaminer/index.html. Users can submit requests on the home page and receive an email with a link to the data analysis results.

thumbnail
Fig 20. The home page of the RNAMiner web service.

Users can submit requests on the home page of the RNAMiner web service, and also can learn how to use RNAMiner, find contact information, and download the test data by clicking the navigation buttons on left.

https://doi.org/10.1371/journal.pone.0125000.g020

1. Submit a request

  1. Prepare RNA-Seq reads files (.fastq). The acceptable formats by the RNAMiner web service include “.fastq.gz” and “.fastq.tar.gz”.
  2. Choose the analysis categories. Each category needs the results in the previous categories. If one category is chosen, the previous categories will be executed automatically. For example, if “predicting gene functions” is chosen, the first three categories will be executed automatically.
  3. Choose the species. RNAMiner can analyze RNA-Seq data on four species: Human, Mouse, Drosophila melanogaster, and Arabidopsis thaliana.
  4. Choose criterion of identifying differentially expressed genes. It is p-value or q-value.
  5. Set threshold of p-value or q-value for identifying differentially expressed genes. The value should be between 0 and 1. The default value is 0.05.
  6. Input email address. An email with a link to the data analysis results will be sent to this email address when the data analysis is finished.
  7. Input sample names.
  8. Upload reads files. The last three categories request users to upload reads files for both two samples. Users can upload more than one reads files for each sample.
  9. Click “Submit”.

After a request is submitted successfully, one web page (Fig 21) will be shown saying the data is in process. If one user submitted one request to the RNAMiner web service and it is running or it is in the waiting queue, he/she cannot submit another request.

thumbnail
Fig 21. One web page showing the successful submission of one request.

After one request is submitted successfully, one web page will be shown which informs users that the data is in process.

https://doi.org/10.1371/journal.pone.0125000.g021

2. Receive the results

When the data analysis is finished, users will receive an email with a link to one web page (Fig 22) with the data analysis information and a result link. The result page (Fig 23) will be shown by clicking the result link. Users can view and download the analysis data on the result page.

thumbnail
Fig 22. One web page with data analysis information and a result link.

After the data analysis is finished, users will receive an email with a link to one web page with data analysis information and a result link. On this page, users can check the data analysis information and go to the result page.

https://doi.org/10.1371/journal.pone.0125000.g022

thumbnail
Fig 23. One web page with data analysis results.

Users can view and download the data analysis results for each analysis category on this web page.

https://doi.org/10.1371/journal.pone.0125000.g023

The time expense of analyzing a set of RNA-Seq data by RNAMiner depends on how big the data is, how many reads files there are in the data set, and how many jobs there are in the waiting queue. Normally a data analysis can be finished by RNAMiner in several hours. However, the time expense will be longer if there are a lot of jobs in the waiting queue. Our server cannot handle too many jobs at the same time because of CPU and space limitations.

Conclusions

The RNAMiner protocol and pipeline can progressively reduce the size of large datasets to produce valuable and comprehensible biological knowledge of manageable size, ranging from gene expression values, differentially expressed genes, gene function predictions, and gene regulatory networks. The test results on six RNA-Seq datasets of four different species help demonstrate its utility and versatility.

In order to further improve the quality of RNA-Seq data analysis, additional tools can be plugged into the RNAMiner protocol. In the future, we will add a high-speed RNA mapping tool—Gsnap [25] and a high-accuracy RNA mapping tool—Stampy [45] into the pipeline to map RNA reads to reference genomes. For identifying differentially expressed genes, we will include baySeq [46], ShrinkSeq [47], and NOISeq [48] into the pipeline in order to handle various sources of noise in RNA-Seq data even better. Furthermore, we will include an in-house tool of constructing biological networks from a group of co-expressed genes to reconstruct highly valuable metabolic networks and signal transduction networks for gene clusters identified by the RNAMiner protocol. Moreover, we will add the capability of analyzing the function of non-coding small RNAs into RNAMiner and use the information during the reconstruction of biological networks. The new improvements will be incorporated into the RNAMiner web service for the community to use.

Acknowledgments

We would like to thank Kishore Banala for helping develop the web service.

Author Contributions

Conceived and designed the experiments: JC JL. Performed the experiments: JC JL. Analyzed the data: JC JL. Contributed reagents/materials/analysis tools: JL LS JMW YL CEN BRM TPM VVM CMG JCW WRF MH DBL JAB JC. Wrote the paper: JL JC. Web service construction: JL JH.

References

  1. 1. Fang Z, Martin J, Wang Z (2012) Statistical methods for identifying differentially expressed genes in RNA-Seq experiments. Cell & Bioscience 2: 26.
  2. 2. Soneson C, Delorenzi M (2013) A comparison of methods for differential expression analysis of RNA-seq data. BMC bioinformatics 14: 91. pmid:23497356
  3. 3. Mortazavi A, Williams BA, McCue K, Schaeffer L, Wold B (2008) Mapping and quantifying mammalian transcriptomes by RNA-Seq. Nature methods 5: 621–628. pmid:18516045
  4. 4. Chen G, Wang C, Shi T (2011) Overview of available methods for diverse RNA-Seq data analyses. Science China Life Sciences 54: 1121–1128. pmid:22227904
  5. 5. Oshlack A, Robinson MD, Young MD (2010) From RNA-seq reads to differential expression results. Genome Biol 11: 220. pmid:21176179
  6. 6. Wang L, Li P, Brutnell TP (2010) Exploring plant transcriptomes using ultra high-throughput sequencing. Briefings in Functional Genomics 9: 118–128. pmid:20130067
  7. 7. Kvam VM, Liu P, Si Y (2012) A comparison of statistical methods for detecting differentially expressed genes from RNA-seq data. American journal of botany 99: 248–256. pmid:22268221
  8. 8. Kim D, Pertea G, Trapnell C, Pimentel H, Kelley R, Salzberg SL (2013) TopHat2: accurate alignment of transcriptomes in the presence of insertions, deletions and gene fusions. Genome Biol 14: R36. pmid:23618408
  9. 9. Langmead B, Salzberg SL (2012) Fast gapped-read alignment with Bowtie 2. Nature methods 9: 357–359. pmid:22388286
  10. 10. Trapnell C, Williams BA, Pertea G, Mortazavi A, Kwan G, van Baren MJ, et al. (2010) Transcript assembly and quantification by RNA-Seq reveals unannotated transcripts and isoform switching during cell differentiation. Nature biotechnology 28: 511–515. pmid:20436464
  11. 11. Anders S (2010) HTSeq: Analysing high-throughput sequencing data with Python. http://www-huber.embl.de/users/anders/HTSeq/doc/overview.html.
  12. 12. Robertson G, Schein J, Chiu R, Corbett R, Field M, Jackman SD, et al. (2010) De novo assembly and analysis of RNA-seq data. Nature methods 7: 909–912. pmid:20935650
  13. 13. Anders S, Huber W (2010) Differential expression analysis for sequence count data. Genome Biology 11: R106. pmid:20979621
  14. 14. Sun L, Fernandez HR, Donohue RC, Li J, Cheng J, Birchler JA (2013) Male-specific lethal complex in Drosophila counteracts histone acetylation and does not mediate dosage compensation. Proceedings of the National Academy of Sciences 110: E808–E817. pmid:23382189
  15. 15. Sun L, Johnson AF, Donohue RC, Li J, Cheng J, Birchler JA (2013) Dosage compensation and inverse effects in triple X metafemales of Drosophila. Proceedings of the National Academy of Sciences 110: 7383–7388. pmid:23589863
  16. 16. Sun L, Johnson AF, Li J, Lambdin AS, Cheng J, Birchler JA (2013) Differential effect of aneuploidy on the X chromosome and genes with sex-biased expression in Drosophila. Proceedings of the National Academy of Sciences 110: 16514–16519. pmid:24062456
  17. 17. Wang Z, Cao R, Cheng J (2013) Three-level prediction of protein function by combining profile-sequence search, profile-profile search, and domain co-occurrence networks. BMC bioinformatics 14: S3. pmid:24564553
  18. 18. Wang Z, Zhang XC, Le MH, Xu D, Stacey G, Cheng J (2011) A protein domain co-occurrence network approach for predicting protein function and inferring species phylogeny. PloS one 6: e17906. pmid:21455299
  19. 19. Zhu M, Deng X, Joshi T, Xu D, Stacey G, Cheng J (2012) Reconstructing differentially co-expressed gene modules and regulatory networks of soybean cells. BMC genomics 13: 437. pmid:22938179
  20. 20. Zhu M, Dahmen JL, Stacey G, Cheng J (2013) Predicting gene regulatory networks of soybean nodulation from RNA-Seq transcriptome data. BMC bioinformatics 14: 278. pmid:24053776
  21. 21. Giardine B, Riemer C, Hardison RC, Burhans R, Elnitski L, Shah P, et al. (2005) Galaxy: a platform for interactive large-scale genome analysis. Genome research 15: 1451–1455. pmid:16169926
  22. 22. Department of Energy Systems Biology Knowledgebase (KBase). http://kbase.us.
  23. 23. Goff SA, Vaughn M, McKay S, Lyons E, Stapleton AE, Gessler D, et al. (2011) The iPlant Collaborative: Cyberinfrastructure for Plant Biology. Frontiers in plant science 2:
  24. 24. Trapnell C, Pachter L, Salzberg SL (2009) TopHat: discovering splice junctions with RNA-Seq. Bioinformatics 25: 1105–1111. pmid:19289445
  25. 25. Wu TD, Nacu S (2010) Fast and SNP-tolerant detection of complex variants and splicing in short reads. Bioinformatics 26: 873–881. pmid:20147302
  26. 26. Karolchik D, Baertsch R, Diekhans M, Furey TS, Hinrichs A, Lu YT, et al. (2003) The UCSC genome browser database. Nucleic acids research 31: 51–54. pmid:12519945
  27. 27. Pruitt KD, Tatusova T, Maglott DR (2007) NCBI reference sequences (RefSeq): a curated non-redundant sequence database of genomes, transcripts and proteins. Nucleic acids research 35: D61–D65. pmid:17130148
  28. 28. Li H, Handsaker B, Wysoker A, Fennell T, Ruan J, Homer N, et al. (2009) The sequence alignment/map format and SAMtools. Bioinformatics 25: 2078–2079. pmid:19505943
  29. 29. Radivojac P, Clark WT, Oron TR, Schnoes AM, Wittkop T, Sokolov A, et al. (2013) A large-scale evaluation of computational protein function prediction. Nature methods 10: 221–227. pmid:23353650
  30. 30. Altschul SF, Madden TL, Schäffer AA, Zhang J, Zhang Z, Miller W, et al. (1997) Gapped BLAST and PSI-BLAST: a new generation of protein database search programs. Nucleic acids research 25: 3389–3402. pmid:9254694
  31. 31. Soding J, Biegert A, Lupas A (2005) The HHpred interactive server for protein homology detection and structure prediction. Nucleic Acids Research 33: W244–W248. pmid:15980461
  32. 32. Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, et al. (2000) Gene ontology: tool for the unification of biology. Nature Genetics 25: 25–29. pmid:10802651
  33. 33. Boeckmann B, Bairoch A, Apweiler R, Blatter MC, Estreicher A, Gasteiger E, et al. (2003) The SWISS-PROT protein knowledgebase and its supplement TrEMBL in 2003. Nucleic Acids Research 31: 365–370. pmid:12520024
  34. 34. Bateman A, Coin L, Durbin R, Finn RD, Hollich V, Griffiths-Jones S, et al. (2004) The Pfam protein families database. Nucleic Acids Research 32: 276–280.
  35. 35. Agresti A (1990) Categorical data analysis. New York: Wiley. Pages 59–66.
  36. 36. Agresti A (2002) Categorical data analysis. Second edition. New York: Wiley. Pages 91–101.
  37. 37. Fisher RA (1935) The logic of inductive inference. Journal of the Royal Statistical Society Series A 98: 39–54.
  38. 38. Fisher RA (1962) Confidence limits for a cross-product ratio. Australian Journal of Statistics 4: 41.
  39. 39. Fisher RA (1970) Statistical Methods for Research Workers. Oliver & Boyd.
  40. 40. Mehta CR, Patel NR (1986) Algorithm 643: FEXACT: a Fortran subroutine for Fisher's exact test on unordered r*c contingency tables. ACM Transactions on Mathematical Software (TOMS) 12: 154–161.
  41. 41. Clarkson DB, Fan Y, Joe H (1993) A Remark on Algorithm 643: FEXACT: An Algorithm for Performing Fisher's Exact Test in r x c Contingency Tables. ACM Transactions on Mathematical Software (TOMS) 19: 484–488.
  42. 42. Patefield WM (1981) Algorithm AS159. An efficient method of generating r x c tables with given row and column totals. Applied Statistics 30: 91–97.
  43. 43. Csardi G, Nepusz T (2006) The igraph software package for complex network research. InterJournal Complex Systems: 1695.
  44. 44. Langmead B, Trapnell C, Pop M, Salzberg SL (2009) Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biol 10: R25. pmid:19261174
  45. 45. Lunter G, Goodson M (2011) Stampy: a statistical algorithm for sensitive and fast mapping of Illumina sequence reads. Genome research 21: 936–939. pmid:20980556
  46. 46. Hardcastle TJ, Kelly KA (2010) baySeq: empirical Bayesian methods for identifying differential expression in sequence count data. BMC bioinformatics 11: 422. pmid:20698981
  47. 47. Van De Wiel MA, Leday GG, Pardo L, Rue H, Van Der Vaart AW, Van Wieringen WN (2013) Bayesian analysis of RNA sequencing data by estimating multiple shrinkage priors. Biostatistics 14: 113–128. pmid:22988280
  48. 48. Tarazona S, García-Alcalde F, Dopazo J, Ferrer A, Conesa A (2011) Differential expression in RNA-seq: a matter of depth. Genome research 21: 2213–2223. pmid:21903743
  49. 49. Foster BA, Gingrich JR, Kwon ED, Madias C, Greenberg NM (1997) Characterization of prostatic epithelial cell lines derived from transgenic adenocarcinoma of the mouse prostate (TRAMP) model. Cancer research 57: 3325–3330. pmid:9269988
  50. 50. Mossine VV, Mawhinney TP (2007) Nα-(1-Deoxy-D-fructos-1-yl)-L-histidine (“D-fructose-L-histidine”): a potent copper chelator from tomato powder. Journal of agricultural and food chemistry 55: 10373–10381. pmid:18004802
  51. 51. Niederhuth CE, Patharkar OR, Walker JC (2013) Transcriptional profiling of the Arabidopsis abscission mutant hae hsl2 by RNA-Seq. BMC genomics 14: 37. pmid:23327667