Intrinsic gene characteristics suffice to predict publications

To test if measurable intrinsic chemical, physical, and biological features of genes and gene products alone suffice to describe the number of publications per gene, we gathered 430 features per gene, which could either be computed from known sequences of these genes or obtained from previously published genome-scale experiments (Fig 1A). Intriguingly, we observed that 33% of the protein-coding genes carrying an official gene name had an incomplete catalog of features. The dominant reasons for the absence of features were the absence of reported insertions within recent Clustered Regularly Interspaced Short Palindromic Repeats (CRISPR) loss-of-function screens (about 13% of genes, depending on assay), the absence of detectable RNA across all tissues and cell lines surveyed by the human protein atlas (6% of genes), the absence of validated RNA molecules within the Genbank reference database of RNA molecules (5% of genes), and the absence of reported protein molecules within the UniProt reference database for protein molecules (3% of genes) (S2A Fig, S1 Table). Foreshadowing our subsequent analyses, the absence of reported features correlated with a lower number of reported publications (S2A Fig). This initial result illustrates limitations in experimental approaches and a surprising degree of uncertainty that remains about human genes and the existence of their gene products.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Physical, chemical, and biological features of genes predict the number of publications.

(A) Illustration of modeling approach and prediction of number of research publications for single genes using information on 430 physical, chemical, and biological features of genes (S1 Data). (B) Research publications on individual genes grouped by t-SNE visualization using the 15 features most important to the models used in (A). Heatmaps show z-scored values of the 15 features for the genes in each cluster. Order of features as indicated in S3A Fig (S1 Data). SRP, signal recognition particle; t-SNE, t-distributed stochastic neighbor embedding.

https://doi.org/10.1371/journal.pbio.2006643.g001

To assess whether the values of these features, rather than solely their presence, would quantitatively inform on the number of publications of individual genes, we proceeded by only considering the 12,948 genes with a complete set of features (S2 Table). Using gradient boosting regressions with out-of-sample Monte Carlo cross-validation [25], we could predict to a significant extent the number of publications on any given gene (Fig 1A, Spearman: 0.64). Remarkably, 15 out of 430 features contributed the most to our model’s accuracy (S3A Fig) and fell into six categories that specify the abundance of gene-encoded RNA and protein molecules across multiple tissues (RNA abundance in adrenal glands, appendix, brain, and liver; fraction of tissues with detectable RNA expression; and protein abundance in HeLa cells), the positive charge of proteins, the hydrophobicity of proteins, the sensitivity of genes towards mutations (incidence rate of missense mutations in human populations, incidence rate of loss-of-function mutations in human populations, tolerance against homozygous or recessive loss-of-function variation in human populations, CRISPR score in KBM7 cells), the length of the corresponding transcript and gene, and the presence of signal sequences that promote the translocation of nascent proteins into the endoplasmic reticulum. These 15 features are sufficient to account for the model’s accuracy because models using exclusively those features yields prediction accuracies highly comparable to those of the full model when trained on the same 12,948 genes with a complete catalog of features (Spearman: 0.61, S3B Fig), or on all 15,056 genes on which these 15 features are defined (Spearman: 0.59, S3C Fig).

We therefore used these 15 features to define a 15-dimensional space for the 15,056 genes that reflects the correlation between publications and individual features and combinations of distinct features (S3 Table). Clusters of genes within this space were enriched for distinct Gene Ontology annotations and thus known biological roles (Fig 1B, S4 Fig). This initial finding demonstrates that the number of publications on genes can be attributed in a large extent solely to a small set of their physical, chemical, and biological characteristics.

Past research priorities strongly impact current initiatives

The 15 features described above have all been suspected to affect the ability to study specific genes by traditional methodologies [23, 26–28]. Prompted by this fact and ample sociological observations on science, that the “rich” can get “richer” [9, 29], we next detailed the consensus between the overall number of publications per gene and past research. In line with the similarity among prior reports on the disparity in the number of publications per gene, we found that the present inequality in the number of publications has stayed constant since the year 2000 (S5A and S5B Fig). Similarly, we found the number of publications per gene to be highly correlated between the current decade and preceding time periods of research (Fig 2A, Spearman: 0.84). Interestingly, we also identify six genes that are presently experiencing a strong increase in their number of publications, which can be traced back to a recent acknowledgment of their medical importance (S4 Table).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Features of genes and homologous genes predict discovery of human genes.

(A) Number of publications per gene for past and recent research. Publications of past research (until 2010) are scaled so that the total number of publications matches present research (2011–2015). Dashed grey lines delimit three standard deviations away from the mean. (B) Prediction of the number of research publications for the model of Fig 1A extended by including the year of the first publication on the specific human gene (S1 Data). (C) Prediction of the year of discovery using the features from Fig 1A (S1 Data). (D) Percentage of publications that cite publications with nonhuman genes more frequently than they cite publications with human genes (S1 Data). (E) Prediction of the year of initial publications on individual genes using the features from Fig 1A and the year of the initial publication on homologous genes of nonhuman model organisms (S1 Data). (F) Prediction of the number of research publications using the features of Fig 1A and the number of publications on homologous genes (S1 Data).

https://doi.org/10.1371/journal.pbio.2006643.g002

In contrast to the alternative hypothesis that research patterns on human genes would be particularly dynamic [1, 2], and generalizing beyond earlier studies on two gene families [6, 21] and genes expressed specifically in the brain [30], we find that human genes that had been reported early—as indicated by an early initial publication date on the genes or their encoded gene products [19, 31]—tend to also be more studied presently (S5C Fig, Spearman: 0.58). For example, all genes that had been reported upon by 1991 (corresponding to 16% of all genes) account for 49% of the literature of the year 2015 (S5D Fig). Initial reports further add to the predictability of the number of publications as an inclusion of their year improves the models’ accuracy (Fig 2B, Spearman: 0.75). To identify the factors associated with the initial reports of genes, we next created separate models with the above 430 features and trained them to predict the year of initial publications. While these predictions are slightly less accurate (Fig 2C, Spearman: 0.48) than predictions on the number of publications, the underlying models again selected for highly similar features—most prominently, the presence of signal peptides, the abundance of transcript and protein molecules, and the sensitivity towards mutations (S5E Fig, S5 Table). This shows that characteristics of genes, which have been important for the initial discovery of genes, remain partially correlated with the number of present publications on those genes.

Similarly, we observe that while the number of publications is correlated between the first entry (e.g., AKT1) and the second entry (e.g., AKT2) of a gene family (S5F Fig, Spearman: 0.69), first entries have more publications (Mann–Whitney U test: p-value < 10⁻²⁴). This demonstrates that even among evolutionary and chemically highly related genes, early initial reports coincide with a higher number of publications (S5F Fig).

Knowledge from model organisms drives research on human genes

Yet, the reduced prediction accuracy observed for the prediction of the year of the initial report may hint at the presence of another factor or factors that were not included in our curation of 430 gene-intrinsic features. Thus, we performed a bibliometric analysis of PubMed to compare individual publications against the genes contained in the publications that they cite. Focusing on the publications reporting the discovery of new human genes, we found an overrepresentation of publications that cite studies of nonhuman genes (Figs 2D and S6A). Inspecting the organisms of these genes, we observed two classes of organisms. The first class preferentially co-occurred together with human genes and consisted of Mus musculus, Rattus norvegicus, Bos taurus, and Gallus gallus (37%, 9.1%, 2.6%, 2.5% of all citations, respectively). The second class preferentially occurred in publications without human genes and consisted of Drosophila melanogaster, Saccharomyces cerevisiae, Escherichia coli, Xenopus laevis, Caenorhabditis elegans, and Schizosaccharomyces pombe (22%, 10%, 4.0%, 2.5%, 1.6%, 1.5% of all citations, respectively) (S6B Fig). Assuming that citations are one proxy of scientific impact, this finding suggests that initial reports on human genes have been particularly influenced by research in model organisms and that multiple model organisms have contributed complementary roles in the discovery of human genes.

With these insights, we dramatically increased the prediction accuracy of the year of initial report of a human gene by including the years of the initial reports on homologous genes of model organisms (Fig 2E, from Spearman: 0.48 to 0.71). Moreover, the years of the initial reports on homologous genes improved prediction accuracy of the number of publications to a greater extent than the year of the initial report on the human genes themselves (S7A Fig, Spearman: 0.81).

Consistent with the picture emerging from these analyses, the homologous genes of unstudied human genes are likewise unstudied in model organisms (S6 Table), and including the number of publications on homologous genes yielded almost perfect predictions of the number of publications for individual human genes (Fig 2F, Spearman: 0.87), while human-specific genes without homologous genes remain significantly less studied (S7B Fig, Mann–Whitney U test: p-value < 10⁻³²). Taken together, these findings demonstrate the impact of research on model organisms on the knowledge acquired on human biology—a hypothesis that had been proposed but not demonstrated previously [32].

Characteristics of genes affect research on important biology

Given the observed historic continuity of scientific endeavors, we wondered whether biomedical research has already identified all particularly important human genes and hence allocates the production of publications accordingly. We follow the naïve assumption that researchers distribute their attention equally across all genes contained in the same publication (S8 Fig). Despite this simplifying assumption, we reassuringly observe that genes that have received the most attention in publications are around three to five times more likely to be sensitive to loss-of-function mutations or to have been identified in genome-wide association studies (GWAS) (Fig 3A). This enrichment is greatest for genes that have been repeatedly identified by several independent studies on the most frequently studied human phenotypic traits. However, we observe an extraordinarily more extreme 13-fold enrichment in the average attention when comparing the genes that have received the least attention to those genes that have received the highest attention (Fig 3A). Hence, while biomedical research does focus on important genes, a disproportionally high amount of research effort concentrates on already well-studied genes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Many potentially important genes are not being studied enough.

(A) Relative enrichment of the presence of genes with genetic loss-of-function (LoF) intolerance, presence of genes with GWAS traits, and the attention within publications. (B) Predicted versus actual NIH budget spending on individual genes (dots). The black line shows a lowess fit and the dashed lines show the two distinct regimes of the prediction (S1 Data). (C) Fraction of disease-linked genes with at least one experimental drug conditioned on the predicted order of discovery according to the model shown in Fig 2B. Error bars show 95% confidence intervals for the estimations. GWAS, genome-wide association study; LoF, loss-of-function; NIH, National Institutes of Health; USD, US dollar.

https://doi.org/10.1371/journal.pbio.2006643.g003

We observe a similar pattern when inspecting the allocation of funding by the National Institutes of Health (NIH) as another proxy of importance. Although not surprising given the correlation between the number of publications per gene and the amount of funding allocation by the NIH (S9A and S9B Fig, Spearman: 0.95), the above modeling strategy accurately predicts the allocation of billions of research dollars (Fig 3B, Spearman 0.70), and would do so particularly well for genes supported by multiple grants (S9C Fig). Yet, prediction accuracy only marginally improves by additionally considering 3,176 features detailing known annotations between genes and diseases (S9D Fig, Spearman: 0.73), and is greatly—but not completely—impaired if only considering the latter (S9E Fig, Spearman 0.43). This shows that the previously uncovered intrinsic characteristics of genes and the year of the initial report of homologous genes not only correlate with research funding, but that they would do so to a larger extent than presently existent knowledge about the role of genes in disease.

Along the same lines, if exclusively considering genes with a reported role in disease, we found that the same models that had predicted the year of the initial publication of genes (Fig 2E) also predicted the likelihood of the existence of both approved and preclinical drugs (Fig 3C, S9F Fig).

Collectively, these findings show that a small number of characteristics of genes and the availability of model organisms exert a strong influence on basic and applied research on human disease and that the resulting research can significantly deviate from the actual biological importance of individual genes.

Feasibility of alternative discovery strategies

The strong correlations uncovered, and earlier work on the availability of reagents [5, 6, 21] suggest, that researchers may face very practical constraints that prevent them from exploring little-studied genes and that there might be a need for alternative discovery strategies [33]. In support of this possibility and extending beyond the above findings on the bulk of accrued knowledge, we observe that the fraction of genes that have been described in focused single-gene studies has only been increasing at a constant rate (Fig 4A). Extrapolating from this trend, we estimate that it would take at least five decades until all genes are sufficiently studied. Similarly, simply studying little-studied genes might not be very informative and could expose junior scientists at an increased career risk (S10A Fig). Along the same lines, grant categories of the NIH dedicated to exploratory research, which do not require preliminary data, and grants categories dedicated to innovative research or the training of scientists all closely reproduce the imbalance observed for the biomedical literature, with 5% of the human protein-coding genes accounting for half of the publications (S10B and S10C Fig). Given a recent bibliometric study, which demonstrated that novelty could, however, be beneficial for the impact of a scientific publication if combined with an established research context [34], we therefore thought to build a resource that provides a context for the exploration of little-studied genes.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Identifying and exploring ignored genes.

(A) Estimation of the years until all genes are studied if scientific enterprise continues to follow trends reported above. Number of genes with at least n focused (single-gene) publications per year. Dashed lines show extrapolation of the bounds of linear regression for recent years. (B) Percentage of highly cited studies (top 5% in number of citations) in the 8 years following their publication. Error bars show 95% confidence intervals. (C) Percentage of genes with a strong RNAi phenotype, at least one tissue with moderate RNA abundance, presence of a Drosophila melanogaster homolog, or membership in a complex with highly studied genes. Highly studied genes show higher percentages for all these characteristics, but many unstudied genes also share those characteristics. (D) Illustration of bias in identification of hits in distinct large-scale experimental approaches. Interaction studies refer to studies labelled as “High throughput” within BioGRID. Relative hits marks fold enrichment over equal occurrence (S1 Data). (E) Genes grouped by t-SNE visualization using the 15 features most important to the models used in Fig 1A. Large circles highlight genes with frequently discovered GWAS traits. Heatmaps show presence of strong genetic evidence (G), experimental potential (E), and homolog in invertebrate model organism (M). Note the lack of a strong correlation between GEM characteristics and research attention. E, experimental potential; FPKM, fragments per kilobase of transcript per million mapped reads; G, strong genetic support; GEM, strong genetic support and experimental potential and homolog in invertebrate model organism; GWAS, genome-wide association study; M, model organism; RNAi, RNA interference; t-SNE, t-distributed stochastic neighbor embedding.

https://doi.org/10.1371/journal.pbio.2006643.g004

Inspecting the properties of existing publications on little-studied genes, we found that these genes tend to occur in large-scale investigations that include most genes (S11A and S11B Fig). Hinting at an ability of large-scale studies to support research on less investigated genes, we observed that these studies serve as a frequent reference for other publications (Fig 4B, S11C Fig) and that single-gene studies that refer to them tend to focus on genes that are less studied than those genes contained in single-gene studies that refer to single-gene studies (S11D Fig).

To determine the extent to which large-scale collections of biological information could already serve as potential starting points for detailed characterizations on most genes, we next extended our resource with databases—such as a collection of public RNA interference (RNAi) experiments [35], a catalog of human protein complexes [36], and a catalog of public differential gene expression experiments [37]—that could potentially be affected by biased experimental choices. We find that the 27% of genes that have never been studied by a full publication (S12A Fig) are less frequently identified in publicly available data of large-scale experiments and that they are less likely to have characteristics associated with a high number of publications (Fig 4C, S12B Fig).

However, we also find that there already exist gene-specific data on possible experimentation for 83% of them and that for 25% of them, there exist at least three qualitatively distinct types of data (S12C Fig). This strongly suggests that the characteristics of genes and homologous genes that prevented their early discovery would no longer prevent their more detailed study.

To facilitate exploration and hypothesis generation, we provide a curated guide that specifically directs to the appropriate sources of gene-specific preliminary data (S7 Table).

Our analysis further shows that distinct large-scale approaches cover distinct areas of the 15-dimensional space, with genes identified in high-throughput interaction studies being strongly enriched in regions containing abundantly expressed genes [23], and genes identified through differential expression studies being enriched in regions containing genes whose transcripts are ubiquitously detected in adult tissues through current technology. In contrast, genes identified through their phenotypes within loss-of-function RNAi screens cover the 15-dimensional space more evenly (Fig 4D). Similarly, genes with a highly reproduced association to genetic traits cover multiple areas of the 15-dimensional space, some predicting a large number of publications and others predicting a small number of publications (Fig 4E, S4 Fig). For illustration, consider the RNA of the heavily studied gene, TERT, the catalytic subunit of telomerase, which is undetectable in most adult tissues. While our analysis shows that this biological characteristic is generally associated with a low number of publications, the absence of TERT restricts excess cell proliferation [38]—a factor that overcomes the difficulty in its study following its ectopic activation. Another interesting illustration is provided by the poorly studied breast cancer gene CCDC170, which encodes for one of the most charged and acidic human proteins but also appears to have some structural role in maintaining the organization of Golgi-associated microtubules [39]. As a final illustration, consider C1orf106, a gene with the second-strongest genetic association to ulcerative colitis. Despite being among the top 20% of genes with the most frequently identified associations in differential gene expression experiments (S7 Table), C1orf106 had never been followed up until recently, when gene-specific pull-down experiments revealed its role in the regulation of the stability of epithelial adherens junctions [40]. This demonstrates that functional studies remain a powerful strategy to discover novel biology that does not reproduce past research biases.

To provide a broader perspective on the strategic options for further exploration, we next introduced aggregate measures for the presence of genetic support and experimental approachability and the existence of homologous genes in invertebrate model organisms. While some of the initially identified clusters (Fig 1B) seem experimentally well accessible in humans or model organisms, other clusters seem resilient to those approaches (Fig 4E). An opposite example is a cluster enriching for transcriptional coactivator activity. It contains several evolutionarily conserved genes that are highly sensitive towards loss-of-function mutations and experimentally approachable. This cluster contains multiple highly studied modulators of cellular physiology, such as the genes MTOR, CLTC, TAF1, and CREBBP. However, this cluster also contains DICER1, which catalyzes the maturation of microRNAs and is a recent recipient of research attention, and whose discovery was perceived as an enormous surprise following a long-held lack of attention towards non–protein-mediated gene regulation [41]. Intriguingly, this cluster includes two still mostly uncharacterized members of large gene families, IPO9 and ANKRD52. This lack of attention illustrates that even genes with seemingly promising characteristics can remain mostly ignored. To facilitate identification of such genes, we are also providing a list of these genes (S8 Table) and a map that identifies them within the vicinity of custom sets of genes (S9 Table). We further add another map that allows probing custom sets of genes for the above aggregate measures (S10 Table).

Data sources

Linkage of genes to publications was obtained from NCBI NIH (https://ftp.ncbi.nlm.nih.gov/gene/DATA/gene2pubmed.gz) in early 2017. Patent data were obtained from Rosenfeld and Mason [57]. Gene Ontologies, mapped to Entrez Gene IDs, were obtained from NCBI in early 2017 (https://ftp.ncbi.nlm.nih.gov/gene/DATA/gene2go.gz). Funding information was obtained from NIH ExPORTER (https://exporter.nih.gov/) in early 2017. Names of genes and chromosomes were obtained from NCBI NIH in early 2017 (https://ftp.ncbi.nlm.nih.gov/gene/DATA/gene_info.gz). Article types and publication titles were obtained from MEDLINE (https://www.nlm.nih.gov/databases/download/pubmed_medline.html) through a local copy of their database in early 2017. Disambiguated authorship information was obtained from Clarivate Analytics.

SwissProt and TrEMBL protein sequences, and mapping tables to Entrez GeneIDs, were obtained from Uniprot in early 2017 (ftp://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_sprot.fasta.gz, ftp://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot_trembl.fasta.gz, ftp://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/idmapping/idmapping_selected.tab.gz). Linkage tables between Entrez Gene IDs and Ensembl Gene IDs were obtained from NCBI NIH in early 2017 (https://ftp.ncbi.nlm.nih.gov/gene/DATA/gene2ensembl.gz). Genes, coding sequences from genomic RNA, and validated RNA sequences were obtained from Genbank (Genome version GRCh38.p10) (ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/001/405/GCF_000001405.36_GRCh38.p10) using a manually reviewed definition of reference chromosomes according to https://ncbi.nlm.nih.gov/genome. Allele frequencies in human populations were obtained from the ExAc database [58]. Compartment information and protein abundance were obtained from Itzhak and colleagues [59]. Loss-of-function information in human cell lines was obtained from Blomen and colleagues [60], Hart and colleagues [61], and Wang and colleagues [62]. Thermal stability on proteins was obtained from Leuenberger and colleagues [63]. Transcript abundance in cells and tissues was obtained from the human protein atlas [64]. Transcript stability was obtained from Tani and colleagues [65].

GWAS were obtained from the NHGRI-EBI Catalog v1.0 [37].

A local copy of the Web of Science Database was obtained from Clarivate Analytics (and formerly Thomson Reuters). Homologene Version 68 was obtained from NCBI NIH (https://ftp.ncbi.nlm.nih.gov/pub/HomoloGene). Associations between genes and diseases were obtained from Genecard’s GeneALaCart service (https://genealacart.genecards.org) in early 2017 through successive batch queries with all official human (HUGO) gene symbols. The BioGRID database [66] was obtained from BioGRID (Version BIOGRID-3.4.147).

Drugs and their targets were obtained from DrugBank (Version 5.0.7).

Bioplex 2.0 complexes were obtained from Huttlin and colleagues [36]. GenomeRNAi v17 was obtained from www.genomernai.org. EBI Gene Expression Atlas (GXA) was downloaded in spring 2017 from www.ebi.ac.uk/gxa.

Data engineering

For genes, we determined the fraction corresponding to every nucleobase, and the combined ratio of cytosine and guanine, and counted the number of all nucleobases. For protein-coding sequences, we additionally determined the fraction corresponding to individual codons and measured the codon bias according to multiple methods [67–70].

For transcripts, we obtained FPKM values from Uhlen and colleagues [64] and additionally determined the fraction of samples with an expression below 1 FPKM analogously as a surrogate for detectable expression [64].

For SwissProt and TrEMBL proteins, we determined the fraction of the primary sequence covered by individual amino acids. Moreover, we used BioPython [71] to determine the fraction of acidic, aromatic, basic, charged, helix affine, hydrophobic, polar, uncharged polar, sheet affine, and turn affine amino acids. We further used BioPython to estimate protein GRAVY, the protein’s isoelectric point, and molecular weight. Additionally, we counted the total amount of amino acids and thus the length of the protein. We used the Python version of RADAR [72] with its default settings to measure the total number of repeats, and the total RADAR score, and the length of the highest scoring repeat. We used SEG [73] (from NCBI’s ftp.ncbi.nlm.nih/pub/) with its default settings to measure the total amount of amino acids, the fraction of the protein residing in low complexity regions, the length of the longest low complexity region, and the fraction of the protein covered by the longest low complexity region, and counted the total number of low complexity regions and the number of low complexity regions longer than 5, 10, 20, and 40 amino acids. We used SignalP [74] with its default settings to determine the presence of a predicted cleavage site, the maximal cleavage score, the presence of at least four transmembrane residues, and the nucleotide position of the mature protein.

Data imputation

In the absence of measurements on transcript expression and stability, we used −1 to indicate the presence of a low expression.

In the absence of a SwissProt protein entry for a gene, TrEMBL protein entries were used for a given gene. In the absence of measurements on protein localization and stability and protein abundance, we used −1 to indicate the presence of a low expression.

Mapping of genes and gene products

Information of genes and gene products was mapped to Entrez GeneIDs. Only unambiguous mappings were considered. In the case of multiple entries mapping to a single Entrez GeneID (e.g., multiple transcripts encoded by the same gene), we used the median of the features.

Reference research publications

Unless specified otherwise (for reviews), we considered publications that were

(a) assigned by MEDLINE to correspond to a “case report,” “classical article,” “clinical trial,” “clinical trial phase I,” “clinical trial phase II,” “clinical trial phase III,” “clinical trial phase IV,” “comparative study,” “historical article,” “journal article,” “meta analysis,” “multicenter study,” “randomized controlled trial,” “twin study,” or “validation study”;

(b) were further not assigned by MEDLINE to also be a “review”; and

(c) were further not occurring in a journal in which 50% or more of all articles were assigned by MEDLINE to be a “review.”

Reference genes

We considered protein-coding genes of Homo sapiens (NCBI taxonomy ID: 9606) that would also contain an official HUGO symbol and be featured in at least one reference research publication.

Clustering of features used in prediction

Features were z-scored across the genes and clustered using Ward’s method.

Predictions of the number of publications

We predicted the log10-transformed number of publications and z-scored the features across genes. We used 90% of the genes as training data and predicted the remaining 10%. We performed at least 400 randomizations using randomly chosen subsets without replacement. This corresponds to a number of iterations in which, within initial test runs, we would not observe changes in the pooled readout within the number of digits provided in this publication. We used Scikit-learn’s [25] (version 0.19) Gradient Boosting Regressor with 300 estimators and a Huber loss function. The results of individual randomizations for individual genes were pooled by taking the median.

Grouping of genes by features

We considered the 15 features with the highest median importance to the gradient boosting regression models. We considered all reference genes for which these 15 features were defined and z-scored every feature separately across these genes. Grouping onto two dimensions was done by Scikit-learn’s implementation of the t-distributed stochastic neighbor embedding [75].

Gene ontologies

We considered entries to be negating if the qualifier started with NOT, or if the evidence code was “ND.” For temporarily valid, computationally predicted entries, we considered the “IEA” and “RCA” evidence codes. We excluded unmapped entries with the evidence code “–” or “NR.”

Enrichment analysis of grouped genes

Highlighted groups were chosen manually to reflect areas with higher local concentration. Terms considered for enrichment were non-negating, non-temporary Gene Ontology annotations with mapped evidence. We used an EASE score [76], an observation-corrected variant of Fisher’s exact test, and determined the false discovery rate through Scikit-learn’s implementation of Benjamini and Hochberg’s procedure using an alpha of 0.05 [25].

Analysis of recent trends

To account for an uneven total number of pairs between genes and publications, when defining the enrichment within recent years, we normalized either time interval to have the same number of pairs between genes and publications.

We performed a manual literature review on genes with the highest log2 fold change in the number of publications, upon filtering for the presence of at least 10 publications in the interval between 2011 and 2015. We performed a manual literature review and citation analysis to identify findings that changed research on those genes in the subsequent years. Genes highlighted in the main figure were chosen manually to cover a broad range of different numbers of publications, while a complete list is given in S4 Table.

Predictions of the year of the initial publication

The prediction of the year itself was done as described above for the prediction of the number of publications. When adding discoveries of homologous genes, we considered the years of the first description of homologous genes and the years of the first single-gene publications of homologous genes of model organisms listed in Homologene, and indicated absent values (indicative of the absence of either a homologous gene or publications) by assigning the value −1.

Estimation of confidence intervals in display items

Confidence intervals of 95% reflect bootstrapped estimates as computed by Python’s seaborn package [25] (versions 0.7 and 0.8).

Citations towards model organisms

We defined publications with a discovery of a new human gene as those publications that would report on a gene within the year in which the first report on the same gene would appear. We counted the number of cited publications that would have at least one human gene, and the number of cited publications that would have at least one nonhuman gene.

Fractional counting of publications (attention)

For analyses showing the fraction of literature, we performed a fractional counting of publications. Rather than counting every publication as 1 towards every gene, the value of a publication towards a given gene would be 1/(number of genes considered in the publication).

Analysis of human-specific genes

We considered genes that would not map to a Homologene group with at least one nonhuman gene. The analysis only included genes with a human Entrez Gene ID that would be smaller or equal to the highest human Entrez Gene ID within the Homologene Database and thus could have been considered for Homologene.

Analysis of attention enrichment

We performed a fractional counting of publications. Enrichment was calculated as the log2 fold change over the (fractional count of publications in indicated time frame) / [(total number of publications in indicated time frame) / (number of reference genes)].

Analysis of GWAS

We considered EBI’s mapping of associations and only considered associations lying within the sequence of one, but not multiple, genes. We counted the occurrence of at least one association per publication between a gene and a trait. For strong association, we only considered traits covered in at least 10 distinct studies and genes that would be associated with more than 20% of the studies for such a trait.

Analysis of strong loss-of-function intolerance

We considered genes with a pLI over 0.9—a threshold that the authors [58] describe as “extreme loss-of-function” intolerance on their accompanying web portal.

Estimation of funding per gene

We considered NIH funding information between 1985, the year in which data of grants would be provided at the resolution of principal investigators, and 2015. We performed inflation correction using the average United States consumer price index. We equally distributed the total money allocated to a given NIH project ID to all publications supported by this project, and subsequently within the individual genes in this project. We used disease associations from Malacards for Unified Diseases, Orphanet, Human phenotypes, and OMIM as disease linkage features and constructed additional features that would count for the total number of entries within each of the four data sets. Because of computational constraints, we subsequently removed disease linkage features with fewer than 10 genes. Notably, prediction accuracy did not improve if keeping all linkages of Unified Diseases (Spearman 0.73 for addition on top of other features—analogously to S9D Fig; Spearman 0.42 for exclusive usage—analogously to S9E Fig) or OMIM (Spearman 0.71 for addition on top of other features—analogously to S9D Fig; Spearman 0.16 for exclusive usage—analogously to S9E Fig).

Analysis of transitioning to a future principal investigator status

As the rank of the popularity, we used the fractional count of publications up until the indicated year. We only considered publications of authors that have not yet transitioned to a principal investigator status. As principal investigator status, we consider authors that have at least two last author publications with at least one fellow coauthor.

Mapping of Web of Science to MEDLINE

We matched publications contained in MEDLINE to records from Web of Science in a two-step process:

(a) if available, we used the digital object identifier (DOI), allowing for an unambiguous identification of publication entries;

(b) otherwise, given the MEDLINE record, we retrieve all publications from Web of Science with the same list of authors’ last names, and that were published in the same year and journal. We then identify the best-matching record by calculating the Levenshtein distance (implemented in seatgeek’s FuzzyWuzzy Python package: https://github.com/seatgeek/fuzzywuzzy) between titles of the MEDLINE and the Web of Science record, respectively. We only considered publications that would map unambiguously and had a mapping score of at least 95 (maximum score 100).

In total, for 97% of all publications in MEDLINE containing a reference to a gene, we were able to identify the corresponding record in Web of Science.

Analysis of fraction of highly cited publications

Following Uzzi and colleagues [34], we counted citations over the 8 years following the year of the publication. Publications with more than two authors and publications with consortium as the sole affiliation were considered to be team publications. For the analysis of BioGRID, we considered BioGRID entries that had been associated with at least one gene in MEDLINE and counted the unique genes of a publication—after pooling the indicated gene A and gene B entries of an interaction—which would usually be indicative of bait and hit, respectively.

Analysis of experimentation

Western blots following affinity purification were obtained from BioGRID. For differential gene expression analysis, we used EBI GXA and considered genes to be differentially expressed if their (nonadjusted) p-value would be below 0.0001. For RNAi, we only considered phenotypes that were not measured through distinct shRNA abundance and only considered genes occurring in at least 20 studies (which could possibly have monitored distinct phenotypes). We considered a gene to have a strong RNAi if more than 30% of the studies containing the gene would report a phenotype for this gene. This was motivated by the (not shown) observation that genes fall into a bimodal distribution according to the fraction of studies reporting a phenotype, separated at the chosen threshold of 30%.

Code availability

Code for the curation of data sets and for analysis is available at github.com/tstoeger/plos_biology_2018_ignored_genes.