Selection criterion

Before describing the algorithmic details of MOSGWA we will formally introduce the selection criterion mBIC2. Consider a GWAS based on SNPs and individuals. For a given model including SNPs our model selection criterion is of the form (1)

Each set of SNPs forms a potential model , and MOSGWA tries to find that model which minimizes mBIC2. Here is the Firth corrected maximum likelihood, which will be discussed in more detail below.

The penalty of the mBIC2 criterion was introduced by Frommlet et al. [21], where its derivation was based on ideas of Abramovich et al. [1]. In particular mBIC2 is closely related to the Benjamini-Hochberg multiple testing procedure [6], and it controls the false discovery rate of detected SNPs. In the context of linear regression certain asymptotic optimality properties of mBIC2 were shown [7], [21]. Roughly speaking selection based on mBIC2 minimizes the misclassification error when both and are large, while the number of regressors of the correct model is relatively small.

An extensive motivation of mBIC2 can be found in Frommlet et al. [23], where the criterion is applied for linear regression models to analyze GWAS with quantitative traits. In contrast we will focus here on case-control studies, and just like HLASSO [26] and GWASelect [29] we make use of logistic regression to model the disease risk of SNPs. To this end let denote the disease status of individual ( for a case, for a control), and let denote the genotype of SNP for individual . If a model includes the SNPs then the corresponding logistic regression model can be written as (2)with the parameter vector . These parameters can be routinely estimated by maximizing the corresponding likelihood , although occasionally the well known problem of separation may occur, where some parameter estimates tend towards infinity [2]. In classical statistical applications, where , separation typically arises only in case of small sample sizes. For GWAS the sample size is usually very large, but the number of potential regressors is even several orders larger, which results in many combinations of SNPs for which separation occurs. Heinze and Schemper [27] suggested to overcome the problem of separation using a bias corrected version of logistic regression which was originally proposed by Firth [18]. The likelihood of the logistic regression model is multiplied with the corresponding Jeffreys prior, which is just the square root of the determinant of the Fisher information matrix . Thus the Firth corrected maximum likelihood from equation (1) is given by (3)and explicit formulas are given for example by Heinze and Schemper [27]. The Firth-corrected log-likelihood includes the penalty term , which guarantees that parameter estimates cannot get excessively large. Note that LASSO based procedures like GWASelect do not run into difficulties with separation because the -penalty yields automatically a shrinkage of parameters.

Search strategy

Having defined the model selection criterion (1) the main task is to find the model which minimizes mBIC2. The resulting problem is an extremely challenging mixed integer program, for which one can only attempt to develop heuristic methods which yield an approximate solution. The search algorithm of the software package MOSGWA repeatedly makes use of a strategy called fast stepwise search (FSS).

The aim of FSS is, starting from some initial model, to perform a search heuristic which finds a model with a smaller value of a given selection criterion. The final call of FSS is performed with the target criterion mBIC2, but within the search it is valuable to work with less stringent criteria to avoid getting stuck in local minima corresponding to models which are missing some of the causal SNPs. Specifically we consider the milder criterion

FSS depends on a pre-specified order of all markers not included in the initial model. This order is either based on some marginal test statistics, or on some conditional score tests as described below. We will thus formally write the fast stepwise search as a function to emphasize that it depends on the initial model, on the specific order of markers according to test, and on the respective criterion. Depending on the order of markers two groups are considered: Group consists of the best SNPs, and group of the best SNPs. Thus , where is the set of SNPs along which directed forward steps are performed (see below), whereas SNPs within might enter the model via so called exchange steps (see below). The exact choice of the parameters and turns out to be not too important. The default values of MOSGWA which are also used for simulations are and (as long as ).

FSS is based on three algorithmic steps which we call directed forward, exchange, and backward step. FSS starts by considering the initial model as the current model. The directed forward step repeatedly tests if enhancing the current model with a SNP decreases the criterion, where SNPs within are considered in the order obtained from the test (therefore directed forward search). The first SNP which improves the current model is added, and an exchange step follows.

In the exchange step all SNPs in the current model are tested whether exchanging them with suitable other SNPs decreases the criterion. Suitable candidates for exchanging SNP are all other SNPs within whose physical distance to on the chromosome is less than . The default cutoff value of MOSGWA is , that is we only consider the 49 closest neighboring SNPs in both directions. The algorithm starts with the first SNP of the current model, and tries to exchange it with all suitable candidate SNPs. In case of improvement the first SNP is substituted with the best possible alternative. Then exchanges of the second SNP with all corresponding candidates are performed, and so on till each SNP of the model has been considered once. The idea behind this strategy is that in the directed search step it might happen that not the optimal SNP was chosen, but a correlated SNP might further improve the model. Also after several SNPs have been added to the model it can happen that exchanging a particular SNP of the model is beneficial. Limiting the exchange to SNPs close to which themselves have reasonably large test statistic makes this strategy computationally feasible.

The third step of FSS is an extended backward elimination step. First a standard greedy elimination step is performed, which means that all models are considered where one SNP is removed from the current model. If this does not improve the model, greedy elimination is repeated up to three times to look for better models. In other words like in a classical stepwise elimination procedure we first remove the SNP which explains the least, if this does not improve the selection criterion we additionally remove the next SNP which explains the least, and if the resulting model again has larger selection criterion than the original model we try one more time to remove the least explanatory SNP. The resulting best model is then the starting point for another directed forward step. Directed forward, exchange, and backward steps are then performed repeatedly till no further improvement of the criterion is achieved.

Starting with the null model the complete search strategy of MOSGWA can be specified as follows:

1.  =  FSS(, Cochran Armitage, mBIC_60)
2.  =  FSS(, Score Test, mBIC_60)
3.  =  FSS(, Score Test, mBIC2)

The general strategy of this algorithm can be motivated as follows. In the first round markers are preselected based on their marginal (Cochran Armitage) test statistic. In the second round score tests conditional on the model of the first round are performed over all remaining SNPs (see [26]). Score tests have the benefit of being computationally much less expensive than likelihood ratio tests, and therefore provide a fast way to preselect markers which might be of importance additional to markers within .

The first two rounds are performed with the milder criterion mBIC_60, which is expected to yield models which are too large. In fact when models are getting too large then the value of from the exchange step is reduced to guarantee reasonable runtime. As mentioned previously the benefit of first working with mBIC_60 is that one reduces the chances of missing out on important SNPs due to local minima. Only in the final round FSS is performed with the target criterion mBIC2, and will then yield a model for which the type I error rate is controlled in terms of FDR.

The software package MOSGWA is available at http://mosgwa.sourceforge.net/. An application note describing the software in more detail is in preparation.

Global null hypothesis

Our first set of simulations is concerned with controlling the type I error under the global null hypothesis. Simulations are based on real SNP data from individuals from the POPRES sample [34], which are included in the POPRES_Genotypes_QC2 dataset. Individuals are randomly allocated as cases and as controls with equal probability; then MOSGWA, HLASSO and GWASelect are applied to evaluate their ability to control the type I error rate. The random allocation was repeated 200 times, and Table 1 presents the average number of observed false positives to estimate the per-family error rate. A graphical illustration is provided in Figure 1.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Illustration of the simulation results under the total null hypothesis.

The average number of false positives for MOSGWA is compared with HLASSO (Fig 1a) and with GWASelect (Fig 1b), for which false positives were clustered. Simulations were performed for four different numbers of chromosomes, resulting in different numbers of SNPs plotted on the x-axis.

https://doi.org/10.1371/journal.pone.0103322.g001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. False positives under the global null hypothesis.

https://doi.org/10.1371/journal.pone.0103322.t001

To study the influence of the number of SNPs we performed four different simulations using SNPs only from chromosome 1, from the first two, the first four and the first six chromosomes, respectively. The resulting number of SNPs, , for these four scenarios is given in the first column of Table 1. GWASelect and single marker tests tend to report a large number of correlated SNPs, and thus for these two procedures the number of false positives was obtained by counting clusters of neighboring SNPs [26], where clustering was performed with the algorithm of Frommlet [20]. For HLASSO and MOSGWA no such clustering is necessary, because these algorithms tend to select only one representative for a genomic region anyway.

Table 1 illustrates that MOSGWA controls the type I error under the total null hypothesis irrespective of the number of SNPs. For MOSGWA no parameter needs to be tuned, whereas the latest version of HLASSO allows to choose a parameter which corresponds to an uncorrected nominal significance level. We consider three Bonferroni-like choices of the form . The results from Table 1 indicate that the type I error tends to remain below the nominal level, which is not too surprising given the positive correlations between neighboring SNPs due to linkage disequilibrium. Type I error rates for are closest to those from MOSGWA, and therefore from now on HLASSO will be used with this parameter setting.

The size of the model selected by GWASelect depends on the stable-selection-threshold , which is defined in the following way; GWASelect reports a SNP as detected when the proportion of random subsamples in which the SNP was selected is larger than . He and Lin [26] recommend a choice between 0.1 and 0.2. However, the results from Table 1 show that for these parameter settings GWASelect completely fails to control the type I error rate under the global null. We therefore considered additionally , for which the per-family error rate is controlled at least to some extent. Interestingly the type I error from GWASelect decreases when the number of SNPs increases.

The last column of Table 1 provides the results of single marker tests performed with PLINK [36]. We considered logistic regression models for each marker including the first four principle components of all SNP genotypes as covariates to account for population structure. This kind of adjustment is in principle not necessary when simulating under the total null hypothesis, but it becomes important for the simulation of complex traits below. We applied the Benjamini Hochberg procedure to account for multiple testing, and we can see that under the total null hypothesis the single marker tests control the type I error rate pretty much at the nominal level .

Complex trait

The second set of simulations is concerned with the power to detect causal SNPs. To this end we consider again the SNPs from the first six chromosomes for the 4077 individuals from the POPRES sample. Simulations are performed for three scenarios, which include 6, 12, and 24 causal SNPs, respectively. All causal SNPs are common (MAF ), equally distributed over the six chromosomes, and with pairwise correlation for each pair. Disease risk was computed for each individual according to equation (2), based on which for each scenario 200 case-control data sets were sampled. Effect sizes were ranging in the interval , yielding causal SNPs with intermediate power. The coefficient of the intercept was chosen such that the number of cases and controls in each simulation run was more or less identical.

Before analyzing the data in each scenario half of the causal SNPs were removed, mimicking the situation where SNPs associated with a trait are not causal themselves, but only in linkage disequilibrium with the cause. SNPs to be removed were selected in such a way that there actually were SNPs in linkage disequilibrium, to make it possible for these signals to be detected. The simulated data were then analyzed with MOSGWA, HLASSO (using parameter ), GWASelect using parameters and with single marker tests (as previously including the four leading principal components in logistic regression models and applying Benjamini Hochberg procedure at nominal level ).

Table 2 and Figures 2, 3 and 4 summarize the corresponding results in terms of estimated power (which is just the average number of correctly detected signals divided by the total number of causal SNPs), the average number of false positives, the average number of misclassifications (that is false positives plus missed causal SNPs) and the estimated false discovery rate.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Simulation results under an alternative with causal SNPs.

The four panels (Fig 2a, Fig 2b, Fig 2c, Fig 2d) show the average power, number of false positives, misclassification rate and false discovery rate as a function of the threshold value which determines if a detection is a true or a false positive. The performance of MOSGWA is compared with single marker tests, HLASSO, and with GWASelect using three different parameters for stability selection.

https://doi.org/10.1371/journal.pone.0103322.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Simulation results under an alternative with causal SNPs.

The four panels (Fig 3a, Fig 3b, Fig 3c, Fig 3d) show the average power, number of false positives, misclassification rate and false discovery rate as a function of the threshold value which determines if a detection is a true or a false positive. The performance of MOSGWA is compared with single marker tests, HLASSO, and with GWASelect using three different parameters for stability selection.

https://doi.org/10.1371/journal.pone.0103322.g003

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 4. Simulation results under an alternative with causal SNPs.

The four panels (Fig 4a, Fig 4b, Fig 4c, Fig 4d) show the average power, number of false positives, misclassification rate and false discovery rate as a function of the threshold value which determines if a detection is a true or a false positive. The performance of MOSGWA is compared with single marker tests, HLASSO, and with GWASelect using three different parameters for stability selection.

https://doi.org/10.1371/journal.pone.0103322.g004

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Summary of simulation results for complex traits.

https://doi.org/10.1371/journal.pone.0103322.t002

A crucial point in computing all these statistics is the definition of true positives and false positives. Of course we know the SNPs which we used to simulate the data, which we will call causal SNPs or correct SNPs. Now there might be several SNPs in close linkage disequilibrium with a causal SNP, and there is the question whether we count a detected SNP which is strongly correlated with the correct SNP as true positive or as false positive. Furthermore half of the causal SNPs under which we simulated were removed before analyzing the data. Therefore to get reasonable results we actually have to count detections which are strongly correlated with a causal SNP as true positives.

It is quite common in this context to use a threshold value on the correlation between causal SNPs and any detected SNP to determine whether a detected SNP is a true positive [26], [29]. If several detected SNPs are closely correlated to one causal SNP we count all of them as one true positive. Just like for the simulation under the total null we have additionally clustered the false positives of GWASelect and of the single marker analysis. This was performed with the algorithm described in Frommlet [20] which computes so called -clusters, that is clusters of SNPs where it is guaranteed that within each cluster all SNPs have pairwise correlation larger than . We used for clustering and for determining true positives always the same constant . Counting the number of clusters rather than the total number of false positives works in favor of the performance of GWASelect and single marker tests. Both for MOSGWA and HLASSO such clustering appears to be unnecessary, because for a genomic region of closely related SNPs usually these procedures select only one representative.

In Figures 2, 3 and 4 we illustrate the dependence of the different statistics on the threshold values for correlations which specify true positives via Results were computed for . One can see that within the range of the dependence on the threshold is relatively minor for all methods. In general the choice of has the biggest impact on the results of single marker tests. For those the number of false positives grows much faster than for the other methods, because the number of false positive clusters increases with stricter clustering threshold . Table 2 specifically reports the results for a threshold .

For all three scenarios MOSGWA has the lowest number of misclassifications, which is in accordance with theoretical results [21]. Of particular interest is the comparison between MOSGWA and HLASSO. For MOSGWA has slightly larger power and lower Type I error. For and specifically for MOSGWA has much larger power than HLASSO, but also larger Type I error. When the number of causal SNPs is increasing then HLASSO is getting more conservative in comparison with MOSGWA, which is in accordance with the way both algorithms are designed. HLASSO tries to control the FWER at a certain level, whereas MOSGWA tries to control the FDR. According to asymptotic theory [7], [24], when aiming at a minimal number of misclassifications in a sparse setting it is preferable to control FDR rather than FWER.

Concerning GWASelect just like under the total null hypothesis the choice of gives way too large models. The choice of works slightly better in terms of controlling the Type I error, but is less powerful than MOSGWA and HLASSO. Given the simulation results under the total null hypothesis one actually should use the setting , but then GWASelect is no longer competitive at all in terms of power.

The single marker tests are performing surprisingly well, and the disadvantage compared to the model selection approaches is much less than it was observed by Frommlet et al. [23] for quantitative traits. However, we believe that this is mainly due to the fact that we have added the first four principle components of SNP genotypes. Although we did not specifically simulate scenarios where population structure would play a major role in itself, when testing a specific SNP the principle components in the model help to adjust for the net effect of all the other causal SNPs. Without adding principle components the performance of the single marker tests was extremely poor. Also note that for MOSGWA we did not specifically take any measures to take into account population structure. Still it is much more powerful than single marker tests in all three scenarios, while at the same time controlling FDR at a comparable level.

Data preprocessing of WTCCC data

We reanalyzed bipolar disorder (BD), coronary heart disease (CAD), hypertension (HT), Crohn's disease (IBD), rheumatoid athritis (RA), type 1 diabetes (T1D) and type 2 diabetes (T2D). Like in the original article [41] all diseases are compared with the same control group of 3000 individuals. In the following filenames we will denote by DIS any of the seven abbreviations for diseases given above. Starting from the already imputed WTCCC data sets we removed individuals for each disease and for the control group according to the files exclusion-list-05-02-2007-DIS.txt and exclusion-list-snps-26_04_2007.txt.

After merging cases with controls we removed SNPs with a minimal allelic frequency smaller than 0.01. Furthermore we tested for Hardy Weinberg equilibrium and SNPs with p-values smaller than 0.0001 were also removed. Finally we only considered SNPs for which the genotype calling algorithm from the WTCCC confirmed good clustering, where we took that information from the files WTCCC_summary_data/7_Diseases/DIS/basic/snptest_DIS_CHR.txt. Here CHR is the chromosome number ranging from 1 to 22. According to private correspondence with He and Lin they had used the file wtccc_DIS_basic_chr_CHR.xml for similar purposes. That file must have differed from the file we have used, but is no longer available from WTCCC. In summary, after preprocessing we will have ended up with a different set of SNPs than used in [26], and according to email correspondence with the authors it is no longer possible to reconstruct which set of SNPs they had used for their own analysis. Not starting from the same set of SNPs explains why we were not able to completely reproduce the results reported by He and Lin [26].

Summary of results

Like in the simulation study for complex traits HLASSO was applied with parameter . For GWASelect we present the results again for , in spite of the fact that we have seen that only controls the type I error rate under the total null hypothesis. Table 3 gives for each disease the number of detected SNPs (and associated regions) obtained from the original WTCCC analysis, MOSGWA, HLASSO, and GWASelect. A more detailed discussion concerning the detected SNPs follows below.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Summary of real data analysis.

https://doi.org/10.1371/journal.pone.0103322.t003

The first observation is that GWASelect with parameter is selecting in 6 out of 7 diseases a much larger number of SNPs than the other methods. Given the results from simulations we are forced to conclude that most of those SNPs might be false positives, and we will not give the detailed results for GS 0.1 in the next section except for rheumatoid arthritis, where interestingly GS 0.1 gives quite similar results to MOSGWA and HLASSO (for a detailed explanation see below). For Crohn's disease and Type I diabetes the results for are relatively close to MOSGWA and HLASSO, whereas for all other diseases might be the best choice for GWASelect. The general conclusion is that the results of GWASelect heavily depend on the choice of , and it is not really possible to know in advance which choice gives reliable results.

The comparison between MOSGWA and HLASSO is quite interesting. For the four diseases for which only a small number of SNPs was detected (BD, CAD, HT, T2D) MOSGWA finds exactly the same regions as HLASSO, though in two cases one SNP less. Similarly MOSGWA has a tendency to select less representatives of a region than HLASSO for the remaining more complex traits. This might have to do with the fact that the coefficient estimates of MOSGWA suffer from even less shrinkage than the estimates from HLASSO. Hoggart et al. [29] provide a thorough discussion of the fact that in case of shrinkage the regressors which enter a model explain less than they would without shrinkage, which results in a higher chance of including further correlated SNPs in the model. This is the main reason why HLASSO works with the NEG prior, which results in less shrinkage than the double exponential prior corresponding to LASSO.

A second observation is that especially for complex traits MOSGWA tends to select more regions of association than HLASSO. This goes along with the fact that HLASSO is designed to control the FWER, whereas MOSGWA controls the FDR. In case of complex traits MOSGWA is therefore bound to find more SNPs than HLASSO, whereas if there are only few signals both methods behave very similarly. Looking more closely into the results for IBD, RA and T1D, we observe that MOSGWA detects SNPs within all regions which were reported as significant according to the original WTCCC analysis [41]. The same is not true for HLASSO, which misses out one region on chromosome 10 for RA, and another region on chromosome 16 for T1D. On the other hand MOSGWA finds exclusively 7 SNPs for IBD, 4 SNPs for RA, and 6 SNPs for T1D, respectively. We will next provide a thorough discussion on the potential relevance of these SNPs which were exclusively found by MOSGWA.

WTCCC Results in detail

For each disease we provide tables as supporting information which list all SNPs which were detected by different methods. Specifically the first column of these tables gives the reference SNP ID number from dbSNP, followed by the chromosome (Chr) and the position (Pos). The column Gene contains information about the closest lying gene according to the databases dbSNP and ImmunoBase:

• dbSNP: http://www.ncbi.nlm.nih.gov/SNP/
• ImmunoBase: http://www.immunobase.org/

The next four columns have bullets whenever a SNP was detected by MOSGWA (M), Hlasso (HL), GWASelect with parameter (G3) or (G2). According to our simulation results we believe that the large number of additional SNPs detected by GWASelect with (G1) will include mainly false positives. Therefore we do not present detailed results for G1, with the exception of rheumatoid arthritis which has a rather particular genetic constellation. If many neighboring SNPs are reported by different methods we consider such groups of SNPs as genetic regions, and we label such groups in the tables using background colors. The crosses in the last column (W) indicate regions which were reported originally in [41].