http://orcid.org/0000-0002-0381-7804
Figures
Abstract
Warfarin dosing remains challenging due to narrow therapeutic index and highly individual variability. Incorrect warfarin dosing is associated with devastating adverse events. Remarkable efforts have been made to develop the machine learning based warfarin dosing algorithms incorporating clinical factors and genetic variants such as polymorphisms in CYP2C9 and VKORC1. The most widely validated pharmacogenetic algorithm is the IWPC algorithm based on multivariate linear regression (MLR). However, with only a single algorithm, the prediction performance may reach an upper limit even with optimal parameters. Here, we present novel algorithms using stacked generalization frameworks to estimate the warfarin dose, within which different types of machine learning algorithms function together through a meta-machine learning model to maximize the prediction accuracy. Compared to the IWPC-derived MLR algorithm, Stack 1 and 2 based on stacked generalization frameworks performed significantly better overall. Subgroup analysis revealed that the mean of the percentage of patients whose predicted dose of warfarin within 20% of the actual stable therapeutic dose (mean percentage within 20%) for Stack 1 was improved by 12.7% (from 42.47% to 47.86%) in Asians and by 13.5% (from 22.08% to 25.05%) in the low-dose group compared to that for MLR, respectively. These data suggest that our algorithms would especially benefit patients requiring low warfarin maintenance dose, as subtle changes in warfarin dose could lead to adverse clinical events (thrombosis or bleeding) in patients with low dose. Our study offers novel pharmacogenetic algorithms for clinical trials and practice.
Citation: Ma Z, Wang P, Gao Z, Wang R, Khalighi K (2018) Ensemble of machine learning algorithms using the stacked generalization approach to estimate the warfarin dose. PLoS ONE 13(10): e0205872. https://doi.org/10.1371/journal.pone.0205872
Editor: Maciej Huk, Wroclaw University of Science and Technology, POLAND
Received: August 17, 2018; Accepted: October 2, 2018; Published: October 19, 2018
Copyright: © 2018 Ma et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: IWPC Data set are freely available from PharmGKB under the section of Published Data (PMID:19228618; http://www.pharmgkb.org/downloads/). Data are not owned by any of the authors. The authors did not receive special privileges from PharmGKB.
Funding: The funder Science Center of Opera Solutions LLC provided support in the form of salaries for the author [R.W.], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.
Competing interests: This commercial affiliation [Science Center of Opera Solutions LLC for R.W.] does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Warfarin is the most widely used oral anticoagulant worldwide. Due to its narrow therapeutic window and large interpatient variability, warfarin dosing remains challenging [1]. The consequence of incorrect warfarin dosing can be devastating, predisposing the patient to thrombosis in the case of under-dosing or bleeding in the case of overdosing. Because of these challenges associated with warfarin use, it is one of the leading causes in emergency department visits and the most often cited cause of drug-related mortality [2]. Clinical factors, demographic variables and genetic variants significantly contribute to interpatient variability in dose requirement for warfarin. While non-genetic factors, such as age, height, weight, race and drug interaction, have been reported to explain 15–20% of the variability in dose, polymorphisms in cytochrome p450, family 2, subfamily C, polypeptide 9 (CYP2C9), and vitamin K epoxide reductase complex, subunit1 (VKORC1) independently correlate with warfarin therapeutic dose [3–5], and combined polymorphisms of those two genes account approximately 30% (20%-25% for VKORC1 rs9923231; 5%-10% for CYP2C9) of the interpatient warfarin dose variability [3–6].
Remarkable efforts have been made to estimate the appropriate warfarin dose to improve the patient care. Many pharmacogenetic algorithms integrating clinical, demographic and genetic variables have been created to predict the dose requirement in individual patients [7–11]. Most of the dosing algorithms are based on multivariate linear regression (MLR). One of the most widely used and tested algorithms is the IWPC pharmacogenetic algorithm [8], which has proved accuracy in multiple studies. Other advanced machine learning approaches such as deep learning (neural networks), tree-based algorithms and support vector machines have also been used to predict warfarin dose [7, 10, 11], but those studies use a single machine learning algorithm to maximize the accuracy of predicting warfarin dose. In a mathematical point of view, a machine learning algorithm is a sophisticated fit to a non-linear function, and a single machine learning model may fit well to a certain subset of patients, but may overfit or underfit to the rest of patients with different genetic and racial background. As a result, the prediction accuracy of a single model may reach an upper limit even with optimal parameters. It is not surprising that the models produced by MLR and support vector regression with a linear kernel were statistically indistinguishable and significantly outperformed all the other approaches in the IWPC cohort [8]. One way to overcome the limitation of a single algorithm is to combine the advantages of several algorithms to break through the upper limit of a single machine learning algorithm (i.e. ensemble method). Recently, the ensemble method “bagging” has been applied to predict warfarin dose [12, 13]. Stacked generalization is another ensemble method that uses a higher-level model to combine lower-level models to achieve higher prediction accuracy [14, 15]. Unlike the “bagging” and “boosting” approaches which can only combine machine learning algorithms of the same type, stacked generalization can combine different types of algorithms through a meta-machine learning model to maximize the generalization accuracy.
In this study, we created novel regression models to estimate warfarin stable dose utilizing stacked generalization frameworks that combine the advantages of distinct machine learning algorithms and significantly improved the prediction accuracy compared to MLR.
Materials and methods
The International Warfarin Pharmacogenetic Consortium (IWPC) Cohort
IWPC Cohort has been described previously [8]. Expanded Data set was downloaded from the PharmGKB website (http://www.pharmgkb.org/downloads/), which contains pooled data on 6256 chronic warfarin users recruited through collaborative efforts of 22 research groups from 4 continents. This data set includes detailed de-identified curated data on demographic factors, clinical features, such as age, weight, height and concomitant use of amiodarone, as well as CYP2C9 and VKORC1 genotypes. Missing values for height and weight were imputed with multivariate linear regression models. Specifically, weight, race, and sex were used for the imputation of the height variable, while height, race, and sex were used for the weight variable. For missing values of the VKORC1 rs9923231, the imputation strategy has been described [8], which is based on linkage disequilibrium in VKORC1 and race. We excluded 17 subjects with CYP2C9*5, *6, *11, *13 and *14, due to low allele frequency and an outlier subject with warfarin stable dose 315 mg/week. A total of 5743 subjects were included in this study.
Stacked generalization framework
Predicting warfarin maintenance dose is a regression task, which requires a function or model f that maps an input vector x ∈ Ρd onto the corresponding continuous label value y ∈ Ρ. Since a training data set {(x1, y1), …, (xn, yn)} is utilized to find f, the task falls into the category of a supervised learning problem. The machine learning problem is formulated as a minimization problem of the form:
(1)
The first term of the objective is the empirical risk described by a loss function S which measures the quality of the function f. A specific case is the squared loss l(ŷ, y) = (ŷ - y)2. The second term of the objective in Eq (1) is a regularization term which measures the complexity or roughness of the function f, which usually is a norm of f or its derivatives. L2 regularization was used in this study.
Stacked generalization is utilized to ensemble different machine learning algorithms, which can be viewed as a means of collectively using several models to estimate their own generalizing biases with respect to a particular learning set, and then filter out those biases [16, 17]. There are two kinds of models in a stacked generalization framework: several base models (level-0 models) and one meta-model (level-1 model). The essence of stacked generalization is to use the level-1 model to learn from the predictions of level-0 models. Generally, a stacked generalization framework can obtain more accurate prediction compared to the best level-0 model [16].
One of the key points is to obtain the training data for level-1 model (Dcv) from cross-validation technique. Given an original data set D = {(yn, xn), n = 1, …, N}, where yn is the target value and xn represents feature vectors of the nth instance, randomly split the data into K almost equal folds D1, D2, …, DK (K = 5 in this study). Define Dk and D(-k) = D–Dk to be the test and training sets for the kth fold of a K-fold cross-validation. Given J different level-0 machine learning algorithms (M1, M2, …, MJ), each Mj is trained by D(-k) and predict each instance x in Dk. Let vk(-j)(x) donate the prediction of the model Mj on x. Then we have:
(2)
At the end of the entire cross-validation process of each Mj, the data assembled from the outputs of the J models is
(3)
Dcv is the training set of level-1 model Mmeta. To complete the training process, level-0 models Mj (j = 1, 2, …, J) are trained using original dataset D, and Mmeta is trained by Dcv.
Now we consider the prediction process, which uses the models Mj, j = 1, 2, …, J, in conjunction with Mmeta. Given a new instance, models Mj produce a vector (z1, …, zJ). This vector is input to the level-1 model Mmeta, whose output is the final prediction result for that instance.
Implementation of machine learning algorithms and parameters
The neural networks (NN), ridge regression (RR), random forest (RF), and extremely randomized trees (ET), support vector regression (SV) were implemented using python library Scikit-learn (version 0.19.1) [18]. Gradient boosting trees (GBT) was implemented using Microsoft’s software ‘LightGBM’ with a python wrapper [19]. Below are the key parameters for each single model: (i) NN layer: 3, hidden layer size: 100, activation function: ‘logistic’, solver: 'lbfgs'; (ii) RR regularization: 1.0; (iii) RF number of trees: 100, maximum depth of the tree: 100; (iv) ET number of trees: 100, maximum depth of the tree: 100; (v) GBT number of trees: 100, Maximum depth of the tree: -1, Learning rate: 0.1; (vi) SV kernel: linear, cache_size: 1000.
In this study, 80% of the eligible patients were randomly chosen as the training set (N = 4594), which was used to train the machine learning models. The rest 20% of the patients (N = 1149) were utilized as the hold-out test set. Various models were then built on the training set and the hold-out test set was used to evaluate the model performance by two metrics. The mean absolute error (MAE), the absolute difference between the predicted and actual maintenance doses, was used to evaluate each model’s predictive accuracy. The mean of the percentage of patients whose predicted dose of warfarin within 20% of the actual stable therapeutic dose (mean percentage within 20%) was utilized to evaluate the clinical significance of each algorithm, as a difference in warfarin dose greater than 20% is likely to be considered to clinically relevant by clinicians. The features/variables in single models were identified based on reported IWPC pharmacogenetic dosing algorithm [8], including height, weight, race, age, enzyme inducer and use of amiodarone. Additional features used in other warfarin stable dose prediction algorithms [7, 9, 10, 20] such as diabetes mellitus, heart failure, valve replacement, smoking, use of statins, smoking status, and other VKORC1 genotypes (S1 Table) were also included to train the stacked generalization models.
Warfarin dose subgroup analysis
To assess the performance of the algorithms in different dose ranges, warfarin stable dose was divided into three subgroups based on the 25% and 75% quantiles according to race: low dose (< 15.0 mg/week), intermediate dose (15.0–28.0 mg/week), and high dose (> 28.0 mg/week) in Asians; low dose (< 30.0 mg/week), intermediate dose (30.0–52.5 mg/week), and high dose (> 52.5 mg/week) in blacks; low dose (< 22.0 mg/week), intermediate dose (22.0–42.5 mg/week), and high dose (> 42.5 mg/week) in whites; low dose (< 22.0 mg/week), intermediate dose (22.0–40.0 mg/week), and high dose (> 40.0 mg/week) in the missing or mixed race.
Statistical analysis
Due to the skewed (with a longer tail at high doses) distribution of warfarin dose, we transformed raw dose into square root of the dose. To obtain robust statistics, 100 rounds of resampling were performed from the IWPC cohort. All the quantitative data are presented as means with 95% confidence intervals (CI). P values < 0.05 were considered to be statistically significant. The MAE and differences of the mean percentage within 20% of the actual stable dose among the algorithms were compared with unpaired Student’s t-test. All statistical analyses were conducted with R (version 3.4.4).
Results
Basic characteristics of the IWPC cohort
The characteristics of the patients are shown in Table 1 and S1 Table. In the IWPC cohort, 5743 patients were included for analyses with a median warfarin stable dose of 28.0 mg/week. The target range for International Normalized Ratio (INR) fell within 1.7 to 3.3, with the majority of being prespecified between 2 and 3. The proportions of patient age less than 50, 50 to 80, and older than 80 were 17.0%, 71.0%, and 12.1%, respectively. There were 4.9% of patients concomitantly taking amiodarone, 19.6% of patients taking statins, 1.1% of patients using enzyme inducers (including phenytoin, carbamazepine, and rifampin). There were 8.4% of patients smoking. Patients with diabetes mellitus, cardiomyopathies and heart failure, and valve replacement were 10.8%, 12.5% and 17.0%, respectively.
Overall performance of the single algorithms
To establish the stacked generation frameworks to ensemble different models, we first examined the predictive performance of several single models. Eight different machine learning algorithms including MLR, SV, RR, NN, GBT, RF, ET, and K nearest neighbors (KN) were trained by the training set and then prediction was made on the hold-out set. The performance of each model measured by the MAE and percentage within 20% is shown in Table 2. The algorithms SV, RR, and MLR performed statistically indistinguishable and were the best among eight algorithms, followed by GBT and NN. The algorithms RF, ET and KN resulted in the least favorable results.
Configuration and performance of stacked generalization framework 1 and 2 (Stack 1 and 2)
To determine whether ensemble predictors constructed using stacked generalization improve the prediction accuracy for warfarin stable dose, we constructed two different stacked generalization frameworks Stack 1 and 2 using the exactly same parameters in individual algorithms. The frameworks in this study consisted of three level-0 models and one level-1 model (meta model). Fig 1 exhibits the detailed configuration of each framework. The individual algorithms (SV, RD, NN and GBT) in the frameworks were chosen based on the type of the algorithm (i.e. linear regression, neural network and tree-based) and individual predictive performance in Table 2. To improve the prediction accuracy, we included diabetes mellitus, heart failure, valve replacement, smoking, use of statins, and other VKORC1 genotypes as additional features to train the frameworks (Stack1 and 2). The MAEs for the Stack1 and 2 were 8.31 and 8.31 mg/week, which were significantly better than 8.53 mg/week in the MLR (Table 3). The mean percentage within 20% produced by Stack 1 and 2 (47.85% and 47.81%) were significantly higher than that (46.31%) of MLR (Table 3).
SV: Support Vector Regression; RR: Ridge Regression; NN: Neural Network; GBT: Light Gradient Boosting Machine.
Evaluating the performance of the stacks by race
Given that genetic diversity is known to be great in different races [21], to further examine the performance of the Stacks in different races, Stack 1 and Stack 2 were evaluated in Asians, whites and blacks. The MAE of prediction was highest in blacks and lowest in Asians across the three algorithms (Table 4). In Asians, Stack 1 and Stack 2 performed significantly better than MLR for both MAE and mean percentage within 20% (Table 4). The mean percentage within 20% for Stack 1 and 2 had a 12.7% improvement (from 42.47% to 47.86% and 47.66%) compared to that for MLR. In whites, the MAEs for Stack 1 and 2 were lower than that of MLR (P = 0.002). The performance of Stack 1 and Stack2 in blacks was better than MLR, but there was no statistical significance (Table 4).
Evaluating the performance of the stacks by warfarin dose range
We next to test the predictive ability of Stack 1 and 2 in three different warfarin dose ranges. The MAE of prediction was highest in high dose and lowest in intermediate dose across the three algorithms (Table 5). In the low-dose group, Stack 1 and Stack 2 provided a significantly better prediction of dose than MLR for both MAE and mean percentage within 20% (Table 5). The mean percentage within 20% for Stack 1 was improved by 13.5% (from 22.08% to 25.05%) compared to that for MLR. Likewise, in the intermediate-dose group, Stack 1 and Stack 2 performed significantly better than MLR for both MAE and mean percentage within 20% (P < 0.001). The mean percentage within 20% for Stack 1 and 2 had an about 2.0% improvement (from 60.90% to 62.13% and 61.96%) compared with that for MLR. In the high-dose groups, the mean percentage within 20% for Stack 2 was improved by 3.0% (from 41.11% to 42.35%) compared to that for MLR (P < 0.001).
Discussion
The individual response to warfarin is highly variable, being influenced by clinical factors and genetic variants such as polymorphisms in CYP2C9 and VKORC1. Given that incorrect warfarin dosing can potentially increase the risk of thrombosis or bleeding, estimating appropriate warfarin dose for individual patient has been an active research area. Recently, personalized genotype-guided warfarin dosing has demonstrated clinical benefits and superior clinical outcomes in major clinical trials [22–24]. In these trials, predicted warfarin maintenance doses were based on MLR. However, due to the complex and non-linear association between warfarin dose and clinical and genetic variables, such limitations related to MLR-based algorithms may affect the prediction accuracy. Because of that, in this study, we set out to develop novel algorithms using the stacked generalization approach to estimate the warfarin dose, where different types of machine learning algorithms function together through a meta-machine learning model to maximize the prediction accuracy. This strategy has been successfully applied to and significantly improved the prediction of molecular atomization energies [25].
To be able to directly compare the performance among different algorithms, we used the IWPC data set and applied this data set to eight different machine learning algorithms with the same covariates in the IWPC algorithm. In line with the MAE (8.5 mg/week, 95% CI (8.0–9.0)) in the IWPC study [8], the MAE for MLR in this study was 8.53 mg/week (95% CI (8.08–8.99)). These data validated our data processing and allowed us to directly compare the performance of algorithms developed in this study to the IWPC algorithms. Moreover, we found that MLR provided a comparable performance as SV and RR and outperformed all other algorithms, including NN and GBT, which is consistent with the results reported in the IWPC study [8]. In order to take advantage of different algorithms, we constructed ensemble predictor Stack 1 and 2 using SV, RR, NN and GBT. Overall, Stack 1 and 2 performed significantly better than MLR. Subgroup analysis revealed that the mean percentage within 20% for Stack 1 and 2 had a 12.7% improvement (from 42.47% to 47.86% and 47.66%) compared to that for MLR in Asians. Similarly, the mean percentage within 20% for Stack 1 was improved by 13.5% (from 22.08% to 25.05%) compared to that for MLR in the low-dose group. The median warfarin weekly dose for Asians was lower compared to whites and blacks [21]. These data suggest that our algorithm would especially benefit patients requiring low warfarin dose, as subtle changes in warfarin dose could result in adverse clinical events such as thrombosis or bleeding.
Many studies developed the pharmacogenetic algorithms for warfarin dosing [7–11], but the performance of these algorithms can hardly be compared due to different data sets with different patient ethnicity and prevalence of CYP2C9 and VKORC1 polymorphisms. For example, we have reported that Asians, whites and blacks have different warfarin sensitivity, resulting from different polymorphisms of CYP2C9 and VKORC1 in the IWPC cohort [21]. For the IWPC data set, additional algorithms such as multivariate adaptive regression splines (MARS), lasso regression (LAR) and Bayesian additive regression trees (BART) have also been tested along with MLR [10]. Interestingly, the MAE for MLR was 9.28 mg/week in that study and they showed that BART, MARS and SVR performed superior than MLR in the prediction of warfarin dose. However, MLR and SVR performed better than MARS in the IWPC study. This discrepancy could be owing to different data processing and independent variables selected. In addition, the ensemble method “bagging” has been used to predict warfarin dose [12, 13], which is a popular method to assemble base models to decrease the variance, but not to improve the predictive force. Therefore, in terms of the coefficient of determination (R2), R2 for the IWPC algorithm in Asians is 0.46 [26], which is not inferior to the “bagging” models (Ms+G) ranging from 0.402 to 0.441 for Chinese in the previous study [13]. “Bagging” method requires designing various feature functions for base models in order to achieve diversity, which is a key requirement for base models [13]. The novel regression models we proposed here can take advantage of distinct machine learning algorithms to achieve high diversity of base models. Designing feature functions is not necessary and all base models can use the same feature space, which can avoid extra information extraction efforts. Moreover, “voting” has been combined with “bagging” to assemble the base models to predict warfarin dose [12]. In contrast to “stacking”, no learning takes place at the meta-level when combining base models by a voting scheme. Of note, “stacking” has been applied to combine different base models to predict warfarin dosing adjustment based on time-series data with patient’s history of warfarin dose and INR [27], whereas in this study we focused on the development of pharmacogenetic models to predict warfarin maintenance dose using clinical and genetic factors. Taken together, the performance of our algorithms based on the stacked generalization framework appears superior to other published models [10]. This is probably attributable to the power of our model to identify complex, nonlinear relationships among genetic and clinical variables and to deal with many sources of inferential trouble such as outliers and collinearity among variables compared to the MLR in the IWPC study.
Our study has several limitations. First, due to the missing values in the IWPC cohort, we imputed missing genotypes of VKORC1 for some patients based on linkage disequilibrium [5]. We also imputed missing values for height and weight using multivariate linear regression models. These imputation strategies, which are generally reliable, could have introduced some errors. Second, many patients required very high doses of warfarin in the IWPC cohort (> 70 mg/week), especially in warfarin normal groups. The polymorphisms explored in VKORC1 and CYP2C9 to classify warfarin sensitivity primarily explain increased sensitivity to warfarin, not increased resistance. Mutations of VKORC1 have been associated with resistance to warfarin [28, 29]. Third, our algorithms are more complex than MLR, which may not be very easy to be implemented in clinical setting.
In conclusion, we created novel regression models using the stacked generalization approach to estimate the warfarin maintenance dose. The performance of our algorithms was superior to the algorithm developed by IWPC, especially in Asians and the low-dose group. Our study offers alternative pharmacogenetic algorithms for clinical trials and practice.
Supporting information
S1 Table. The genotypes of VKORC1 in the IWPC cohort.
https://doi.org/10.1371/journal.pone.0205872.s001
(DOCX)
References
- 1. Johnson JA. Warfarin pharmacogenetics: a rising tide for its clinical value. Circulation. 2012;125(16):1964–6. Epub 2012/03/21. pmid:22431866; PubMed Central PMCID: PMCPMC3337679.
- 2. Shaw K, Amstutz U, Kim RB, Lesko LJ, Turgeon J, Michaud V, et al. Clinical Practice Recommendations on Genetic Testing of CYP2C9 and VKORC1 Variants in Warfarin Therapy. Ther Drug Monit. 2015;37(4):428–36. Epub 2015/07/18. pmid:26186657.
- 3. D'Andrea G, D'Ambrosio RL, Di Perna P, Chetta M, Santacroce R, Brancaccio V, et al. A polymorphism in the VKORC1 gene is associated with an interindividual variability in the dose-anticoagulant effect of warfarin. Blood. 2005;105(2):645–9. Epub 2004/09/11. pmid:15358623.
- 4. Joffe HV, Xu R, Johnson FB, Longtine J, Kucher N, Goldhaber SZ. Warfarin dosing and cytochrome P450 2C9 polymorphisms. Thromb Haemost. 2004;91(6):1123–8. Epub 2004/06/04. pmid:15175798.
- 5. Rieder MJ, Reiner AP, Gage BF, Nickerson DA, Eby CS, McLeod HL, et al. Effect of VKORC1 haplotypes on transcriptional regulation and warfarin dose. N Engl J Med. 2005;352(22):2285–93. Epub 2005/06/03. pmid:15930419.
- 6. Schalekamp T, Brasse BP, Roijers JF, Chahid Y, van Geest-Daalderop JH, de Vries-Goldschmeding H, et al. VKORC1 and CYP2C9 genotypes and acenocoumarol anticoagulation status: interaction between both genotypes affects overanticoagulation. Clin Pharmacol Ther. 2006;80(1):13–22. Epub 2006/07/04. pmid:16815313.
- 7. Grossi E, Podda GM, Pugliano M, Gabba S, Verri A, Carpani G, et al. Prediction of optimal warfarin maintenance dose using advanced artificial neural networks. Pharmacogenomics. 2014;15(1):29–37. Epub 2013/12/18. pmid:24329188.
- 8. International Warfarin Pharmacogenetics C, Klein TE, Altman RB, Eriksson N, Gage BF, Kimmel SE, et al. Estimation of the warfarin dose with clinical and pharmacogenetic data. N Engl J Med. 2009;360(8):753–64. Epub 2009/02/21. pmid:19228618; PubMed Central PMCID: PMCPMC2722908.
- 9. Lenzini P, Wadelius M, Kimmel S, Anderson JL, Jorgensen AL, Pirmohamed M, et al. Integration of genetic, clinical, and INR data to refine warfarin dosing. Clin Pharmacol Ther. 2010;87(5):572–8. Epub 2010/04/09. pmid:20375999; PubMed Central PMCID: PMCPMC2858245.
- 10. Liu R, Li X, Zhang W, Zhou HH. Comparison of Nine Statistical Model Based Warfarin Pharmacogenetic Dosing Algorithms Using the Racially Diverse International Warfarin Pharmacogenetic Consortium Cohort Database. PLoS One. 2015;10(8):e0135784. Epub 2015/08/26. pmid:26305568; PubMed Central PMCID: PMCPMC4549222.
- 11. Liu KE, Lo CL, Hu YH. Improvement of adequate use of warfarin for the elderly using decision tree-based approaches. Methods Inf Med. 2014;53(1):47–53. Epub 2013/10/19. pmid:24136011.
- 12. Hu YH, Wu F, Lo CL, Tai CT. Predicting warfarin dosage from clinical data: a supervised learning approach. Artif Intell Med. 2012;56(1):27–34. Epub 2012/04/28. pmid:22537823.
- 13. Tao Y, Chen YJ, Fu X, Jiang B, Zhang Y. Evolutionary ensemble learning algorithm to modeling of warfarin dose prediction for Chinese. IEEE J Biomed Health Inform. 2018. Epub 2018/07/12. pmid:29993619.
- 14. Breiman L. Stacked regressions. Mach Learn. 1996;24(1):49–64
- 15. Wang SQ, Yang J, Chou KC. Using stacked generalization to predict membrane protein types based on pseudo-amino acid composition. J Theor Biol. 2006;242(4):941–6. Epub 2006/06/30. pmid:16806277.
- 16. Wolpert DH. Stacked generalization. Neural Netw. 1992;5(2):241–59.
- 17. Ting KM, Witten IH. Stacked Generalization: When Does It Work? In Joint Conf Artif Intell. 1997:866–71.
- 18. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, et al. Scikit-learn: Machine Learning in Python. J Mach Learn Res. 2011;12:2825–30.
- 19. Ke G, Meng Q, Finley T, Wang T, Chen W, Ma W, et al. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. Adv Neural Info Proc Sys. 2017:3149–57.
- 20. Gage BF, Eby C, Johnson JA, Deych E, Rieder MJ, Ridker PM, et al. Use of pharmacogenetic and clinical factors to predict the therapeutic dose of warfarin. Clin Pharmacol Ther. 2008;84(3):326–31. Epub 2008/02/29. pmid:18305455; PubMed Central PMCID: PMCPMC2683977.
- 21. Ma Z, Cheng G, Wang P, Khalighi B, Khalighi K. Clinical Model for Predicting Warfarin Sensitivity. Submitted. 2018.
- 22. Anderson JL, Horne BD, Stevens SM, Woller SC, Samuelson KM, Mansfield JW, et al. A randomized and clinical effectiveness trial comparing two pharmacogenetic algorithms and standard care for individualizing warfarin dosing (CoumaGen-II). Circulation. 2012;125(16):1997–2005. Epub 2012/03/21. pmid:22431865.
- 23. Gage BF, Bass AR, Lin H, Woller SC, Stevens SM, Al-Hammadi N, et al. Effect of Genotype-Guided Warfarin Dosing on Clinical Events and Anticoagulation Control Among Patients Undergoing Hip or Knee Arthroplasty: The GIFT Randomized Clinical Trial. JAMA. 2017;318(12):1115–24. Epub 2017/10/04. pmid:28973620; PubMed Central PMCID: PMCPMC5818817.
- 24. Kimmel SE, French B, Kasner SE, Johnson JA, Anderson JL, Gage BF, et al. A pharmacogenetic versus a clinical algorithm for warfarin dosing. N Engl J Med. 2013;369(24):2283–93. Epub 2013/11/21. pmid:24251361; PubMed Central PMCID: PMCPMC3942158.
- 25. Wang R. Significantly Improving the Prediction of Molecular Atomization Energies by an Ensemble of Machine Learning Algorithms and Rescanning Input Space: A Stacked Generalization Approach. J Phys Chem C. 2018;122(16):8868–73.
- 26. Limdi NA, Wadelius M, Cavallari L, Eriksson N, Crawford DC, Lee MT, et al. Warfarin pharmacogenetics: a single VKORC1 polymorphism is predictive of dose across 3 racial groups. Blood. 2010;115(18):3827–34. Epub 2010/03/06. pmid:20203262; PubMed Central PMCID: PMCPMC2865873.
- 27. Martin B, Filipovic M, Rennie L, Shaw D. Using Machine Learning to Prescribe Warfarin. AIMSA 2010. p. 151–60.
- 28. Rost S, Fregin A, Ivaskevicius V, Conzelmann E, Hortnagel K, Pelz HJ, et al. Mutations in VKORC1 cause warfarin resistance and multiple coagulation factor deficiency type 2. Nature. 2004;427(6974):537–41. Epub 2004/02/07. pmid:14765194.
- 29. Scott SA, Edelmann L, Kornreich R, Desnick RJ. Warfarin pharmacogenetics: CYP2C9 and VKORC1 genotypes predict different sensitivity and resistance frequencies in the Ashkenazi and Sephardi Jewish populations. Am J Hum Genet. 2008;82(2):495–500. Epub 2008/02/07. pmid:18252229; PubMed Central PMCID: PMCPMC2427171.