Figures
Abstract
Background
Preventive strategies to reduce clinically significant medication errors (MEs), such as medication review, are often limited by human resources. Identifying high-risk patients to allow for appropriate resource allocation is of the utmost importance. To this end, we developed a predictive model to identify high-risk patients and assessed its impact on clinical decision-making.
Methods
From March 1st to April 31st 2014, we conducted a prospective cohort study on adult inpatients of a 1,644-bed University Hospital Centre. After a clinical evaluation of identified MEs, we fitted and internally validated a multivariate logistic model predicting their occurrence. Through 5,000 simulated randomized controlled trials, we compared two clinical decision pathways for intervention: one supported by our model and one based on the criterion of age.
Results
Among 1,408 patients, 365 (25.9%) experienced at least one clinically significant ME. Eleven variables were identified using multivariable logistic regression and used to build a predictive model which demonstrated fair performance (c-statistic: 0.72). Major predictors were age and number of prescribed drugs. When compared with a decision to treat based on the criterion of age, our model enhanced the interception of potential adverse drug events by 17.5%, with a number needed to treat of 6 patients.
Conclusion
We developed and tested a model predicting the occurrence of clinically significant MEs. Preliminary results suggest that its implementation into clinical practice could be used to focus interventions on high-risk patients. This must be confirmed on an independent set of patients and evaluated through a real clinical impact study.
Citation: Nguyen T-L, Leguelinel-Blache G, Kinowski J-M, Roux-Marson C, Rougier M, Spence J, et al. (2017) Improving medication safety: Development and impact of a multivariate model-based strategy to target high-risk patients. PLoS ONE 12(2): e0171995. https://doi.org/10.1371/journal.pone.0171995
Editor: Gianpaolo Reboldi, Universita degli Studi di Perugia, ITALY
Received: February 6, 2016; Accepted: January 30, 2017; Published: February 13, 2017
Copyright: © 2017 Nguyen et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Ethical and legal restrictions prohibit the authors from making the minimal data set publicly available. Data are available from the Nîmes University Hospital Institutional Data Access / Ethics Committee for researchers who meet the criteria for access to confidential data. To request the data, contact the corresponding author (Tri-Long Nguyen) or Direction de la Recherche Clinique, CHU Nîmes, France: drc@chu-nimes.fr.
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Drug-related problems result in substantial medical and economic burden, with an estimated cost of $177.4 billion per year [1]. They are commonly defined as “events or circumstances involving drug therapy that actually or potentially interfere with desired health outcomes” [2]. These include both adverse drug events, which result in actual harm, and clinically significant medication errors (called “potential adverse drug events”), which have the potential to cause harm in absence of correction. Review of the literature suggests that medication errors (MEs) are significantly common, occurring in up to 67% of hospitalized patients [3]. Given their preventability, frequency, and potential association with adverse patient outcomes [4–6], strategies for reducing clinically significant MEs are of the utmost importance.
Several studies have highlighted the role of pharmacist-led medication review for preventing potential adverse drug events [6–16]. By closely collaborating with each other, physician and pharmacist complement one another and attempt to address patient needs. This inter-professional intervention, done in association with the medical team, consists of optimization of medication regimens after home medication reconciliation at admission, pharmaceutical analysis and review of currently prescribed drugs [6–16]. The process is very time consuming and, because of competing clinical demands, review of all inpatients by a clinical pharmacist is usually not possible. As a consequence, those at increased risk of potential adverse drug events need to be identified and considered a priority for evaluation.
Targeting “high-risk” patients might be a key aspect of successful interventions [6, 16]. Nevertheless, commonly chosen criteria, such as age or number of medications [16], often fail to identify these patients [6]. Instead of considering a single predictor strategy, we propose a multivariate model-based strategy to detect these high-risk patients. Several approaches to the prediction of actual or potential adverse drug events are described in the literature [17–21], though they vary in terms of definition. While some focus on a subcategory [17–20], others are specific to certain age groups or diagnostic populations [17, 19, 20]. Rather than focus on a specific subgroup, our approach aims to develop a generalizable tool for daily practice that includes non-invasive predictors of avoidable outcomes that are readily available at the time of hospital admission.
We report the derivation of a model predicting in-hospital significant medication errors (PRISMOR), according to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement [22, 23] and assess its clinical impact on decision-making compared to a strategy based on age.
Methods
Data collection for the study was approved by the French National Commission of Information Technology and Civil Liberties (CNIL, Commission Nationale de l’Informatique et des Libertés; agreement No. 1738666) and the Institutional Review Board of the Nîmes University Hospital Centre, which waived the need for written informed consent (No. IRB 14/01-03). Patient information were de-identified prior to analysis.
Design
We conducted a prospective cohort study in the Nimes University Hospital Centre from March 1st to April 31st, 2014, targeting a sample size that would result in 20 events per variable [24, 25]. Based on a recent systematic review reporting 45% of patients with at least one clinically significant ME [6], we estimated a required sample size of 1,000 patients. All patients over 17 years old admitted to hospital during the study period were included. Patients were followed from admission to discharge. During admission, all prescribed medications were reviewed by one of 15 clinical pharmacists, all of whom were trained in standardized medication reconciliation, prescription analysis and ME reporting.
Endpoint and predictors
The primary outcome was the occurrence of at least one clinically significant ME at any time during in-hospital stay. MEs were defined as either unintentional medication discrepancies (i.e. involuntary divergence between admission prescription and best possible medication history obtained from a minimum of 3 sources of information) or prescribing errors (i.e. non-adherence to recognized clinical prescribing guidelines). Given their more imprecise nature, two clinicians (a physician and a pharmacist) independently reviewed all MEs using the method described by Quélennec et al. [26], a European adaptation of the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Index [27]. Potential clinical impact of MEs was rated regarding patient’s clinical characteristics, medications, biological data and number of MEs. MEs were classified into 3 categories: “without potential harm (NCC MERP category C)”, “requiring potentially monitoring or intervention to preclude harm (NCC MERP category D)” and “with potential harm (NCC MERP category E and above)”. Disagreements were resolved by a third evaluator. MEs without potential harm were not considered as events of interest in the primary analysis.
For risk prediction, we extracted patient-, hospitalisation-, and prescription-related covariates from the electronic medical record. Specifically, age, sex, admission details, admission type (medical or surgical), hospital admission within the preceding 30 days, hour (day or night) and day of admission (weekday or week-end), number of prescribed drugs and drug classes (according to the Anatomical Therapeutic Chemical (ATC) classification system). We also documented whether a current treatment had been initiated before admission and whether a best possible medication history had been obtained before the admission medication order.
Statistical analysis
Univariate analyses were performed using Wilcoxon-Mann-Whitney test for continuous variables and Chi-square test for binary variables. Relations between drug ATC classes and the endpoint were explored using association rules mining. This method is useful to detect co-occurrence of an endpoint with multiple binary covariates in the absence of an a priori hypothesis [28]. The algorithm was applied assuming a minimal support (i.e. minimal probability of co-occurrence) of 5% and a minimal confidence (i.e. minimal probability of endpoint occurrence given the covariates) of 10%. ATC classes found to be significant using association rules were then included in logistic regression. Variables with a P less than 0.50 were considered eligible for inclusion in the multivariate model.
For the multivariate model development, a logistic regression was performed. Because a strict selection criterion would lead to discarding informative variables, the alpha risk threshold (P <0.30) was intentionally set as large in order to provide reliable estimates and performances [25, 29, 30]. Multivariable fractional polynomials analysis was conducted to take into account non-linearity of continuous variables. The apparent performance of the model was described using c-statistic and a calibration curve. Corrected performances were reported after bootstrapping (b = 500) for internal validation [24, 31, 32].
In order to evaluate the impact of implementing our predictive model into clinical practice we compared two strategies for pharmaceutical intervention. The first involved clinical decision-making based on age, whereas the second was based on the PRISMOR model. To this end, we conducted a series of simulated randomized controlled trials (RCTs), which allowed us to test our model under the conditions of variable pharmacist staff coverage (Fig 1). The endpoints of these trials were the interception of at least one harmful ME during the intervention and the failure to identify at least one harmful ME in the absence of intervention. We simulated the situation, in which such a model (training model) would be applied on an external sample (testing sample) [31, 32]. After fitting a similar predictive model on a bootstrap sample (training model), we randomly assigned patients from our initial sample (testing sample) into one of two groups in a 1:1 ratio. In group A (control), older patients were identified as a priority for pharmacist review whereas, in group B, patient priority was determined by the predicted probability of ME, provided by the training model. This simulation was repeated 1,000 times within five scenarios, within each the pharmacy intervention coverage was set to 10%, 30%, 50%, 70% or 90% of patients.
The analysis was reiterated 1,000 times within 5 scenarios, within each the intervention coverage k was fixed at 10%, 30%, 50%, 70% and 90%.
Analyses were conducted with R 2.15.2 (R Development Core Team. R Foundation for Statistical Computing, Vienna, Austria).
Results
A total of 1,408 patients admitted to 21 hospital units (4 surgical and 17 medical) were included. Median age was 68 years [IQR: 57–80] and 738 (52.4%) were males. Median length of stay was 4 nights [IQR: 2–8]. Baseline characteristics are reported in Table 1.
During the study period, pharmacists identified 609 MEs. Of these, 475 were considered as events of interest due to their association with harm or potential harm (Table 2). Most occurred on the first day of hospitalization (Fig 2), with a majority of unintentional discrepancies resulting from inaccurate information about current medications, which were corrected by physician after notification. Three MEs actually led to adverse drug events, due to a failure of correction at time (Table 2). During their stay, 365 (25.9%) of patients included experienced at least one clinically significant ME. Occurrence of ME was associated with 5 ATC classes: antithrombotic agents (B01), antibacterial agents for systemic use (J01), psycholeptics (N05), blood substitutes and perfusion solutions (B05) and analgesics (N02).
For clinically significant ME prediction, we fitted a multivariate logistic model composed of 11 predictors (Table 3). ME was significantly associated with the presence of a current treatment initiated before admission (OR = 5.64, 95% CI: 2.38–13.36, P< 0.001), number of prescribed drugs (OR = 1.16 per drug, 95% CI: 1.10–1.23, P< 0.001) and increasing age (P = 0.005 and 0.010 for quadratic and cubic terms, respectively). The relationship between age and ME risk followed an “n-shaped” curve (S1 Fig) indicating a maximal risk around 75 years old (OR = 4.26, 95% CI: 3.56–5.09). Risk of ME decreased significantly when medication history was completed prior to admission medication orders (OR = 0.50, 95% CI: 0.37–0.67, P< 0.001). Three predictors had associations with ME that almost achieved statistical significance: admission to a surgical ward (OR = 1.36, 95% CI: 1.00–1.87, P = 0.061), admission to hospital within the last month (OR = 0.68, 95% CI: 0.44–1.04, P = 0.067) and prescription of psycholeptic medication (ATC class N05) (OR = 1.39, 95% CI: 0.96–2.02, P = 0.084). As suggested by Steyerberg et al. [29, 30], other informative predictors were also included in the final model despite their non-significance.
Analysis of apparent performance revealed fair discrimination (c-statistic: 0.718, 95% CI: 0.689–0.748) and good calibration (intercept equal to 0 and a slope equal to 1). The local regression curve of calibration indicated a slight over-estimation of high probabilities (S2 Fig). After bootstrapping, corrected discrimination remained fair (c-statistic: 0.707) and corrected calibration suggested slight over-fitting (corrected intercept: -0.069 and corrected slope: 0.926). Regression coefficients were therefore corrected by a uniform shrinkage factor of 0.926 (Table 3).
RCT simulations (n = 5,000) demonstrated the effect of the PRISMOR model on the identification and interception of MEs when compared to identification based on age (Fig 3). In all clinical coverage scenarios, clinical pharmacists were more likely to identify MEs when the PRISMOR model was used to target high-risk patients. This effect was strongest in the 10% coverage scenario, with 17.5% mean improvement over age-based selection (statistical significance P <0.05 in 83.2% of simulations) and an absolute probability of ME interception of 50.6%. This corresponds to a number needed to treat of 6 to avoid at least one harmful ME and, when compared to the absence of clinical pharmacist evaluation, a number needed to treat of 2. Conversely, the simulation demonstrated that by using our prediction model, clinicians would have missed fewer MEs, with an effect that increased with clinical coverage. Similar analysis was conducted to compare the decision-making supported by our model versus the one based on the number of medications (S3 Fig). Although the enhancement in intercepting potential adverse drug events is more limited (mean improvement of 5.4% in the 10% coverage scenario), using our model as decision-support reduced the risk to miss clinical significant MEs in absence of intervention (absolute risk difference of -9.9%, 1 ME avoided every 10 decisions in the 90% coverage scenario).
Discussion
The incidence [3, 4, 6, 13, 33, 34] and timing [3, 4, 34, 35] of MEs in our sample is consistent with that seen in previous studies. This literature emphasizes the importance of implementing a strategy to prevent MEs early in hospitalization, ideally prior to admission medication prescription. Using predictors identified within our sample, we developed and tested a model to inform such a strategy. As expected, age and number of medications were significant predictors of MEs. Additionally, the availability of accurate information about current medications and doses prior to prescription substantially decreased the risk of MEs. In particular, a best possible medication history could have been obtained by other clinicians. As it is usually the same as that listed in the discharge summary from any recent hospitalization, hospitalization within the previous 30 days was associated with lower risk of MEs. To inform clinical triage of patients, we evaluated the potential of broadly-defined MEs to cause harm. We chose to develop a generalizable predictive model that could be implemented in everyday practice to intercept avoidable outcomes. This model was reported using international standards [22, 23] and assessed for robustness using resampling methods.
The randomized experiments showed that targeting high-risk patients with our model had the potential to substantially improve the efficiency of pharmaceutical intervention. Implementing our model in practice could enhance the identification and interception of potentially adverse drug events, a result that was consistent across levels of clinical pharmacist coverage. In all scenarios, results suggest that our model may assist clinicians to select patients who are more likely to benefit from pharmaceutical intervention. In situations of limited pharmacy coverage, our model would theoretically enable clinicians to intercept a greater number of clinically significant MEs by intervening whereas, in high coverage scenarios, fewer MEs would be missed in those patients not undergoing pharmacy review. This trend is not surprising, since greater clinical pharmacy coverage would result in a greater proportion of patients who receive intervention, including higher-risk subjects. When complete coverage is provided, targeting high-risk patients would not improve ME interception, as pharmacists already review all subjects. In contrast, focusing interventions on high-risk patients might be essential in the setting of resource limitations. Using our model-based strategy, a new clinical pharmacy service covering 10% of inpatients would avoid a potential adverse drug event every two interventions.
A small number of studies have previously reported statistical models developed to predict medication errors [17–21]. Though apparent discriminations were comparable to ours [17–21], our study improves on these through its transparent reporting of calibration and optimism [22, 23]. More importantly, we assessed the potential impact of our tool on clinical practice. In addition, because we did not restrict our study to a specific age range or field of medicine [17, 19, 20], it can be generalized to both medical and surgical inpatient settings.
Previous studies have restricted their studies to actual adverse events [17, 19, 20]; we chose to be more inclusive, including both actual and potential adverse events. As our study was based on observational (i.e., “real-life”) data, medication reviews were being conducted by clinical pharmacists as part of routine clinical practice. As such, when they were identified, mitigating action was taken to prevent medication errors which could cause harm. Had we not taken them into account, our model would have underestimated the incidence of the outcome.
Several limitations to our study need to be considered. Our model does not take into account biological markers [19] or diagnostic category [21]. Given that data consistently demonstrates that the highest risk of potential adverse drug events is on the first day admitted to hospital, we chose to exclude these potential predictors, as we felt that the most clinically valuable model would be limited to predictors that are readily available at the time of admission to hospital. Though patient comorbidities are potential predictors, the size of our derivation set limited the inclusion of these variables in the model. Having added them as independent predictors would almost certainly have resulted in over-fitting, a recognized problem to which models derived from small cohorts are prone [36–38]. Summarizing indices, such as Charlson’s index or Elixhauser’s index, would have been inappropriate given that they were developed to predict mortality, an outcome which is very different from ours [39, 40]. Moreover, recent studies have suggested that these indices have relatively poor discrimination and calibration in their prediction of nonfatal outcomes [41, 42]. Diagnoses could have been grouped in other ways; we felt it was impractical with regard to our model, however. Indeed, comorbidities were recorded in only 723 (51.3%) patient discharge summaries, which raised a real concern for the usability of a model including these predictors. In case of no recorded information (i.e. no input value), the patient’s probability of outcome would no longer be computable using the model formula.
In spite of including an appropriate number of variables for the number of outcomes [24, 25], we had to shrink initial estimates to address over-fitting [38]. That being said, bootstrap-corrected models developed based on values of 20 events per variable tend to demonstrate similar performance in validation studies [24]. Finally, our impact analysis is limited by the assumption that reducing potential adverse events would result in reduction of actual outcomes. Our preliminary results must therefore be confirmed on an independent set of patients, ideally from other hospital centers, and the usability of our predictive tool must be evaluated through a clinical impact study.
In conclusion, we developed and internally validated a model to predict potential adverse drug events as a strategy to improve pharmacist human resource allocation and subsequent patient safety. Once external validation and evaluation of concrete clinical outcomes takes place, its incorporation into clinical practice could potentially allow institutions to identify at-risk patients at their time of arrival to hospital, thus allowing for efficient, patient-specific allocation of clinical pharmacy services.
Supporting information
S1 Fig. Age effect on drug-related problem risk.
Age effect (solid line) and 95% confidence interval (dotted lines) were estimated by multivariable fractional polynomials analysis.
https://doi.org/10.1371/journal.pone.0171995.s001
(TIF)
S3 Fig. Comparison of two strategies to focus interventions on high-risk patients: decision-making supported by predictive model versus decision-making based on number of medications (1,000 simulated randomized controlled trials for each coverage value).
https://doi.org/10.1371/journal.pone.0171995.s003
(TIF)
Acknowledgments
We are thankful to the clinical pharmacy team members of the Nîmes Hospital Centre for their time and contribution: Dr. Caroline Blanco, Dr. Emmanuelle Delage, Dr. Armelle Develay-Rambourg, Dr. Harmonie Faure, Dr. Anne-Sophie Guirriec, Dr. Julie Lebrun, Dr. Mathieu Parmier, Dr. Hélène Poujol, Dr. Hélène Richard, Dr. Jérémy Rolain, Dr. Aurélie Vicens, Justine Bouquerand, Camille Favier and Tanguy Le Gall.
Author Contributions
- Conceptualization: T-LN GL-B J-MK PL.
- Data curation: T-LN GL-B J-MK PL.
- Formal analysis: T-LN YLM.
- Investigation: T-LN GL-B J-MK PL.
- Methodology: T-LN GL-B J-MK YLM PL.
- Project administration: J-MK PL.
- Resources: J-MK PL.
- Software: T-LN YLM.
- Supervision: J-MK PL.
- Validation: T-LN GL-B J-MK CR-M MR JS YLM PL.
- Visualization: T-LN GL-B J-MK CR-M MR JS YLM PL.
- Writing – original draft: T-LN GL-B PL.
- Writing – review & editing: T-LN GL-B J-MK CR-M MR JS YLM PL.
References
- 1. Ernst FR, Grizzle AJ. Drug-related morbidity and mortality: updating the cost-of-illness model. Journal of the American Pharmaceutical Association. 2001;41(2):192–9. pmid:11297331
- 2.
Pharmaceutical Care Network Europe Foundation. The PCNE Classification V 6.2. 2010.
- 3. Tam VC, Knowles SR, Cornish PL, Fine N, Marchesano R, Etchells EE. Frequency, type and clinical importance of medication history errors at admission to hospital: a systematic review. CMAJ: Canadian Medical Association journal = journal de l'Association medicale canadienne. 2005;173(5):510–5. PubMed Central PMCID: PMC1188190. pmid:16129874
- 4. Basey AJ, Krska J, Kennedy TD, Mackridge AJ. Prescribing errors on admission to hospital and their potential impact: a mixed-methods study. BMJ quality & safety. 2014;23(1):17–25.
- 5. Dean B, Schachter M, Vincent C, Barber N. Prescribing errors in hospital inpatients: their incidence and clinical significance. Quality & safety in health care. 2002;11(4):340–4. PubMed Central PMCID: PMC1758003.
- 6. Kwan JL, Lo L, Sampson M, Shojania KG. Medication reconciliation during transitions of care as a patient safety strategy: a systematic review. Annals of internal medicine. 2013;158(5 Pt 2):397–403.
- 7. Hadi MA, Alldred DP, Briggs M, Munyombwe T, Closs SJ. Effectiveness of pharmacist-led medication review in chronic pain management: systematic review and meta-analysis. The Clinical journal of pain. 2014;30(11):1006–14. pmid:24480911
- 8. Hatah E, Braund R, Tordoff J, Duffull SB. A systematic review and meta-analysis of pharmacist-led fee-for-services medication review. British journal of clinical pharmacology. 2014;77(1):102–15. PubMed Central PMCID: PMC3895351. pmid:23594037
- 9. Lee JK, Slack MK, Martin J, Ehrman C, Chisholm-Burns M. Geriatric patient care by U.S. pharmacists in healthcare teams: systematic review and meta-analyses. Journal of the American Geriatrics Society. 2013;61(7):1119–27. pmid:23796001
- 10. Martins AC, Giordani F, Rozenfeld S. Adverse drug events among adult inpatients: a meta-analysis of observational studies. Journal of clinical pharmacy and therapeutics. 2014;39(6):609–20. pmid:25219403
- 11. Santschi V, Chiolero A, Burnand B, Colosimo AL, Paradis G. Impact of pharmacist care in the management of cardiovascular disease risk factors: a systematic review and meta-analysis of randomized trials. Archives of internal medicine. 2011;171(16):1441–53. pmid:21911628
- 12. Lehnbom EC, Stewart MJ, Manias E, Westbrook JI. Impact of medication reconciliation and review on clinical outcomes. The Annals of pharmacotherapy. 2014;48(10):1298–312. pmid:25048794
- 13. Leguelinel-Blache G, Arnaud F, Bouvet S, Dubois F, Castelli C, Roux-Marson C, et al. Impact of admission medication reconciliation performed by clinical pharmacists on medication safety. European journal of internal medicine. 2014;25(9):808–14. pmid:25277510
- 14. Bladh L, Ottosson E, Karlsson J, Klintberg L, Wallerstedt SM. Effects of a clinical pharmacist service on health-related quality of life and prescribing of drugs: a randomised controlled trial. BMJ quality & safety. 2011;20(9):738–46.
- 15. Fertleman M, Barnett N, Patel T. Improving medication management for patients: the effect of a pharmacist on post-admission ward rounds. Quality & safety in health care. 2005;14(3):207–11. PubMed Central PMCID: PMC1744029.
- 16. Mueller SK, Sponsler KC, Kripalani S, Schnipper JL. Hospital-based medication reconciliation practices: a systematic review. Archives of internal medicine. 2012;172(14):1057–69. PubMed Central PMCID: PMC3575731. pmid:22733210
- 17. Onder G, Petrovic M, Tangiisuran B, Meinardi MC, Markito-Notenboom WP, Somers A, et al. Development and validation of a score to assess risk of adverse drug reactions among in-hospital patients 65 years or older: the GerontoNet ADR risk score. Archives of internal medicine. 2010;170(13):1142–8. pmid:20625022
- 18. Pippins JR, Gandhi TK, Hamann C, Ndumele CD, Labonville SA, Diedrichsen EK, et al. Classifying and predicting errors of inpatient medication reconciliation. Journal of general internal medicine. 2008;23(9):1414–22. PubMed Central PMCID: PMC2518028. pmid:18563493
- 19. Tangiisuran B, Scutt G, Stevenson J, Wright J, Onder G, Petrovic M, et al. Development and validation of a risk model for predicting adverse drug reactions in older people during hospital stay: Brighton Adverse Drug Reactions Risk (BADRI) model. PloS one. 2014;9(10):e111254. PubMed Central PMCID: PMC4214735. pmid:25356898
- 20. Trivalle C, Burlaud A, Ducimetière P. Risk factors for adverse drug events in hospitalized elderly patients: A geriatric score. European Geriatric Medicine. 2011;2(5):284–9.
- 21. Urbina O, Ferrandez O, Grau S, Luque S, Mojal S, Marin-Casino M, et al. Design of a score to identify hospitalized patients at risk of drug-related problems. Pharmacoepidemiology and drug safety. 2014;23(9):923–32. pmid:24817497
- 22. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Bmj. 2015;350:g7594. pmid:25569120
- 23. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Annals of internal medicine. 2015;162(1):W1–73. pmid:25560730
- 24. Austin PC, Steyerberg EW. Events per variable (EPV) and the relative performance of different strategies for estimating the out-of-sample validity of logistic regression models. Statistical methods in medical research. 2014.
- 25. Steyerberg EW, Eijkemans MJ, Harrell FE Jr., Habbema JD. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. Statistics in medicine. 2000;19(8):1059–79. pmid:10790680
- 26. Quelennec B, Beretz L, Paya D, Blickle JF, Gourieux B, Andres E, et al. Potential clinical impact of medication discrepancies at hospital admission. European journal of internal medicine. 2013;24(6):530–5. pmid:23514919
- 27.
National Coordinating Council for Medication Error Reporting and Prevention. NCC MERP Index for Categorizing Medication Errors. 2001.
- 28.
Agrawal R, Srikant R. Fast algorithms for mining association rules. Research Report RJ 9839, IBM Almaden Research Center, San Jose, California. 1994.
- 29. Steyerberg EW, Eijkemans MJ, Habbema JD. Stepwise selection in small data sets: a simulation study of bias in logistic regression analysis. Journal of clinical epidemiology. 1999;52(10):935–42. pmid:10513756
- 30.
Steyerberg EW. Clinical prediction models: A practical approach to development, validation, and updating. New York, NY: Springer; 2009.
- 31. Steyerberg EW, Harrell FE Jr., Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. Journal of clinical epidemiology. 2001;54(8):774–81. pmid:11470385
- 32. Efron B, Gong G. A leisurely look at bootstrap, jacknife and cross-validation. The American Statistician. 1983;37(1):36–48.
- 33. Gleason KM, McDaniel MR, Feinglass J, Baker DW, Lindquist L, Liss D, et al. Results of the Medications at Transitions and Clinical Handoffs (MATCH) study: an analysis of medication reconciliation errors and risk factors at hospital admission. Journal of general internal medicine. 2010;25(5):441–7. PubMed Central PMCID: PMC2855002. pmid:20180158
- 34. Vira T, Colquhoun M, Etchells E. Reconcilable differences: correcting medication errors at hospital admission and discharge. Quality & safety in health care. 2006;15(2):122–6. PubMed Central PMCID: PMC2464829.
- 35. Cornish PL, Knowles SR, Marchesano R, Tam V, Shadowitz S, Juurlink DN, et al. Unintended medication discrepancies at the time of hospital admission. Archives of internal medicine. 2005;165(4):424–9. pmid:15738372
- 36. Harrell FE Jr., Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Statistics in medicine. 1996;15(4):361–87. pmid:8668867
- 37. Steyerberg EW, Bleeker SE, Moll HA, Grobbee DE, Moons KG. Internal and external validation of predictive models: A simulation study of bias and precision in small samples. Journal of Clinical Epidemiology. 2003;56(5):441–7. pmid:12812818
- 38. Steyerberg EW, Eijkemans MJC, Harrell FE, Habbema JDF. Prognostic Modeling with Logistic Regression Analysis: In Search of a Sensible Strategy in Small Data Sets. Medical Decision Making. 2001;21(1):45–56. pmid:11206946
- 39. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. Journal of chronic diseases. 1987;40(5):373–83. pmid:3558716
- 40. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Medical care. 1998;36(1):8–27. pmid:9431328
- 41. Antoniou T, Ng R, Glazier RH, Kopp A, Austin PC. Comparison of comorbidity classification methods for predicting outcomes in a population-based cohort of adults with human immunodeficiency virus infection. Annals of epidemiology. 2014;24(7):532–7. pmid:24837611
- 42. Austin PC, Stanbrook MB, Anderson GM, Newman A, Gershon AS. Comparative ability of comorbidity classification methods for administrative data to predict outcomes in patients with chronic obstructive pulmonary disease. Annals of epidemiology. 2012;22(12):881–7. PubMed Central PMCID: PMC4617831. pmid:23121992