Figures
Abstract
Background
The clinical decision support system(CDSS) has potential to improving medication safety. However, the effects of the intervention were conflicting and uncertain. Meanwhile, the reporting and methodological quality of this field were unknown.
Objective
The aim of this overview is to evaluate the effects of CDSS on medication safety and to examine the methodological and reporting quality.
Methods
PubMed, Embase and Cochrane Library were searched to August 2015. Systematic reviews (SRs) investigating the effects of CDSS on medication safety were included. Outcomes were determined in advance and assessed separately for process of care and patient outcomes. The methodological quality was assessed by Assessment of Multiple Systematic Reviews (AMSTAR) and the reporting quality was examined by Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).
Results
Twenty systematic reviews, consisting of 237 unique randomized controlled trials(RCTs) and 176 non-RCTs were included. Evidence that CDSS significantly impacted process of care was found in 108 out of 143 unique studies of the 16 SRs examining this effect (75%). Only 18 out of 90 unique studies of the 13 SRs reported significantly evidence that CDSS positively impacted patient outcomes (20%). Ratings for the overall scores of AMSTAR resulted in a mean score of 8.3 with a range of scores from 7.5 to 10.5. The reporting quality was varied. Some contents were particularly strong. However, some contents were poor.
Conclusions
CDSS reduces medication error by obviously improving process of care and inconsistently improving patient outcomes. Larger samples and longer-term studies are required to ensure more reliable evidence base on the effects of CDSS on patient outcomes. The methodological and reporting quality were varied and some realms need to be improved.
Citation: Jia P, Zhang L, Chen J, Zhao P, Zhang M (2016) The Effects of Clinical Decision Support Systems on Medication Safety: An Overview. PLoS ONE 11(12): e0167683. https://doi.org/10.1371/journal.pone.0167683
Editor: Robert K. Hills, Cardiff University, UNITED KINGDOM
Received: February 5, 2016; Accepted: November 18, 2016; Published: December 15, 2016
Copyright: © 2016 Jia et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: All relevant data are within the paper and its Supporting Information files.
Funding: This project was supported by the National Natural Science Foundation NO.70973083.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Various interventions using information technology (IT) have been developed to improve medication safety, and IT-based interventions such as clinical decision support system(CDSS) plays an integral role in this field[1]. According to the definition of the US Food and Drug Administration, medication safety means risk management, medication errors, and surveillance for adverse drug reactions[2]. Medication errors are recognized as the single most preventable cause of patient harm, and their reduction is of increasing importance [1]. CDSS used to promote medication safety by facilitating evidence-informed medication use[3], reducing the incidence of harmful medication errors[4], and improving healthcare system efficiency[5]. A CDSS has been defined as a computerized system that uses case-based reasoning to assist clinicians in assessing disease status, in making a diagnosis, in selecting appropriate therapy or in making other clinical decisions[6].Characteristics of individual patients are matched to a computerized knowledge base, and software algorithms generate patient-specific information in the form of assessments or recommendations[7]. CDSS can improve medication safety and reduce medication-related expenditures because it encompasses a wide range of computerized tools directed at improving patient care, including computerized reminders and advice regarding drug selection, dosage, interactions, allergies, and the need for subsequent orders[8].
Medication errors and adverse drug events (ADEs) are common costly and clinical important problems[8]. More than half a million patients are injured or die each year in hospital from adverse drug events, which may cost up to USD 5.6 million annually per hospital in America[9]. Medication errors could occur at any stage of the medication management process, including prescription, transcription, preparation and administration[10]. A review identifying that prescribing errors occur in up to 11% of all prescriptions[11]. Two inpatient studies found that medication errors occurred at rates of more than 5% and nearly half of all medication errors occurred at the stage of drug ordering [8, 12]. Analysis of medication error suggests that prevention strategies targeting systems rather than individuals are most effective in reducing errors [13]. CDSS has been widely promoted as the most promising approaches that target the ordering stage of medications, where most medication errors and preventable ADEs occur [8].
Recently, several systematic reviews (SRs) have summarized the effects of CDSS on practitioner performance and patient outcome [7, 14–17] and the evidence has been synthesized into an overview[18].The overview demonstrated modest benefits on practitioner performance and patient outcomes [10]. Meanwhile, concerns have arisen regarding the impact of CDSS on medication safety and the evidence has been synthesized into several SRs, though findings from various SRs are conflicting. For example, a CDSS team conducted three SRs to evaluate the impact of CDSS on drug prescribing and management[4], medication dosing assistants[19] and therapeutic drug monitoring and dosing[20]; Two SRs summarized the effects of CDSS on ADEs[21] and medication safety[8]. Given the growing awareness of the importance of CDSS on medication safety and the numerous SRs on this topic, it is surprising that the overviews of the effects of CDSS in this field are lacking. As reported by Brown2014[22], overview brings SRs together into one coherent document and serve as a user-friendly “digest” by evaluating and synthetizing current evidence which can be used by clinicians and policy makers in making decisions about optimal treatment, so for the first time, we set out to conduct an overview to examine CDSS interventions on medication safety in health care settings. The purpose of our study is: to evaluate the effects of CDSS on medication safety; to examine the methodological and reporting quality and to highlight areas where more research is needed.
Methods
Inclusion criteria
Types of participants.
The participants should be health care professionals such as: physicians, nurses, pharmacists, and other practitioners with responsibility for patient care. We excluded practitioners who are indirectly involved in patient care at ancillary clinical departments such as radiology, pathology departments.
Types of interventions.
The CDSS combining clinical knowledge with patient characteristics and can provide either basic (e.g., drug-allergy checking) or advanced (e.g., drug dosing support for renal insufficiency) guidance to the participants.
Outcome measures.
Within the SRs, at least one outcome relating to medication safety should be measured, such as: medication error, adverse drug events, prescribing errors, dosing errors and medication/drug related outcomes.
Selection of studies.
Only SRs were considered for inclusion. To assess whether a study was a SR, we used the checklist of Cochrane Handbook for Systematic Reviews of Interventions [23].Two reviewers independently scanned titles and abstracts to exclude obviously irrelevant studies and potentially relevant studies were investigated as full text. Disagreements were resolved by discussion with a third reviewer.
Search strategy
To identify relevant SRs, we searched the PubMed, Embase and Cochrane Library up to Aug. 2015 and elected English-only publications. Multiple keywords and Medical Subject Headings terms for CDSS were used. The search terms were customized for different databases (see S1 File: Search Strategy). No limitations were made on the outcomes. Additionally, we also handsearched reference lists and relevant reviews to identify SRs.
Data extraction and management
Two reviewers independently extracted the data. We developed a data extraction sheet, pilot-tested it on 5 randomly-selected included studies, and refined it accordingly. The following data were extracted: general information, study design, study population, intervention and main study outcomes. Separate summaries were made for the outcomes. Within these summaries, the outcomes were determined in advance and assessed separately for process of care and patient outcomes (Table 1). A process of care outcome represents quality of care, such as practitioner performance and clinical intermediate outcome. A patient outcome is directly measured patient’s health and always be endpoint outcome, such as the number of symptomatic hypoglycaemic episodes, death or bleeding[20]. Results on process of care and patient outcomes were aggregated by grading them on the strength of evidence for improvement. The evidence strength is based on the included randomized controlled trials (RCTs) in the SRs[18]:
strong evidence: results based on RCTs and effect in 50% of more of the studies;
limited evidence: results based on RCTs and effect in 40–50% of the studies; results based on both RCTs and non-randomized studies and effect in 50% of the studies.
insufficient evidence: results based on non-randomized studies or effects in less than 40% of the studies.
Quality assessment
We used two criteria sets for evaluating SRs because they have different foci. In the first stage, Assessment of Multiple Systematic Reviews (AMSTAR) was used to evaluate the methodological quality. AMSTAR is an 11-item measurement tool for the assessment of multiple systematic reviews that have good reliability and validity[24]. It was scored as “Yes”, “Partially yes”, “No”, “Can’t answer” or “Not applicable”. A criterion was defined as “Partially yes” if it was half met. For example, the fifth criterion, “A list of included and excluded studies should be provided”, was scored as “Partially yes” if either the included or the excluded studies were listed[25]. Each item is given a score of 1 if the specific criterion is met, a score of 0.5 if the criterion is partially met, or a score of 0 if the criterion is not met, is unclear, or is not applicable[24]. We summarized the final scores at three levels based on the criterion used by Beverley and Jeremy: 9 to 11 is high quality; 5 to 8 is medium quality and 0 to 4 is low quality[26].
In the second stage, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) was used to evaluate the reporting quality. The checklist consists of 27 items and focuses on ways in which authors can ensure the transparent and complete reporting of systematic reviews and meta-analyses[27]. To indicate the degree of compliance, each checklist item was assigned one of four responses: ‘Yes’ for total compliance; ‘partial’ for partial compliance; ‘No’ for noncompliance and ‘NA’ for not applicable[28].
Data analysis
We planned to conduct a meta-analysis if the treatment outcomes considered were comparable. Continuous measures were reported as mean differences and standard deviations or as standardized mean differences. Dichotomous outcomes were reported as odds ratio or rate ratio. Where studies provided sufficient data for meta-analysis, the Cochrane Collaboration’s RevMan software was used to perform a meta-analysis [23]. In studies in which outcome data were not suitable for meta-analysis, the data were described narratively.
Results
Search results
The initial search provided a pool of 9,002 SRs published in English. After eliminating based on duplicates, title and abstract, 42 SRs remained for full texts sifting and 29 finally proved to fulfil the inclusion criteria. Then two reviewers independently assessed the 29 SRs using the checklist of handbook to determine whether a publication was a SR (Table 2, Table 3). Finally, 20 SRs were included (Fig 1).
General study characteristics
The 20 SRs were published between 1994 and 2014 and conducted in diverse and mixed study settings, such as: inpatient settings[16 17 29–31], outpatient settings[4 29 30 32], ambulatory settings[17 31 33 34], hospital setting[14 29 32] and primary medical setting[29 33], involving over 256,980 health care practitioners and 1,683,675 patients. All the studies were from developed countries, of which 9 studies were undertaken in Canada[4 14–16 19 20 31 32 35], followed by the United Kingdom[29 36–39], Australia[10 17 33 34],Netherlands[40]and France[30]. In total these SRs included 629 references with 401 RCTs. All studies evaluated multiple medications: anticoagulant[16 17 30 31 34 36], theophylline or aminophylline[15 16 20 30 31 36],insulin[20 30 31],aminoglycoside[19 20 36]. Fourteen studies reported funding sources and all were supported by public funding[4 10 14 16 17 19 20 32 34–39], while six studies did not report [15 29–31 33 40](Table 4).
Quality assessment
Methodological quality.
Ratings for the overall scores of AMSTAR resulted in a mean score of 8.3 with a range of scores from 7.5 to 10.5. Based on the criterion used by Beverley and Jeremy[26]: four studies is high quality [20 30 36 38]; Sixteen is medium quality [4 10 14–17 19 29 31–35 37 39 40] and 0 to 4 no study is low quality (Table 4). Meanwhile, the 20 SRs varied in the tools used to assess the methodological quality of their including studies: 10-point scale was used by six studies[4 16 19 20 32 34]; Two SRs conducted the processes based on the criteria developed by the Cochrane Effective Practice and Organization of Care (EPOC) Group [30 39]; The Cochrane Collaboration’s tool for assessing risk of bias was used by Bennett2003[33] and Alldred2013[38] and one study used the Downs and Black tool[37] (Table 5).
Reporting quality.
Overall, none of the included 20 SRs fulfilled all 27 items of PRISMA. The “rationale for the review, study selection, definition of data items, synthesis of results, presentation of study characteristics, summary of evidence and conclusions” were well described across all SRs; The item “Identify the report as a systematic review, meta-analysis, or both” were adequately reported in 75% of the SRs. Search strategy for at least one major database was reported by 15 SRs; “Risk of bias of individual studies” were evaluated completely in 17 SRs. Total compliance with PRISMA items was less than 30.0% in structured summary (item 2, 5%), objectives (item 4, 15%), data collection process (item 10, 5%), summary measures (item 13, 25%), results of individual studies (item 20, 25%); No study described additional analyses in the methods, such as meta-regression or sub-group analyses (Table 6).
Synthesis of evidence
We planned to conduct a meta-analysis if the treatment outcomes considered were comparable. However, the variability in methods and the ways outcomes measured and presented made the generation of pooled estimates impossible. We presented the results in narrative and tabular form. The 20 SRs included 629 references, 413 of which represented unique studies, with 237 RCTs. Of these unique 237 RCTs, 178 RCTs studied process of care or patient outcomes. Nineteen of the 20 SRs examined the influence of CDSS on process of care: medication error (prescription)[4 10 17 29 31 34 36], medication error (dose)[4 10 15 16 19 20 32 34 36 37], adherence[4 17 20 31 33 35 40], medication or drug related intermediate outcome[4 10 16 19 20 30–32 36]. Evidence that CDSS significantly impacted process of care was found in 108 out of 143 unique studies of the 19 SRs that examined this effect (75%).Twelve of these 19 SRs found strong evidence that CDSS improved process of care: medication error (prescription)[4 17 31 36], medication error (dose)[4 15 19 20 37], adherence[4 20 31 35 40], medication or drug related intermediate outcome[4 19 20 30 31]; Seven of these 19 SRs found limited evidence that CDSS improved process of care: medication error (prescription)[10 29 34], medication error (dose)[10 16 32 34 36], medication or drug related intermediate outcome[10 16]; Six of these 19 SRs found insufficient evidence that CDSS improved process of care: medication error (dose)[14] adherence[17 33], medication or drug related intermediate outcome[32 36] (Table 4).
Thirteen out of the 20 SRs studied the impact of CDSS on patient outcomes. Evidence that CDSS significantly impacted patient outcomes was found in 18 out of 90 unique studies of the 13 SRs that examined this effect (20%).Only two of the 13 SRs found strong evidence that CDSS impacted patient outcomes: computer support for determining drug dose [36]and computerized medication dosing assistants [19] Three found limited evidence [16 17 29] and the remaining 8 SRs found insufficient evidence: computerized drug-lab alerts[31], computerized drug dose[15 20 30], computerized clinical decision support systems for drug prescribing[4 32 37 38] (Table 4).
Discussion
Currently, an overview focuses on the effects of CDSS on practitioner performance and patient outcome has been published[18] and AMSTAR was used to evaluated the quality of the included SRs. To the best of our knowledge, this is the first overview focus on examining the effects of CDSS on medication error which is more important and direct on patient safety, specifically the drug-related injuries[21]. It is, furthermore, the first overview assessed the quality of SRs on the effectiveness of CDSS interventions using AMSTAR and PRISMA. AMSTAR focuses on methodological quality, while PRISMA incorporates items deemed important for the transparent reporting of a SR. Both the methodological and reporting quality are important for judging the overall strength of evidence on given research questions and are considered in assessment of the quality of a research design, implementation and reporting rather than the intervention’s true effect in the process of research[41]. The study by Spyridon demonstrated that the evaluation of the reporting quality of published SRs is very useful, as it is directly related to the study’s methodology and conclusions[25].
Our recent initial search yielded 9,002 SRs, of which only 20 SRs met inclusion criteria. We planned to examine the effects of CDSS on medication safety, four studies directly targeting medication error[10 39]or adverse drug events[37 38]. The majority of our studies (16 studies) focused on the medication/drug related outcome (i.e. INR time in therapeutic range, serum concentration, time to achieve stabilization, physiological parameter change, drug dose/prescribing). This situation was true in our study, but changes in medication/drug related outcomes are also important. A study by Lainer2013 showed that the CDSS interventions consistently improve medication/drug related outcome that may provide indirect evidence for the improvement of medication safety[1]. So the influence of CDSS on medication safety may be direct, or indirect. Although a surrogate measure, changes in accordance with best practice guidelines and underpinned by evidence from high-quality would be expected to deliver improved medication safety, even if the evidence was not captured in these studies[17].
Quality of Included Studies
Methodological quality.
Our study included 20 SRs over a 20-year span and 16 of the studies demonstrated medium quality, only four studies found high quality. No increase in SR quality, with regard to fulfillment of the AMSTAR criteria, was visible over the years, a similar finding was convinced by Monique2011[18] and Seo2012[41], but Dereck [15]and Amit [16] indicated that the number and methodological quality of trials have improved over time. Three studies [4 19 20] used a “vote-counting” method to establish general conclusions. “Vote-counting” is a method that the overall results of the trials were reported by taking the number of trials with statistically significant results and dividing them by the total number of trials[19]. This method does not consider the magnitude of effects and may have underestimated the overall efficacy[20]. Hedge concluded that aggregated outcome relied upon vote counting increasing the risk of type 2 (false negative) error and formal assessment for publication bias using funnel plots was not possible with the vote-counting technique[4]. Meanwhile, a common bias, as reported by Gillaizeau2013[30] and Walton1999[36], of the included studies was that when studies were randomized by participant, the same healthcare professional may have given treatment both to intervention and control groups. It is possible that the CDSS influenced the treatment of the control groups.
Reporting quality.
The reporting quality of 20 SRs varied significantly. Some contents were particularly strong, such as the reporting of search of studies, selection of articles, and synthesis of results. However, there are also some realms need to be improved. One of the issues may be the insufficient reporting of publication bias. Sahota2011 reported that publication bias exists in this field and the presence of publication bias may be a significant confounding factor when authors are trying to aggregate results[19]. This problem results in studies with small sample sizes and non-significant effects being left out of the aggregated pool[42]. The second significant issue is the lack of quantitative analysis, such as meta-analysis, sensitivity or subgroup analyses, meta-regression. One of the possible reason may be the bulk of the qualitative literature[42]. Only 3 of the 20 SRs attempted to compute effect sizes, and all of those were in the domain of medication dose. The majority of included SRs (85%) used narrative descriptions or a “vote-counting” method to establish general conclusions. Another possible factor is the heterogeneity. In our study, the variation in the study design, clinical setting, study population, software specifications, and CDSS workflow integration led the results were not consistent among the 20 SRs, even when evaluating the same drug or the same disease[20]. The reporting flaws would affect the method quality, the integrity and accuracy of researches, so the quality of reporting still needs further improvement [28].
Synthesis of the systematic reviews results
It is clear from our synthesis that most SRs (95%) measured process of care outcomes and the results indicated that the application of CDSS would improve the outcomes. Similar findings were reported by a number of published studies [4 19 20 30 31 34 36]. In addition, our study found that CDSS that alert or reminder participants’ adherence to medication were most likely to impact the medication errors and patient care by improving the process of care outcomes. The degree to which participants adherence to the computerized medication advice will vary depending on the specificity level of computer-generated advice. As reported by Monique 2011[18], the specificity level of computer-generated advice is known to highly influence the chance that physicians adhere to the advice, with low specificity resulting in computer-advice fatigue and in situations where physicians ignore the advice. Simultaneously, consistent with previous studies[17 29 32], the results of this overview suggests that CDSS has a positive effect in changing prescribing outcomes. Alldred2013 illustrated that CDSS improved the prescribing by discontinuing inappropriate medication; commencing beneficial medicines; and ensuring appropriate monitoring of long-term conditions and medicines[38]. Consequently, this process may lead to a reduction in medication error and the improvement of medication safety.
However, while 65% of the included studies measured a patient outcome, only a small proportion demonstrated positive findings. Regarding lack of positive findings on patient outcomes, the related study by Monique 2011 showed that it was likely attributed to the small sample sizes in the original studies that consequently were underpowered[18]. Furthermore, follow-up periods in some studies were always too short to assess long-term differences on patient outcomes associated with the computerized interventions[43]. Outcomes such as hospital length of stay, death and bleeding complications will be influenced by factors other than better medication management[17]. Our study also found that CDSS for medication dosing assistants were most likely to impact the medication errors and patient care by improving the related patient outcome. An explanation for the findings may be that computers help doctors to tailor drug doses more accurately to individual patients, bringing benefits for patients and reducing the time that they spent in hospital[36]. This was also elucidated in the study by Kuperman who found that computer-generated orders are more legible than those written by hand [44]. A knowledge-based CDSS can assure that the order is safe and compliant with guidelines [45] because it introduces automation at the time of ordering, a key process in health care[44].
Strengths and Limitations
Our study has several strengths. We innovatively assessed the quality of SRs on the effectiveness of CDSS interventions using AMSTAR and PRISMA. Secondly, the literature search was comprehensive and detailed inclusion and exclusion criteria were developed to ensure transparency and reproducibility in the judgments. Thirdly, we manually searched the reference lists of the selected SRs to identify SRs that we could have missed in our literature search. In addition, the utilization of two independent reviewers for preselection of SRs, the assessment of SRs’ quality and the final data extraction would be a great help to avoid mistakes and subjective judgments.
Meanwhile, our study also has some limitations. Firstly, we were unable to use meta-analysis to pool effect sizes, given the substantial differences among the outcomes evaluated. Secondly, we defined improvement in 50% or more of the RCTs as strong evidence, in 40–50% of the RCTs as limited evidence, and in less than 40% of the RCTs or non-randomized studies as insufficient evidence respectively. The methods, combined with the strict inclusion criteria, may have underestimated the effects of CDSS intervention. Thirdly, we tried to interpret the overall effects of CDSS by aggregating all included studies, the results of high quality studies are given the same weight as low-quality studies. Fourthly, there may be some overlaps of our included studies which may have led to an overestimation the effects of CDSS. We therefore analyzed the number of unique studies from the total number included in all SRs and provided overall estimates of the evidence that CDSS significantly impacted process of care outcomes and patient outcomes [18]. Lastly, we classified the methodological quality at three levels based on the final scores of each studies, but scoring systems are controversial [46].
Future Directions
First, larger samples and longer-term studies are required to ensure more reliable evidence base on the effects of CDSS on medication error. Decision makers should balance the effects of CDSS against its burden on costs and workflow. However, the increasing number and quality of CDSS trials in the past few years and the rapid assimilation of technological information into clinical settings bode well for the future of improving the effectiveness and efficiency of clinical care[15].
Conclusions
CDSS reduces medication error by obviously improving process of care, but inconsistently improving patient outcomes. There is significant evidence that CDSS for medication alerts or reminders can positively impact process of care outcomes and CDSS related to medication dosing assistants were most likely to impact patient outcomes. The methodological and reporting quality of the included studies were varied. The changes of process care may be the most immediate effect of CDSS, nevertheless, like other health care interventions, CDSS must demonstrate benefit on patient outcomes related to drug efficacy and safety before extensively recommended into clinical care, particularly given the cost of implementing and maintaining computerization.
Supporting Information
S1 File. Search Strategy for Embase, Pubmed and Cochrane library.
https://doi.org/10.1371/journal.pone.0167683.s002
(DOCX)
Author Contributions
- Conceptualization: MMZ.
- Data curation: PLJ.
- Formal analysis: PLJ LHZ.
- Funding acquisition: MMZ.
- Investigation: PLJ JJC.
- Methodology: MMZ PLJ.
- Project administration: MMZ.
- Resources: PLJ PJZ.
- Supervision: MMZ.
- Validation: PLJ PJZ.
- Visualization: PLJ MMZ.
- Writing – original draft: PLJ.
- Writing – review & editing: PLJ MMZ.
References
- 1. Lainer M, Mann E, Sonnichsen A. Information technology interventions to improve medication safety in primary care: a systematic review. Int J Qual Health Care. 2013;25: 590–598. pmid:23771745
- 2. Montesi G, Lechi A. Prevention of medication errors: detection and audit. Br J Clin Pharmacol. 2009;67: 651–655. pmid:19594533
- 3. Teich JM, Osheroff JA, Pifer EA, Sittig DF, Jenders RA. Clinical Decision Support in Electronic Prescribing: Recommendations and an Action Plan: Report of the Joint Clinical Decision Support Workgroup. J Am Med Inform Assoc. 2005;12: 365–376. pmid:15802474
- 4. Hemens BJ, Holbrook A, Tonkin M, Mackay JA, Weise-Kelly L, Navarro T, et al. Computerized clinical decision support systems for drug prescribing and management: a decision-maker-researcher partnership systematic review. Implement Sci. 2011;6: 89. pmid:21824383
- 5.
Berner ES. Clinical decision support systems: State of the Art Rockville [June 2009]. www.ahrq.gov.
- 6.
Chiang SJ, Daniel BH. Clinical Decision Support Systems:An Effective Pathway to Reduce Medical Errors and Improve Patient Safety [January 2010]. www.intechopen.com.
- 7. Haynes RB, Wilczynski NL. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: Methods of a decision-maker-researcher partnership systematic review. Implement Sci. 2010;5: 12. pmid:20181104
- 8. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med. 2003;163: 1409–1416. pmid:12824090
- 9. Brahma D, Wahlang J, Marak M, Sangma M. Adverse Drug Reactions in the Elderly. J Pharmacol Pharmacother. 2013;4: 91–94. pmid:23761706
- 10. Manias E, Williams A, Liew D. Interventions to reduce medication errors in adult intensive care: a systematic review. Br J Clin Pharmacol. 2012;74: 411–423. pmid:22348303
- 11. Sandars JE, Aneez . The frequency and nature of medical error in primary care: understanding the diversity across studies. Fam Pract. 2003;20: 231–236. pmid:12738689
- 12. Bates DW, Cullen DJ, Laird N, Petersen LA, Small SD, Servi D, et al. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. JAMA. 1995;274: 29–34. pmid:7791255
- 13. Leape LL, Bates DW, Cullen DJ, Cooper J, Demonaco HJ, Gallivan T, et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 1995;274: 35–43. pmid:7791256
- 14. Johnston ME, Langton KB, Haynes RB, Mathieu A. Effects of computer-based clinical decision support systems on clinician performance and patient outcome. A critical appraisal of research. Ann Intern Med. 1994;120: 135–142. pmid:8256973
- 15. Dereck L, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA. 1998;280: 1339–1346. pmid:9794315
- 16. Amit X, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293: 1223–1238. pmid:15755945
- 17. Robertson J, Walkom E, Pearson SA, Hains I, Williamsone M, Newby D. The impact of pharmacy computerised clinical decision support on prescribing, clinical and patient outcomes: a systematic review of the literature. Int J Pharm Pract. 2010;18: 69–87. pmid:20441116
- 18. Monique W, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011;18: 327–334. pmid:21422100
- 19. Sahota N, Lloyd R, Ramakrishna A, Mackay JA, Prorok JC, Weise-Kelly L, et al. Computerized clinical decision support systems for acute care management: a decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implement Sci. 2011;6: 91. pmid:21824385
- 20. Nieuwlaat R, Connolly SJ, Mackay JA, Weise-Kelly L, Navarro T, Wilczynski NL, et al. Computerized clinical decision support systems for therapeutic drug monitoring and dosing: a decision-maker-researcher partnership systematic review. Implement Sci. 2011;6: 90. pmid:21824384
- 21. Wolfstadt J, Gurwitz JH, Field TS, Lee M, Kalkar S, Wu W, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med. 2008;23: 451–458. pmid:18373144
- 22. Julie B, Cindy F. Endometriosis: an overview of Cochrane Reviews. Cochrane Database Syst Rev. 2012;CD009590.
- 23.
Higgins JPT, Green S. Cochrane handbook for systematic reviews of interventions 5.1.0 [March 2011]. www.cochranehandbook.org.
- 24. Sequeira-Byron P, Fedorowicz Z, Jagannath V, Sharif M. An AMSTAR assessment of the methodological quality of systematic reviews of oral healthcare interventions published in the Journal of Applied Oral Science (JAOS). J Appl Oral Sci. 2011;19: 440–447. pmid:21986647
- 25. Papageorgiou SN, Papadopoulos MA, Athanasiou AE. Reporting characteristics of meta-analyses in orthodontics: Methodological assessment and statistical recommendations. Eur J Orthod. 2014;74–85. pmid:23493385
- 26. Beverley SJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7: 10. pmid:17302989
- 27. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gotzsche PC, Ioannidis JP, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62: e1–34. pmid:19631507
- 28. Ge L, Wang JC, Li JL, Liang L, An N, Shi XT, et al. The assessment of the quality of reporting of systematic reviews/meta-analyses in diagnostic tests published by authors in China. PLoS One. 2014;9: e85908. pmid:24465781
- 29. Schedlbauer A, Prasad V, Mulvaney C, Phansalkar S, Stanton W, Bates DW, et al. What evidence supports the use of computerized alerts and prompts to improve clinicians' prescribing behavior? J Am Med Inform Assoc. 2009;16: 531–538. pmid:19390110
- 30. Gillaizeau F, Chan E, Trinquart L, Colombet I, Walton RT, Rege-Walther M, et al. Computerized advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev. 2013;11: CD002894.
- 31. Bayoumi I, Balas MA, Handler SM, Dolovich L, Hutchison B, Holbrook A. The effectiveness of computerized drug-lab alerts: A systematic review and meta-analysis. Int J Med Inform. 2014;83: 406–415. pmid:24793784
- 32. Tawadrous D, Shariff SZ, Haynes RB, Iansavichus AV, Jain AK, Garg AX. Use of clinical decision support systems for kidney-related drug prescribing: a systematic review. Am J Kidney Dis. 2011;58: 903–914. pmid:21944664
- 33. Bennett JW, Glasziou PP. Computerised reminders and feedback in medication management: A systematic review of randomised controlled trials. Medical Journal of Australia. 2003;178: 217–222. pmid:12603185
- 34. Pearson SA, Moxey A, Robertson J, Hains I, Williamson M, Reeve J, et al. Do computerised clinical decision support systems for prescribing change practice? A systematic review of the literature (1990–2007). BMC Health Serv Res. 2009;9: 154. pmid:19715591
- 35. Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009;CD001096. pmid:19588323
- 36. Walton R, Dovey S, Harvey E, Freemantle N. Computer support for determining drug dose: systematic review and meta-analysis. BMJ. 1999;318: 984–990. pmid:10195972
- 37. Loganathan M, Singh S, Franklin BD, Bottle A, Majeed A. Interventions to optimise prescribing in care homes: systematic review. Age Ageing. 2011;40: 150–162. pmid:21262782
- 38. Alldred DP, Raynor DK, Hughes C, Barber N, Chen TF, Spoor P. Interventions to optimise prescribing for older people in care homes. Cochrane Database Syst Rev. 2013;CD009095. pmid:23450597
- 39. Keers RN, Williams SD, Cooke J, Walsh T, Ashcroft DM. Impact of interventions designed to reduce medication administration errors in hospitals: a systematic review. Drug Saf. 2014;37: 317–332. pmid:24760475
- 40. Vervloet M, Linn AJ, Weert JC, Bakker DH, Bouvy ML, Dijk L. The effectiveness of interventions using electronic reminders to improve adherence to chronic medication: a systematic review of the literature J Am Med Inform Assoc. 2012;696–704. pmid:22534082
- 41. Seo H, Kim K. Quality assessment of systematic reviews or meta-analyses of nursing interventions conducted by Korean reviewers. BMC Med Res Methodol. 2012;12: 129. pmid:22928687
- 42. Weir CJ, Lees KR, MacWalter RS, Muir KW, Wallesch CW, McLelland EV, et al. Cluster-randomized, controlled trial of computer-based decision support for selecting long-term anti-thrombotic therapy after acute ischaemic stroke. Qjm. 2003;96: 143–153. pmid:12589012
- 43. Shamliyan TA, Duval S, Du J, Kane RL. Just what the doctor ordered. Review of the evidence of impact of compurized physician order entry systems on medication errors. Health Serv Res. 2008;43: 32–53. pmid:18211517
- 44. Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, et al. Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc. 2007;14: 29–40. pmid:17068355
- 45. David W, Bates DW, Gawande Atul A. improving safety with information technology. N Engl J Med. 2003;348: 2526–2534. pmid:12815139
- 46. Fery-Lemonnier E, Landais P, Loirat P, Kleinknecht D, Brivet F. Evaluation of severity scoring systems in ICUs—Translation, conversion and definition ambiguities as a source of inter-observer variability in Apache II, SAPS and OSF. Intensive Care Med. 1995;21: 356–360. pmid:7650260