Figures
Abstract
Background
Older patients are at high risk for experiencing Adverse Drug Events (ADEs) during hospitalization. To be able to reduce ADEs in these vulnerable patients, hospitals first need to measure the occurrence of ADEs, especially those that are preventable. However, data on preventable ADEs (pADEs) occurring during hospitalization in older patients are scarce, and no ‘gold standard’ for the identification of ADEs exists.
Methodology
The study was conducted in three hospitals in the Netherlands in 2007. ADEs were retrospectively identified by a team of experts using a comprehensive and structured patient chart review (PCR) combined with a trigger-tool as an aid. This ADE identification strategy was applied to a cohort of 250 older hospitalized patients. To estimate the intra- and inter-rater reliabilities, Cohen’s kappa values were calculated.
Principal Findings
In total, 118 ADEs were detected which occurred in 62 patients. This ADE yield was 1.1 to 2.7 times higher in comparison to other ADE studies in older hospitalized patients. Of the 118 ADEs, 83 (70.3%) were pADEs; 51 pADEs (43.2% of all ADEs identified) caused serious patient harm. Patient harm caused by ADEs resulted in various events. The overall intra-rater agreement of the developed strategy was substantial (κ = 0.74); the overall inter-rater agreement was only fair (κ = 0.24).
Conclusions/Significance
The ADE identification strategy provided a detailed insight into the scope of ADEs occurring in older hospitalized patients, and showed that the majority of (serious) ADEs can be prevented. Several strategy related aspects, as well as setting/study specific aspects, may have contributed to the results gained. These aspects should be considered whenever ADE measurements need to be conducted. The results regarding pADEs can be used to design tailored interventions to effectively reduce harm caused by medication errors. Improvement of the inter-rater reliability of a PCR remains challenging.
Citation: Klopotowska JE, Wierenga PC, Stuijt CCM, Arisz L, Dijkgraaf MGW, Kuks PFM, et al. (2013) Adverse Drug Events in Older Hospitalized Patients: Results and Reliability of a Comprehensive and Structured Identification Strategy. PLoS ONE 8(8): e71045. https://doi.org/10.1371/journal.pone.0071045
Editor: Heiner K. Berthold, Charité University Medicine Berlin, Germany
Received: November 12, 2012; Accepted: June 28, 2013; Published: August 5, 2013
Copyright: © 2013 Klopotowska et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This work was supported by The Netherlands Organization for Health Research and Development (ZonMW grant SG300001). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Patient harm due to medication, also known as Adverse Drug Events (ADEs), is the second most frequent complication occurring during hospitalization [1]–[4]. Between 6 and 30% of hospitalized patients experience an ADE during their hospitalization [5]. A recent Dutch study estimated that on average, an ADE can result in an excess length of stay of 6.2 days and additional costs of €2,507 [6]. Older patients are at higher risk for ADEs [7]. This higher risk is often related to the presence of multimorbidity and related polypharmacy [8], as well as age-related changes in pharmacokinetics and pharmacodynamics that influence drug elimination and response [9].
Because of ageing and an increasing life expectancy [10], the reduction of ADEs in these vulnerable patients has become a major patient safety goal in various settings [11], [12].
Furthermore, as stated by the Expert Group on Safe Medication Practices, proper steps should be taken to establish appropriate methods to identify ADEs, with the aim of evaluating the effect of medication safety practices and initiatives intended to reduce ADEs [13]. Preventable ADEs (pADEs) should be specifically addressed, i.e., harm caused by medication errors [14], [15]. However, data on pADEs occurring in older patients during hospitalization are scarce [1], [16]–[19]. Furthermore, the type of ADE identification method used can have consequences for the insight gained into the occurrence of ADEs in a specific setting. The lesser that is known about the extent of local medication safety risks, the more comprehensive the ADE identification method used should be [20].
A patient chart review (PCR) has been a widely used method because of its high ADE yield and its specificity in detecting pADEs [20]–[27]. This method can be used prospectively or retrospectively. Usually, nurses, physicians, or pharmacists review hospital data such as medical and nursing notes, medication charts, and laboratory results to search for events that could have been caused by medication. When identified, such events are assessed, usually by physicians and/or pharmacists, for the drug causality, severity, and preventability, and this assessment takes into account all available and relevant information on the considered case [28], [29]. Especially when assessing complex cases, such as those of older patients, the use of clinical judgment is mandatory. However, PCR is time-consuming, ADEs can be easily missed because of the often unstructured review process, and its implicit character often results in low inter-rater reliability [21], [30], [31].
To circumvent these limitations, use of explicit screening tools is advocated [20], [30], [32]. The Institute for Healthcare Improvement (IHI) developed an explicit screening tool specifically for the detection of ADEs in hospitalized patients [33]. This so called ‘trigger-tool’ is a list of sentinel words, for example specific medications or abnormal laboratory values, that could indicate the occurrence of an ADE [23], [30]. If a trigger is found in a patient chart, this chart is then further reviewed by experts to assess whether an ADE has occurred. The use of this trigger-tool has been shown to improve the sensitivity as well as the reliability of ADE measurements [20], [30], [32]. However, the IHI ADE trigger-tool was only validated in the general inpatient population in the United States, and developed to measure the overall level of harm [30], [33]. A study by Franklin et al. [34] showed that when IHI ADE trigger-tool was applied in a British hospital, whole classes of pADEs were missed.
Therefore, when prompted with the need to conduct first-time measurement of ADE occurrence in older patients during hospitalization, we chose a PCR as the basis, structured it, and combined it with the IHI ADE trigger-tool [33] as an aid. Our aim was to gain a more comprehensive and reliable ADE measurement. In this paper, we present results obtained by this ADE identification strategy when applied in a cohort of 250 older hospitalized patients. Additionally, the intra-rater and inter-rater reliability were evaluated.
Methods
Ethics Statement
The integral WINGS study protocol [35] was presented to The Medical Ethics Committee of the University of Amsterdam. The Medical Ethics Committee discussed the protocol and exempted it from review and official approval. According to the Dutch Medical Research Involving Human Subjects Act (WMO), such review and approval was not required because the study did not involve direct interaction with human subjects. This research used retrospective data to assess the occurrence of ADEs. Therefore, the integrity of the patients was not influenced and all patient data were analyzed anonymously by coding every included patient with a 6-digit number.
Participants and Setting
For the purpose of this study, 250 patients from the baseline measurement of ‘Ward-oriented pharmacy In Newly admitted Geriatric Seniors’ (WINGS) study were included [35]. The patients were 65 years and older, used five or more medications upon admission and were consecutively admitted to Internal Medicine wards of the Academic Medical Centre, Amsterdam (university teaching hospital with 1003 beds) for 24 hours or longer between April and November 2007. Patients were excluded if they were admitted for planned chemotherapy, radiation therapy or transplantation, or if they were transferred from other non-medical wards within the same hospital or other hospitals.
The WINGS study was conducted within the CAREFUL (pharmacist Coordinated ADE Reducing Efforts For Use in all Levels of healthcare) research program based on a cooperation of Leiden University Medical Centre, Academic Medical Centre Amsterdam, University Medical Centre Groningen and University Medical Centre Utrecht/Utrecht University, The Netherlands.
ADE Identification Strategy
The overall criteria for the ADE identification strategy were as follows: 1) the strategy should have a high ADE yield in order to gain comprehensive insight into the occurrence and types of ADEs, 2) it should account for the causality, severity, and preventability of ADEs, and 3) it should reliably measure potential changes in ADE occurrence as a result of future medication safety interventions.
We addressed the concerns about the limitations of a PCR in four ways.
First, to assure a structured and reliable data collection and ADE assessment, we developed a Case Report Form (CRF) that was based on previous research on adverse events [36], drafted a manual for this CRF, provided training in data collection for nurses and pharmacy students involved in data collection in this study, and provided training in PCR and ADE assessment for expert team members.
Second, to assure comprehensive data collection [13], we incorporated the IHI ADE trigger-tool [33] into the CRF. However, we did not use this trigger-tool for patient pre-selection because of aforementioned limitations. In contrast, in the present study, patient charts in which triggers were detected, as well as patient charts in which no triggers were detected, underwent further review by the expert team. Also, several modifications to the IHI ADE trigger-tool [33] were necessary to account for drug formularies and laboratory tests references prevailing in the Netherlands (Table S1).
Third, no restrictions were made regarding medication process stages at which ADEs can occur, and broad and internationally accepted definitions were applied.
Fourth, we assigned a multidisciplinary team consisting of both a physician and a pharmacist to identify and assess ADEs because their knowledge and experience is complementary [37], [38]. Therefore, such an approach could increase the ADE yield of the ADE identification strategy.
CRF and data collection.
The CRF included the following components:
- general patient characteristics,
- laboratory findings including therapeutic drug monitoring findings: a spreadsheet to chronologically register any decreased or increased laboratory values,
- diagnostic procedures: a spreadsheet to chronologically register any diagnostic procedures such as gastroscopy, echocardiograms, microbiology findings, consultations by other medical specialists,
- allergies,
- reason(s) for hospital admission and when applicable in-hospital transfers to intensive care or medium care units,
- medication (home, hospital, discharge, as needed medication),
- symptoms/complaints upon admission and during hospitalization: a spreadsheet to chronologically register pre-defined symptoms/complaints such as bleeding, hypotension, constipation, skin rash,
- (differential) diagnoses upon admission and during the hospital stay,
- when applicable, information regarding readmission within three months of the reviewed hospitalization.
Using the manual, trained research nurses and pharmacy students gathered data about the index hospitalization of the included patients and completed CRFs based on these data. The triggers included in the modified IHI ADEs trigger-tool were checked off by the research nurses or pharmacy students when they were detected. As mentioned before, patient charts in which triggers were detected, as well as charts in which no triggers were detected, underwent further review by the independent expert team.
An additional information source about medication use were print-outs of drug safety alerts overridden by the physicians while prescribing. In all participating hospitals computerized physician order entries (CPOEs) were operational. All CPOEs used in Dutch hospitals use the Dutch drug database ‘G-standaard’, which contains safety information on all drugs registered in the Netherlands, including drug-drug interactions, duplicate orders and dosing guidelines. It also provides standardized alert texts, i.e., a Clinical Decision Support (CDS) [39], [40]. When physicians enter prescriptions, alerts are shown on screen intrusively. Alerts show the drugs involved, their dosage regimens, and an explanation including a recommendation. Overridden alerts are logged for hospital pharmacy review [40]. This review is usually conducted by a hospital pharmacist once daily. If an overridden drug safety alert is judged by a hospital pharmacist as clinically relevant, he or she contacts by telephone either a nurse or a physician to provide additional advice. This advice can either be accepted, or rejected based on personal clinical opinion. When changes in pharmacotherapy are required, usually the physician changes the drug orders in the CPOE.
Definitions.
An Adverse Drug Event (ADE) was defined as any harmful event occurring during drug therapy and resulting either from appropriate care (nonpreventable ADEs) or from medication errors (preventable ADEs) [41]. ADEs resulting in clinical symptoms (with or without abnormal laboratory values), as well as ADEs resulting in abnormal laboratory values only, were included. In the present study we focus on ADEs occurring during hospitalization.
The causality between a drug and an adverse event was assessed using an adapted causality assessment of the World Health Organization – Uppsala Medical Centre (WHO-UMC) [42]. Adaptation of the WHO-UMC causality assessment was necessary to be able to also assess ADEs due to drug omissions, since only drug intake (i.e. drug commission) is assumed as the starting point. The following modifications were made: option ‘drug omission’ was added whenever ‘drug intake’ was stated, option ‘drug initiation’ was added whenever ‘drug withdrawal’ was stated, and rechallenge could be either initiating a drug previously withdrew or withdrawing a drug previously initiated.
The severity of ADEs was scored according to the Common Terminology Criteria for Adverse Events version 3.0 (CTCAEv3) developed by the U.S. National Cancer Institute, which includes the following 5-points scale of seriousness: mild, moderate, severe, life-threatening, and fatal [43]. In the present study no restrictions were made regarding the ADE severity.
An ADE was judged as preventable if caused by a medication error. Medication errors included both errors of commission or omission [41]. Medication errors were classified according to the Dutch Central Medication Incidents Registration [44] and were defined as errors in drug prescribing, dispensing, administration, or monitoring [44]. To assess whether a medication error occurred, we utilized prevailing national [45] and local pharmacotherapy guidelines.
ADE assessment.
The independent expert team consisted of a senior physician specializing in internal medicine (LA) and a senior clinical pharmacist specializing in geriatric medicine (CS), both of whom are well-acquainted with the PCR methodology.
Both experts received all data gathered about the index hospitalization and used the completed CRFs to guide them through the chart review. First, the two experts reviewed all data independently and recorded what in their opinion could be ADEs. Subsequently, they presented their findings during scheduled meetings. Events discussed were either events identified only by the physician, or only by the pharmacist, or identified by both experts. For each event brought forward during scheduled meetings, the experts shared their knowledge on the specific case, and conducted a joined causality assessment according to adapted WHO-UMC criteria. In the present study, only events assessed as having possible, probable, or nearly certain causality with drug commission or omission were registered. These ADEs were further assessed by the experts on severity, preventability, and when applicable, on type of medication error. In cases where there was initial disagreement between the experts during joint ADE assessment, this disagreement was solved by a consensus between the experts.
A flowchart of ADE identification and assessment strategy is shown in Figure 1. No attempt was made to assess sensitivity or specificity of this strategy, as a clear “gold standard” for ADE identification is lacking [20], [22]–[27].
1 CPOE-CDS - Computer Physician Order Entry with Clinical Decision Support. 2 ADEs - Adverse Drug Events.
Reliability Assessment
The reliability of the developed ADE identification strategy was assessed by calculating the intra- and inter-rater agreements for overall ADE occurrence, causality, severity, and preventability. The intra-rater agreement was defined as the test-retest of ADE judgments following a repeated ADE identification by the assigned expert team (LA and CS combined as a team compared at two different time points). For this purpose, a random sample of 10% of the included patients (25 cases) was re-reviewed by the assigned expert team one year after their first review in 2007. The inter-rater agreement was defined as the test-retest reliability of the ADE judgments following repeated ADE identification by a different physician-pharmacist expert team, which consisted of a senior physician specializing in internal medicine (HA) and a senior hospital pharmacist (PK), i.e., LA and CS combined as a team versus HA and PK combined as a team. Prior to the reviews, the second pair of experts received training in the method applied in the present study. Except for one accidental replacement, the same random sample of 10% of the included patients (25 cases), which was reviewed by CS and LA in 2007, was also reviewed by the second expert team (HA and PK).
Statistical Methods
The chi-square test was used to compare the proportions of pADEs and npADEs. To estimate the intra- and inter-rater reliabilities for judgments on the occurrence of ADEs, we calculated Cohen’s kappa (κ; with 95% confidence interval) and the corresponding percentages of positive and negative agreement (single measure agreement). Furthermore, inter-rater and intra-rater agreements were also calculated for judgments on the causality, preventability, and severity of ADEs. A kappa value of 0.00 was considered to be poor agreement, 0.01–0.20 to be slight agreement, 0.21–0.40 to be fair agreement, 0.41–0.60 to be moderate agreement, 0.61–0.80 to be substantial agreement, and 0.81–1.00 was considered to be almost perfect agreement [46]. In the absence of a prior sample size calculation for this sub-study of the WINGS study [35], the results are reported at the descriptive level. A P-value smaller than 0.05 was considered statistically significant. All analyses were performed with SPSS version 18.0 (Chicago Illinois) and PEPI (Pairsetc) version 1.11 (Copyright J.H. Abramson 2003–4).
Results
The mean age of the 250 included older hospitalized patients was 76.9 years (SD±7.5). Most patients were acutely admitted to the hospital (85.2%; Table 1). Upon admission, these patients were taking an average of 7.3 medications (SD±3.2), and during hospitalization, they were taking an average of 11.0 medications (SD±4.1).
Using the ADE identification strategy, the expert team (LA and CS) identified a total of 118 ADEs, yielding a rate of 47.2 ADEs per 100 hospitalizations (Table 2). These 118 ADEs occurred in 62 patients; 83 ADEs (70.3%) were assessed as preventable, of which 51 (43.2% of all ADEs identified) caused serious patient harm (severe, life-threatening, or fatal patient harm). The 83 pADEs occurred in 37 patients. Overall, pADEs were more serious in comparison to npADEs (p = 0.001; Table 2). The majority of ADEs (92 ADEs; 78.0%) was assessed as having nearly certain causality. Only 43 ADEs (36.4%) of which 25 (58.1%) serious ADEs were related to a trigger included in the modified IHI ADE trigger-tool, while of the 75 ADEs not related to the triggers (i.e. those identified by chart review only), 35 ADEs (46.7%) were serious (p = 0.230). Of the 118 ADEs, 32 ADEs (27.1%) were solely identified by the physician (LA), 28 ADEs (23.7%) solely by the pharmacist (CS), and 58 ADEs (49.2%) were identified by both the physician and the pharmacist.
Various types of ADEs were identified, and the patterns in frequencies differed between npADEs and pADEs (Table 3). Metabolic/laboratory events were the most common clinical manifestation related to ADEs (30.5%), followed by infections (11.0%), and coagulation events (10.2%). Of these most common ADE types, the majority (70.5%) was classified as pADEs. Table 4 shows that pADEs resulting in metabolic/laboratory events were mainly related to use of contra-indicated medications (10 of 24 pADEs), causing hypo- or hyperglycemia in case of blood glucose lowering drugs, or hypo- or hyperkalemia in case of diuretics or Renin-Agiotensin-Aldosterone-System (RAAS) inhibitors. Most pADE infection events (8 of 12 pADEs) were cases of inappropriate choice of empirical antibiotic therapy resulting in delayed recovery, or no clinical improvement of infections. The coagulation pADEs, were all raised INR events caused mainly (5 of 7 pADEs) by interaction between vitamin K antagonists and antibiotics/antifungal medication. Omissions of iron supplements in patients with chronic cardiovascular disease who were hospitalized due to (excessive) blood loss, in whom anemia was not (sufficiently) corrected, corresponded to 6 out of 7 blood/bone marrow pADEs. Omissions of laxatives in patients taking opiates causing constipation or ileus corresponded to 4 of 10 gastrointestinal pADEs.
Prescribing contra-indicated medications, undertreatment, drug-drug interactions, and inappropriate choice are all sub categories of prescribing errors, the leading cause of pADEs in this study (85.5%). Preventable ADEs due to errors in administration or monitoring stage were rarely identified (Table 4). No pADEs due to medication errors in the dispensing stage were detected. Medication associated with the 83 pADEs was clustered in several therapeutic classes. The top five medication classes involved were: antibacterials for systemic use (22 pADEs, 26.5%), blood glucose lowering drugs including insulin (8 pADEs, 9.6%), diuretics (7 pADEs, 8.4%), antithrombotic agents (7 pADEs, 8.4%), and iron preparations (6 pADEs, 7.2%).
Diuretics, lipid modifying agents, beta-blocking agents, cardiac glycosides, RAAS-inhibitors, and antiadrenergic agents grouped as ‘drugs acting on cardiovascular system’ were together related to 20 pADEs (24.1%).
Reliability Assessment
In nine out of the 25 patients who were randomly selected to measure the intra-rater reliability, the assigned expert team (LA and CS) identified no ADEs during their initial review in 2007. In the remaining 16 patients, 34 ADEs were identified (ten ADEs related to triggers and 24 ADEs detected by chart review only). During their second review, conducted one year later, the assigned expert team again identified all ten ADEs related to triggers from their initial review. However, two ADEs that were detected by chart review only were missed, and two additional ADEs that were detected by chart review only were identified. In Table 5, the calculated Cohen’s kappa values along with the percentages of positive and negative agreement are shown. The intra-rater agreement regarding the presence of an ADE was substantial (κ = 0.74). The intra-rater agreement regarding the causality, preventability and severity of the agreed-upon ADEs ranged from substantial to almost perfect (κ = 0.67 to κ = 0.93).
Table 5 also shows the inter-rater agreement results. In the set of 25 randomly selected patients used for the inter-rater measurement (24 patients from the set used for the intra-rater measurement in 2007 plus one new randomly selected patient), the assigned expert team (LA and CS) identified 37 ADEs (of which 10 ADEs were related to triggers) during their initial review, and the second pair of reviewers (HA and PK) identified 36 ADEs (of which 10 ADEs were related to triggers). The two teams of experts agreed upon 29 ADEs and disagreed upon 15 ADEs. Of the total of 11 ADEs that were related to triggers, the teams agreed upon nine. The inter-rater agreement on overall ADE occurrence, and preventability was fair (κ = 0.24 and κ = 0.38, respectively), and agreement was only slight for the causality (κ = 0.19). The inter-rater agreement for ADE severity was almost perfect (κ = 0.85).
Discussion
By applying the ADE identification strategy to a cohort of older hospitalized patients, the assigned independent expert team was able to comprehensively identify various types of ADEs, of which the majority was preventable and/or resulted in serious patient harm. The strategy proved to be reliable if used by one team of experts.
In comparison to most ADE studies in older hospitalized patients [1], [17]–[19], [47]–[51], the ADE rate in the present study was 1.1 to 2.7 times higher. This high ADE yield seems to result not merely from a high number of mild and moderate ADEs. In contrast, of the 118 ADEs identified, 60 ADEs (50.8%) resulted in serious patient harm. We identified only one study, by Egger et al. [16], that reported a higher rate of ADEs during hospitalization, and their rate was 133.1 ADEs per 100 hospitalizations. The results gained by the ADE identification strategy may be explained by several methodological aspects, as well as specific aspects of the setting or the study itself.
Methodological Aspects
In contrast to the present study, none of the studies cited above [1], [16]–[19], [47]–[51] used the IHI ADE trigger-tool [33] as an aid during PCR to enhance ADE identification. The trigger-tool methodology has been shown to increase the ADE yield merely because the review process is more structured, and the reviewers are more focused [32]. However, given that only 36.4% of ADEs identified in the present study were detected by the modified IHI ADE trigger-tool, a pre-selection of patients based on this tool would have substantially underestimated the occurrence of (serious) ADEs in our cohort of older hospitalized patients. In other words, the time invested in reviewing all included patients according to our strategy was worthwhile. This finding is in line with others [34] and raises concerns regarding the use of trigger-tools without prior validation in specific settings or in specific patient populations.
Up to now, several studies using computerized ADE surveillance have been published [52], [53]. However, specific for older inpatients, the use of such computerized tool as an aid for ADE identification was only reported by Egger et al. [16]. In this study, a computerized drug database was used along with PCRs by an expert team. This database automatically detected possible drug-drug interactions and drug side effects. In total, 64 out of 163 ADEs were identified by database signals, of which the majority (45 ADEs, 70.3%) were drug side effects signals. In contrast, in the present study the Dutch drug safety database [39] was incorporated in CPOEs and not used as a separate ADE surveillance tool. Also, it did not generate alerts related to drug side effects. These differences may explain the higher ADE yield in the study by Egger et al. [16] in comparison to the present study; database validation results were not provided by the authors.
The prospective data collection applied by Egger et al. [16] may have additionally contributed to their results. During prospective data collection, the patients included in a study are intensively monitored by researchers for ADEs occurring during hospitalization. Nurses and physicians working on the wards can be approached to clarify or report events and/or to present additional information [54].
Another explanation for the high ADE yield in the present study may be the assignment of both a physician and a pharmacist as expert reviewers. These two experts reviewed all patient charts to identify ADEs and assessed all identified ADEs for the causality, severity, and preventability. Especially in the often clinically complex cases of older patients, a physician reviewer is mandatory. However, the pharmacotherapy and medication safety knowledge of a hospital/clinical pharmacist is also of essential and added value. Phansalkar et al. [37] showed that pharmacist reviewers identified almost twice as many ADEs as non-pharmacists (p = 0.003). Only Egger et al. [16] appointed, similar to the present study, a multidisciplinary team consisting of physicians and a pharmacist to conduct PCRs and to assess ADEs.
In the present study, all types of medication errors, including omission of medications (i.e. undertreatment), were considered. The latter category was not taken into account by other ADE studies in older hospitalized patients [1], [16]–[19], [47]–[51]. Yet, undertreatment is common in older patients [55], and in the present study, 22.9% of pADEs were related to this type of prescribing error. In line with ADE studies in older [1], [16]–[19], [47]–[51] and other patient populations [14], [56], [57], the findings of the present study show that most pADEs (85.5%) were a result of medication errors occurring at the stage of prescribing. This could be expected, as a PCR is especially suitable for detecting prescribing errors and is less suitable for detecting medication errors in other medication processes [58]. However, prescribing errors are the most clinically relevant medication errors because they more often result in patient harm in comparison to errors in other medication processes [14].
Specific Aspects of the Setting/Study
Prescribing contra-indicated medications, undertreatment, and dosing errors were the most frequent prescribing errors in the present study. These errors occurred despite CPOEs with CDS (CPOEs-CDS) operating in the participating hospitals. CPOEs-CDS have shown to be effective tools in reducing pADEs [59]. However, to capture the above-stated prescribing errors, CPOEs-CDS should combine data on diagnosis, medication use/underuse, and laboratory findings, and should provide specific dosing advices for older patients. Such advanced systems are not yet fully developed [59], [60]. On the other hand, having CPOEs-CDS in place in the participating hospitals may explain why only 9.9% of pADEs occurring at the stage of prescribing was a result of drug-drug interactions and duplicate medication orders errors. The fact that this type of pADEs still occurred is probably related to frequent overriding of the alerts by the physicians or to alert fatigue due to low specificity of the alerts [61]. The review of overridden alerts by hospital pharmacists seems also a weak barrier.
Nevertheless, even if advanced CPOEs-CDS will be available in near future, to improve prescribing in older patients, clinical judgment and expertise of medical professionals skilled in geriatric care and pharmacotherapy is irreplaceable. When prescribing for older patients, one should take into account life expectancy and quality of life, select essential medications, and avoid drugs with a poorer benefit-to-risk ratio [62]. Given the limited data on treatment effects in older patients, selecting appropriate medications in this patient population is even more challenging. Furthermore, multimorbidity, polypharmacy, and reduced physical and cognitive functions are other factors that need to be taken into account [11]. Therefore, because of this complexity of older patients’ cases, a multidisciplinary approach to care for older hospitalized patients, such as participation of a clinical pharmacist in medical/geriatric teams [63]–[66], seems appropriate.
However, at the time of this study, routine on-ward participation of clinical pharmacists was not offered by the hospital pharmacy departments of the participating hospitals. The clinical services were mainly reactive and included: on-call availability of hospital pharmacists for pharmacotherapy or toxicology consultations, review of overridden CPOEs-CDS alerts and consultations if needed, and therapeutic drug monitoring and advices. Clinical consultations by geriatricians were also available on request. This kind of organization of pharmacy and geriatric services is common in Dutch hospitals, as well as in hospitals in other countries [67], [68]. The fact that, in line with previous findings [1], [16]–[19], [47]–[51], the ADEs identified in the present study affected almost every organ system and involved various medication classes, emphasizes the need to rethink the organization of (hospital) care for older patients [69].
Furthermore, in the participating hospitals, the daily care of patients was provided by junior medical residents who were supervised by attending senior physicians. Gaps in geriatric knowledge and skills across all care settings and levels of medical experience are of major concern [68]. Recently, criteria to detect undertreatment and inappropriate medication use (e.g. contra-indicated medications) have shown promising results [55]. Because pADEs caused by these two types of medication errors were frequent in the present study, the use of these criteria by physicians while prescribing could be of additional value.
Evaluation of the Reliability
The intra-rater agreement on all aspects of ADE assessment was substantial to almost perfect. Structuring the chart review and ADE assessment processes, providing a CRF, a manual, and trainings, may all have contributed to this result. However, despite these efforts, the inter-rater agreements for the occurrence of ADEs and ADE preventability remained fair (κ = 0.24 and κ = 0.38, respectively) and were only slight for the causality (κ = 0.19). Because both teams included senior pharmacists and senior internal medicine physicians, these findings indicate a high degree of subjectivity in the review process, rather than disparities in the experience level of the reviewers. The subjectivity might have been reduced by a discussion between all four reviewers and by sharing knowledge during expert panel meetings. However, such an approach has been shown to improve agreement within a pair of reviewers but not between pairs of reviewers [70]. This seems to be the case in the present study. In contrast, the inter-rater agreement for the severity was almost perfect (κ = 0.85). To assess severity, the CTCEA classification for adverse events were used [43]. The explicit character of this tool, which provides five categories of severity for various types of events, may have decreased the subjectivity of expert judgments, leading to more reliable and consistent results between the two expert teams involved. The use of explicit measures has been advocated in patient and medication safety studies to improve the understanding of the results and to facilitate the comparison between different settings and/or patient populations [20]. However, in the case of complex patients, a patient-specific approach requiring implicit judgments on the causality and preventability of ADEs is unavoidable.
The ADEs related to triggers, which were included in the intra-rater agreement measurement, were all identified during repeated review by our expert team (LA and CS), and nine out of 11 ADEs related to triggers, previously detected by our expert team, were also identified by the second team of experts (HA and PK). However, because only 36% of ADEs indentified in this study were related to triggers included in the modified IHI ADE trigger-tool, the advantages of this explicit approach on the inter-rater agreement remains limited.
Limitations
This study has several limitations. Clearly, the ADE identification strategy would be too resource- and time-intensive to undertake routinely in the inpatient setting. However, as shown by the results of the present study such investment was worthwhile and should be considered whenever an initial and a detailed insight into ADE occurrence is required. Such insight is necessary to be able to develop tailored interventions for a specific setting. A tailored approach has shown to be successful [71].
Prospective data collection has been stated as preferred study type, especially for the identification of preventable events [54]. Mainly because medical teams working on the wards can be involved in the identification of events and provide additional information whenever necessary. When data are collected retrospectively, which was the case in the present study, documentation bias can occur because reviewers must relay on information recorded in patient charts to identify and assess ADEs [72]. On the other hand, collecting data prospectively can introduce bias due to higher awareness of physicians and nurses working on the wards [20]. Therefore, if the goal of an ADE measurement is to investigate an effect of interventions (purpose of the WINGS study [35]), retrospective data collection seems more appropriate [54]. Furthermore, the comprehensiveness of patient data available in the present study may have reduced the difficulty in assessing occurrence and preventability of ADEs by our experts.
Given the fair inter-rater reliability of the ADE identification strategy, generalization of present study results to other settings or patient populations should be exercised with caution. Furthermore, in the present study, the modified IHI ADE trigger-tool was not used independently of the PCR. Therefore, it is not possible to assess the individual contributions of this tool to the total ADE yield. Combining methods that have consistently identified ADEs, such as PCR and (computerized) trigger-tools is, however, advocated [58].
Conclusions
By using structured and comprehensive PCR combined with a modified IHI ADE trigger tool as an aid, we were able to gain detailed insight into the types of ADEs occurring in older patients. This insight can be used to develop tailored interventions to effectively reduce pADEs. Several aspects of the ADE identification strategy may have contributed to the high ADE yield gained. Others can use this knowledge to make well-founded choices regarding methodology whenever ADEs need to be measured. The improvement of the inter-rater reliability of a PCR conducted by experts remains challenging. To be able to use the ADE identification strategy for evaluation of future interventions, the same team of experts must be assigned for ADE identification and assessment. Intensified training of the assessors within an expert team should be aimed for.
Supporting Information
Table S1.
The Institute for Healthcare Improvement (IHI) Adverse Drug Event Trigger Tool as modified to the Dutch hospital setting. *Antiemetics: alizapride, domperidone, droperidol, metoclopramide, prochlorperazine, ondansetron, granisetron, palonosetron, tropisetron, aprepitant, fosaprepitant **Anti-diarrheal drugs: loperamide, activated charcoal (mostly used in drug intoxication).
https://doi.org/10.1371/journal.pone.0071045.s001
(DOCX)
Acknowledgments
We thank all staff of Internal Medicine and Hospital Pharmacy departments of the participating hospitals for their support, and in particular Wendy van den Berg, José Popma-de Koning, Anouk Verburg-Eisma, Carla Kamp, Vera Ruijter, and Jolande van der Wildt for their assistance in data collection, and Miranda Roskam for building the database for the study. We would also like to acknowledge the help of pharmacy students Kayan Tsoi and Mila Tjoa in the data entry for this study. Members of WINGS (Ward-oriented pharmacy In Newly admitted Geriatric Seniors) Study Group are: Joost L.B. Hoekstra, MD, PhD, Department of Internal Medicine; Minke E.P. Jansen, PharmD, MSc, Department of Hospital Pharmacy, Academic Medical Centre, Amsterdam, The Netherlands; Wim G. Meijer, MD, PhD, Department of Internal Medicine, Westfriesgasthuis Hospital, Hoorn, The Netherlands; Bea M. van der Kleij, PharmD, MSc, Department of Hospital Pharmacy, Westfriesgasthuis Hospital, Hoorn, The Netherlands; Anne M. Lagaay, MD, PhD, Department of Internal Medicine and Geriatrics, Spaarne Hospital, Hoofddorp, The Netherlands; Ruud T.M. van der Hoeven, PharmD, MSc, Director of Pharmacy Foundation of Haarlem Hospitals, Haarlem, The Netherlands.
Author Contributions
Conceived and designed the experiments: JEK PCW MGWD SER LL SMS. Performed the experiments: JEK CCMS LA PFMK HA. Analyzed the data: JEK MGWD SER LL SMS. Wrote the paper: JEK MGWD SER LL SMS PCW. Critically reviewed the manuscript: JEK PCW CCMS LA PFMK HA SER LL SMS.
References
- 1. Thomas EJ, Brennan TA (2000) Incidence and types of preventable adverse events in elderly patients: population based review of medical records. BMJ 320: 741–744.
- 2. De Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA (2008) The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 17: 216–223.
- 3. Zegers M, De Bruijne MC, Wagner C, Hoonhout LHF, Waaijman R, et al. (2009) Adverse events and potentially preventable deaths in Dutch hospitals: results of a retrospective patient record review study. Qual Saf Health Care 18: 297–302.
- 4. Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, et al. (1991) The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 324: 377–384.
- 5. Thürmann PA (2003) Detection of drug-related adverse events in hospitals. Expert Opin Drug Saf 2: 447–449.
- 6. Hoonhout LHF, De Bruijne MC, Wagner C, Asscheman H, Van der Wal G, et al. (2010) Nature, occurrence and consequences of medication-related adverse events during hospitalization: a retrospective chart review in the Netherlands. Drug Saf 33: 853–864.
- 7. Evans RS, Lloyd JF, Stoddard GJ, Nebeker JR, Samore MH (2005) Risk factors for adverse drug events: a 10-year analysis. Ann Pharmacother 39: 1161–1168.
- 8. Lazarou J, Pomeranz BH, Corey PN (1998) Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA 279: 1200–1205.
- 9. McLean AJ, Le Couteur DG (2004) Aging biology and geriatric clinical pharmacology. Pharmacol Rev 56: 163–184.
- 10. Hartholt KA, Van der Velde N, Looman CWN, Panneman MJM, Van Beeck EF, et al. (2010) Adverse drug reactions related hospital admissions in persons aged 60 years and over, The Netherlands, 1981–2007: less rapid increase, different drugs. PloS One 5: e13977.
- 11. Steinman MA, Hanlon JT (2010) Managing Medications in Clinically Complex Elders: “There is Got to Be a Happy Medium”. JAMA 304: 1592–1601.
- 12. Spinewine A (2008) Adverse Drug Reactions in Elderly People: The challenge of safer prescribing. BMJ 336: 956–957.
- 13.
Expert Group On Safe Medication Practices (2006). Creation of a better medication safety culture in Europe: Building up safe medication practices. Strasbourg, France. 38 p.
- 14. Kanjanarat P, Winterstein AG, Johns TE, Hatton RC, Gonzalez-Rothi R, et al. (2003) Nature of preventable adverse drug events in hospitals: a literature review. Am J Health Syst Pharm 60: 1750–1759.
- 15. Bates DW, Leape LL, Petrycki S (1993) Incidence and preventability of adverse drug events in hospitalized adults. J Gen Intern Med 8: 289–294.
- 16. Egger T, Dormann H, Ahne G, Runge U, Neubert A, et al. (2003) Identification of adverse drug reactions in geriatric inpatients using a computerised drug database. Drugs Aging 20: 769–776.
- 17. Gray SL, Sager M, Lestico MR, Jalaluddin M (1998) Adverse drug events in hospitalized elderly. J Gerontol A Biol Med Sci 53: M59–M63.
- 18. Trivalle C, Cartier T, Verny C, Mathieu A-M, Davrinche P, et al. (2010) Identifying and preventing adverse drug events in elderly hospitalised patients: a randomised trial of a program to reduce adverse drug effects. J Nutr Health Aging 14: 57–61.
- 19. Tangiisuran B, Davies JG, Wright JE, Rajkumar C (2012) Adverse drug reactions in a population of hospitalized very elderly patients. Drugs & aging 29: 669–679.
- 20. Brown C, Hofer T, Johal A, Thomson R, Nicholl J, et al. (2008) An epistemology of patient safety research: a framework for study design and interpretation. Part 3. End points and measurement. Qual Saf Health Care 17: 170–177.
- 21. Kaushal R (2002) Using chart review to screen for medication errors and adverse drug events. Am J Health Syst Pharm 59: 2323–2325.
- 22. Lilford RJ, Mohammed MA, Braunholtz D, Hofer TP (2003) The measurement of active errors: methodological issues. Qual Saf Health Care 12 Suppl 2ii8–12.
- 23. Morimoto T, Gandhi TK, Seger AC, Hsieh TC, Bates DW (2004) Adverse drug events and medication errors: detection and classification methods. Qual Saf Health Care 13: 306–314.
- 24. Nebeker JR (2002) Developing a Taxonomy for Research in Adverse Drug Events: Potholes and Signposts. J Am Med Inform Assoc 9 (Suppl 2)80–85.
- 25. Macedo AF, Marques FB, Ribeiro CF (2006) Can decisional algorithms replace global introspection in the individual causality assessment of spontaneously reported ADRs? Drug Saf 29: 697–702.
- 26. Murff HJ, Patel VL, Hripcsak G, Bates DW (2003) Detecting adverse events for patient safety research: a review of current methodologies. J Biomed Inform 36: 131–143.
- 27. Egbring M, Far E, Knuth A, Roos M, Kirch W, et al. (2011) Performance of different data sources in identifying adverse drug events in hospitalized patients. Eur J Clin Pharmacol 67: 909–918.
- 28. Agbabiaka TB, Savović J, Ernst E (2008) Methods for causality assessment of adverse drug reactions: a systematic review. Drug Saf 31: 21–37.
- 29. Arimone Y, Miremont-Salamé G, Haramburu F, Molimard M, Moore N, et al. (2007) Inter-expert agreement of seven criteria in causality assessment of adverse drug reactions. Br J Clin Pharmacol 64: 482–488.
- 30. Rozich JD, Haraden CR, Resar RK (2003) Adverse drug event trigger tool: a practical methodology for measuring medication related harm. Qual Saf Health Care 12: 194–200.
- 31. Neale G, Woloshynowych M (2003) Retrospective case record review: a blunt instrument that needs sharpening. Qual Saf Health Care 12: 2–3.
- 32. Resar RK, Rozich JD, Classen D (2003) Methodology and rationale for the measurement of harm with trigger tools. Qual Saf Health Care 12 Suppl 339–45.
- 33.
Institute for Helathcare Improvement (2004) Trigger Tool for Measuring Adverse Drug Events. Available: http://www.ihi.org/knowledge/Pages/Tools/TriggerToolforMeasuringAdverseDrugEvents.aspx. Accessed 2013 Apr 16.
- 34. Franklin BD, Birch S, Schachter M, Barber N (2010) Testing a trigger tool as a method of detecting harm from medication errors in a UK hospital: a pilot study. Int J Pharm Pract 18: 305–311.
- 35. Klopotowska JE, Wierenga PC, De Rooij SE, Stuijt CC, Arisz L, et al. (2011) The effect of an active on-ward participation of hospital pharmacists in Internal Medicine teams on preventable Adverse Drug Events in elderly inpatients: protocol of the WINGS study (Ward-oriented pharmacy In Newly admitted Geriatric Seniors). BMC Health Serv Res 11: 124.
- 36. Zegers M, De Bruijne MC, Wagner C, Groenewegen PP, Waaijman R, et al. (2007) Design of a retrospective patient record study on the occurrence of adverse events among patients in Dutch hospitals. BMC Health Serv Res 7: 27.
- 37. Phansalkar S, Hoffman JM, Nebeker JR, Hurdle JF (2007) Pharmacists versus nonpharmacists in adverse drug event detection: a meta-analysis and systematic review. Am J Health Syst Pharm 64: 842–849.
- 38. Van Doormaal JE, Mol PGM, Van Den Bemt PMLA, Zaal RJ, Egberts ACG, et al. (2008) Reliability of the assessment of preventable adverse drug events in daily clinical practice. Drug Saf 17: 645–654.
- 39.
The G-Standaard: the medicines standard in healthcare. Available: http://www.knmp.nl/downloads/producten-diensten/g-standaard-1/algemeen/GStandaarddecisionsupportversionoct2011.pdf. Accessed 2013 Jun 18.
- 40. Van der Sijs H, Mulder A, Van Gelder T, Aarts J, Berg M, et al. (2009) Drug safety alert generation and overriding in a large Dutch university medical centre. Pharmacoepidemiol Drug Saf 18: 941–947.
- 41.
Committee of Experts on Management of Safety and Quality in Health Care (SP-SQS), Expert Group on Safe Medication Practices (2005) Glossary of terms related to patient and medication safety. Strasbourg, France. p 16.
- 42.
The Uppsala Monitoring Centre. The use of the WHO-UMC system for standardised case causality assessment. Available: http://who-umc.org/Graphics/24734.pdf. Accessed 2013 Apr 16.
- 43.
Cancer Therapy Evaluation Program. Common Terminology Criteria for Adverse Events v3.0. 0. Available: http://ctep.cancer.gov/protocoldevelopment/electronic_applications/docs/ctcaev3.pdf. Accessed 2013 Apr 16.
- 44.
Dutch Hospital Pharmacists Association. Centrale Medicatiefouten Registratie (CMR). The Hague, The Netherlands.
- 45.
Dutch Drug Information. Available: http://kennisbank.knmp.nl. Accessed 2013 Apr 16.
- 46. Kunac DL, Reith DM, Kennedy J, Austin NC, Williams SM (2006) Inter- and intra-rater reliability for classification of medication related events in paediatric inpatients. Qual Saf Health Care 15: 196–201.
- 47. Page RL, Ruscin JM (2006) The risk of adverse drug events and hospital-related morbidity and mortality among older adults with potentially inappropriate medication use. Am J Geriatr Pharmacother 4: 297–305.
- 48.
Azad N, Tierney M, Victor G, Kumar P (2002) Adverse drug events in the elderly population admitted to a tertiary care hospital. J Healthc Manag 47: 295–305; discussion 305–6.
- 49. Bowman L, Carlstedt BC, Hancock EF, Black CD (1996) Adverse drug reaction (ADR) occurrence and evaluation in elderly inpatients. Pharmacoepidemiol Drug Saf 5: 9–18.
- 50. Harugeri A, Joseph J, Parthasarathi G, Ramesh M, Guido S (2010) Potentially inappropriate medication use in elderly patients: a study of prevalence and predictors in two teaching hospitals. J Postgrad Med 56: 186–191.
- 51. Passarelli MCG, Jacob-Filho W, Figueras A (2005) Adverse drug reactions in an elderly hospitalised population: inappropriate prescription is a leading cause. Drugs Aging 22: 767–777.
- 52. Bates DW (2002) Using information technology to screen for adverse drug events. Am J Health Syst Pharm 59: 2317–2319.
- 53. Kilbridge PM, Campbell UC, Cozart HB, Mojarrad MG (2006) Automated surveillance for adverse drug events at a community hospital and an academic medical center. J Am Med Inform Assoc 13: 372–377.
- 54. Michel P, Quenon JL, De Sarasqueta AM, Scemama O (2004) Comparison of three methods for estimating rates of adverse events and rates of preventable adverse events in acute care hospitals. BMJ 328: 199.
- 55. Gallagher PF, O’Connor MN, O’Mahony D (2011) Prevention of Potentially Inappropriate Prescribing for Elderly Patients: A Randomized Controlled Trial Using STOPP/START Criteria. Clin Pharmacol Ther 89: 845–854.
- 56. Thomsen LA, Winterstein AG, Søndergaard B, Haugbølle LS, Melander A (2007) Systematic review of the incidence and characteristics of preventable adverse drug events in ambulatory care. Ann Pharmacother 41: 1411–1426.
- 57. Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD, et al. (2001) Medication errors and adverse drug events in pediatric inpatients. JAMA 285: 2114–2120.
- 58.
Manias E (2012) Detection of medication-related problems in hospital practice: a review. Br J Clin Pharmacol Nov 30.
- 59. Wolfstadt JI, Gurwitz JH, Field TS, Lee M, Kalkar S, et al. (2007) The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 23: 451–458.
- 60. Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, et al. (2007) Medication-related clinical decision support in computerized provider order entry systems: a review. J Am Med Inform Assoc 14: 29–40.
- 61. Van der Sijs H, Aarts J, Vulto A, Berg M (n.d.) Overriding of drug safety alerts in computerized physician order entry. J Am Med Inform Assoc 13: 138–147.
- 62. Page RL, Linnebur SA, Bryant LL, Ruscin JM (2010) Inappropriate prescribing in the hospitalized elderly patient: defining the problem, evaluation tools, and possible solutions. Clin Interv Aging 5: 75–87.
- 63. Bakker FC, Robben SHM, Olde Rikkert MGM (2011) Effects of hospital-wide interventions to improve care for frail older inpatients: a systematic review. BMJ Qual Saf 20: 680–691.
- 64. Kucukarslan SN, Peters M, Mlynarek M, Nafziger DA (2003) Pharmacists on rounding teams reduce preventable adverse drug events in hospital general medicine units. Arch Intern Med 163: 2014–2018.
- 65. Gillespie U, Alassaad A, Henrohn D, Garmo H, Hammarlund-Udenaes M, et al. (2009) A comprehensive pharmacist intervention to reduce morbidity in patients 80 years or older: a randomized controlled trial. Arch Intern Med 169: 894–900.
- 66. Spinewine A, Swine C, Dhillon S, Lambert P, Nachega JB, et al. (2007) Effect of a collaborative approach on the quality of prescribing for geriatric inpatients: a randomized, controlled trial. J Am Geriatr Soc 55: 658–665.
- 67. Doloresco F, Vermeulen LC (2009) Global survey of hospital pharmacy practice. Am J Health Syst Pharm 66: S13–9.
- 68. Besdine R, Boult C, Brangman S, Coleman EA, Fried LP, et al. (2005) Caring for older Americans: the future of geriatric medicine. J Am Geriatr Soc 53: S245–56.
- 69. Tinetti ME, Fried TR, Boyd CM (2012) Designing health care for the most common chronic condition–multimorbidity. JAMA 307: 2493–2494.
- 70. Hofer TP, Bernstein SJ, DeMonner S, Hayward RA (2000) Discussion between reviewers does not improve reliability of peer review of hospital quality. Medical Care 38: 152–161.
- 71.
Baker R, Camosso-Stefinovic J, Gillies C, Shaw EJ, Cheater F, et al.. (2010) Tailored interventions to overcome identified barriers to change: effects on professional practice and health care outcomes. Cochrane Database Syst Rev: CD005470.
- 72. Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P (2000) How well does chart abstraction measure quality? A prospective comparison of standardized patients with the medical record. Am J Med 108: 642–649.
- 73.
Anatomical Therapeutic Chemical (ATC) classification by World Health Organization Collaborating Centre for Drug Statistics Methodology. Available: http://www.whocc.no/atc_ddd_index/. Accessed 2013 Jun 18.