Repeat adverse drug events associated with outpatient medications: a descriptive analysis of 3 observational studies in British Columbia, Canada
================================================================================================================================================

* Corinne M. Hohl
* Stephanie A. Woo
* Amber Cragg
* Maeve E. Wickham
* Christine Ackerley
* Frank Scheuermeyer
* Diane Villanyi

## Abstract

**Background:** Adverse drug events are an important cause of preventable emergency department visits and hospital admissions. We examined repeat adverse drug events associated with outpatient medications resulting in acute care utilization.

**Methods:** This descriptive analysis combined data from 3 prospective multicentre observational studies, in which clinical pharmacists and physicians independently evaluated patients who visited the emergency department for adverse drug events in 3 hospitals in British Columbia. During these studies, an independent committee adjudicated all discordant and uncertain cases using a standardized algorithm. For the current study, we retrospectively reviewed the medical and research records of all patients 19 years of age and older who had been diagnosed with an adverse drug event during the primary studies to determine the proportion of repeat events. We used multivariable logistic regression to identify factors associated with repeat events; we adjusted for clustering at the hospital level for patient-level analyses and at the patient level for event-level analyses.

**Results:** Among 12 977 patients, 1178 were diagnosed with 1296 adverse drug events at the point of care. Of these events, 32.5% (421 of 1296; 95% confidence interval [CI] 29.8%–35.1%) were repeat events, of which 75.3% (317 of 421; 95% CI 71.1%–79.5%) were deemed probably or definitely preventable as re-exposure to the culprit medication or repeat withdrawal of an indicated medication was inconsistent with best medical practice. Patients presenting with repeat events were more likely to have renal failure (odds ratio [OR] 2.01; 95% CI 1.32%–3.07%) or a mental health diagnosis (OR 1.39; 95% CI 1.02%–1.88%).

**Interpretation:** A high proportion of adverse drug events were repeat events, most of which were deemed preventable. Interventions to ensure that care providers are aware of previously diagnosed adverse drug events when prescribing or dispensing need to be developed and evaluated and may reduce unintentional re-exposures to previously harmful medications.

Adverse drug events are a leading cause of emergency department visits and unplanned hospital admissions in Canada1–3 and a key focus of patient safety initiatives. 4 In 2017, the World Health Organization called for a commitment to reduce severe, avoidable medication-related harms by 50% over the next 5 years.5 In the United States, the National Action Plan for Adverse Drug Event Prevention called for coordinated efforts in surveillance, oversight and research to develop effective evidence-based strategies to reduce adverse drug events.6

Identifying system-level weaknesses that contribute to adverse drug events may allow us to target a broader range of events, develop and evaluate innovative system-level interventions, understand why previous prevention efforts have had limited success, and develop new metrics for evaluation.5 Preliminary studies suggest that unintentional re-exposures to culprit or high-risk medications represent a safety risk.7–9 However, this evidence is limited, and repeat adverse drug events are poorly understood.

If repeat adverse drug events are common, they are likely to warrant new systems-level approaches for prevention. To date, health information technologies have focused on improving clinician adherence to treatment and monitoring guidelines, ensuring medications are prescribed and administered within specified dosing ranges and as intended, and avoidance of drug interactions.10–12 However, few have emphasized the interoperability of health information systems between health sectors or their integration with drug information systems to enable timely communication of adverse drug event information across health care silos and between provider groups.13 This has the potential to create informational discontinuity, which may place patients at risk of unintentionally being re-exposed to drugs, or drug classes, that were previously identified as contraindicated. Our main objective was to determine the proportion of patients presenting to hospital with repeat adverse drug events associated with outpatient medications.

## Methods

### Study design

This was a descriptive analysis of patients seen in the emergency department who had been diagnosed with adverse drug events in 1 of 3 primary prospective studies (Supplementary Table S1, Appendix 1, available at [www.cmajopen.ca/content/7/3/E446/suppl/DC1](http://www.cmajopen.ca/content/7/3/E446/suppl/DC1)).14–17 The first 2 studies were prospective, observational, multicentre cohort studies, in which we derived (*n* = 1591)14 and subsequently validated clinical decision rules (*n* = 1529) to identify patients at high risk of presenting to the emergency department with an adverse drug event.15 The third study was a prospective, multicentre controlled clinical trial in which we evaluated the impact of pharmacist-led medication reviews in high-risk emergency department patients (*n* = 10 807).16 We combined data from the primary studies and retrospectively reviewed the medical records to determine the proportion of repeat adverse drug events, as these studies used the same patient selection algorithms and clinical assessments, which enabled us to compile a case series of adverse drug events diagnosed prospectively.

### Study patients

We reviewed the medical and research records of all patients diagnosed with 1 or more medication-related problems or adverse drug events in the 3 previously completed prospective studies (Supplementary Table S1, Appendix 1).14–17 In the prospective studies, patients aged 19 years of age and older had been enrolled at Vancouver General Hospital and St. Paul’s Hospital, which are tertiary care centres in Vancouver, British Columbia, and at Lions Gate Hospital, an urban community hospital in North Vancouver. These sites were chosen because of their ability to participate in the primary studies and to make charts available for review. In each prospective study, research assistants enrolled patients seen in the emergency department using a standardized algorithm to generate a representative sample of patients (Supplementary Figure S1, Appendix 1).

### Prospective data collection

In the primary studies, clinical pharmacists completed medication reviews for all enrolled patients in the emergency department by completing a best-possible medication history and documenting medication-related problems in the patients’ medical records. The pharmacists assessed causality of all suspected adverse drug events using the modified Naranjo algorithm, a validated tool that scores the likelihood that an event is drug related.18 Clinical pharmacists discussed their medication review findings with treating physicians in the emergency department and followed patients through their hospital course, and after discharge if required. After discharge, an independent committee consisting of a different pharmacist and physician adjudicated all cases in which the treating pharmacists’ and physicians’ diagnoses had been discordant or uncertain using a standardized algorithm (Supplementary Figure S2, Appendix 1).

### Inclusion and exclusion criteria

For the present study, we reviewed the records of all patients who had been diagnosed with a medication-related problem or adverse drug event in 1 of the 3 prospective studies who presented to Vancouver General Hospital, Lions Gate Hospital and St. Paul’s Hospital.14–16 We excluded all patients in whom an alternative diagnosis was identified and who did not meet our case definition of an adverse drug event. We excluded patients with illegible records.

### Outcome measure

Our primary outcome was a repeat adverse drug event, defined as an event that had been diagnosed and recorded in the patient’s medical record during an episode of care that preceded their enrolment in the prospective study, at the same hospital in which the index event was diagnosed. Repeat events had to be classified as the same type of adverse drug event, present with the same or similar symptoms, and be the result of a re-exposure to, or repeat withdrawal from, the same or same-class medication as a previously documented event.

To identify repeat adverse drug events, the pharmacist (S.W.) manually reviewed the entire available hospital record for any patient diagnosed with an adverse drug event on the index visit (recorded in the prospective studies’ research records). If the pharmacist identified a preceding adverse drug event that met our study definition, a physician involved in the study (C.H., F.S. or D.V.) reviewed the chart to confirm the diagnosis.

We did not rely on any diagnostic codes to identify adverse drug events (index or repeat events), as we have previously shown that diagnostic codes have poor sensitivity for adverse drug events.19,20

### Definitions

We defined adverse drug events as unintended events arising from the appropriate or inappropriate use of a drug, consistent with its clinical practice definition.21 Adverse drug events included adverse drug reactions, undesirable effects that occurred within the therapeutic dosing range,22,23 drug interactions, supra- and sub-therapeutic doses, events due to patient-related nonadherence, inappropriate drug withdrawal, and cases in which the patient was on an ineffective drug or on no drug despite previous documentation of an indication for and absence of a contraindication to a drug (e.g., a patient presenting with an ischemic stroke with a previously documented history of atrial fibrillation and a transient ischemic attack who was not on anticoagulation).21 For events with abnormal vital signs, we defined cut-offs a priori (Supplementary Table S2, Appendix 1). For events involving laboratory abnormalities, we used the hospitals’ reference values. Pharmacists used a modified causality algorithm to determine the causality between a patient’s presentation and the drug.3,18

We categorized severity as mild when the event required no change in medical management, moderate when it required a change in medical management and severe when it was the primary reason for hospital admission, caused permanent disability or was life threatening.3,24,25

We categorized events as preventable when they resulted from medical care that could have been mitigated by heightened monitoring or was inconsistent with best practice,3,6,25,26 on the basis of current treatment and monitoring guidelines for given medical conditions, or the experience of the clinical pharmacist (S.W.) and physician review team (C.H., F.S., D.V.) for cases where no explicit guidelines were available.25 This rating was important, as re-exposures to medications after an adverse drug event may have been consistent with best medical practice (e.g., re-exposure to warfarin in a patient with atrial fibrillation and a high CHADS score [a scoring mechanism for atrial fibrillation encompassing history of congestive heart failure, hypertension, age > 75 yr, diabetes mellitus and previous stroke or transient ischemic attack symptoms], and a recent gastrointestinal bleed) and therefore would have been deemed nonpreventable.

### Data collection

Patient demographics, number and types of medications used, comorbid conditions, Canadian Triage Acuity Score, ambulance arrival, disposition and medication review details were derived from the databases of the 3 prospective studies.14–16 All other reported data were derived from the medical records. During chart review, a pharmacist (S.W.) and a physician (F.S., D.V. or C.H.) independently reviewed all paper-based and electronic medical and research records of patients diagnosed with an adverse drug event, from the hospital where the index event had been diagnosed. We rated the index event as a repeat event if either of the reviewers identified a prior event meeting our case definition and the second reviewer verified it. Reviewers independently rated the preventability of each adverse drug event.25 The initial reviewers discussed any disagreements until consensus was reached. In cases in which uncertainty remained, a third reviewer adjudicated the case.

### Statistical analysis

We produced descriptive analyses for demographic data (A.C., M.W.). The proportion of repeat events was the number of adverse drug events documented in a prior episode of care, over all events identified. The proportion of preventable repeat events was the number of repeat adverse drug events that were rated as probably or definitely preventable, over all repeat events. We assessed interrater agreement between pharmacist and physician preventability ratings using Cohen’s κ with 95% confidence intervals [CIs] for definitely and probably preventable events versus nonpreventable events for the initial reviewer ratings. We used multivariable logistic regression to identify factors associated with repeat events. As our sample was clustered, our effect estimates were adjusted for clustering at the hospital level for patient-level analyses and at the patient level for event-level analyses. We used purposeful selection modelling, an iterative approach to developing the best multivariate model for analysis.27 We tested the univariate associations between each variable and the outcome and included variables significant at *p* < 0.25 as candidates in the final model. Variables included in the multivariable model were removed one at a time, and if the coefficient of any independent variable changed by 10% or more upon its removal, the variable was included as a confounder. Exposures of interest and patient-level variables deemed important to be controlled for (age, sex, comorbidities) were included regardless of statistical significance.

### Ethics approval

The institutional review boards of the University of British Columbia and of all participating hospitals approved the study protocol.

## Results

Among 12 977 patients in the primary studies, 1178 were prospectively diagnosed with 1296 adverse drug events (Figure 1). The patients’ ages ranged from 20 to 99 years at the index visit. Their mean age was 65.4 ± 20.5 years, and 56.2% were women; their mean number of prescribed medications was 9.1 ± 5.7 (Table 1). The most common comorbidities were hypertension (45.2%), diabetes (21.0%) and atrial fibrillation (20.6%). Over one-third of patients presented with at least 1 repeat adverse event: 32.2% (95% CI 24.0%–40.4%) presented with 1 repeat event, and 1.8% (95% CI 0.0%–4.3%) presented with 2. Among patients with repeat events, 38.4% (95% CI 20.3%–56.5%) were admitted to hospital.

![Figure 1:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/7/3/E446/F1.medium.gif)

[Figure 1:](http://www.cmajopen.ca/content/7/3/E446/F1)

Figure 1: 
Flow diagram of patients through the study. Note: ADE = adverse drug event, ED = emergency department.

View this table:
[Table 1:](http://www.cmajopen.ca/content/7/3/E446/T1)

Table 1: 
Characteristics of 1178 patients with repeat and first-occurrence adverse drug events

Of the 1296 events identified, 32.5% (95% CI 29.8%–35.1%) were repeat events (Table 2), of which most (75.3%; 95% CI 71.1%–79.5%) were deemed preventable. The interrater agreement for the preventability of repeat events was 0.53 (95% CI 0.48–0.59). The most common repeat events were adverse drug reactions (31.4%; Table 2). Most repeat adverse drug events were moderate in severity (66.5%) and resulted in temporary harm (81.5%). Among repeat events, 64.6% were due to re-exposures to previously harmful medications, while the remainder were attributed to other causes, including repeat medication withdrawals or dosing problems (e.g., repeat nonadherence with antiepileptics causing repeat seizures). Most repeat events were attributable to a single drug (75.5%; 95% CI 71.4%–79.7%; Supplementary Table S3, Supplementary Table S4, Appendix 1), with coumarin derivatives (12.4%), opiates (12.1%) and insulins (8.1%) most commonly implicated.

View this table:
[Table 2:](http://www.cmajopen.ca/content/7/3/E446/T2)

Table 2: 
Characteristics of repeat and first-occurrence adverse drug events (*n* = 1296)

Patients presenting with repeat events were more likely to have renal failure (odds ratio [OR] 2.01; 95% CI 1.32–3.07) or a mental health diagnosis (OR 1.39; 95% CI 1.02–1.88) compared with patients experiencing adverse drug events for the first time (Table 3).

View this table:
[Table 3:](http://www.cmajopen.ca/content/7/3/E446/T3)

Table 3: 
Multivariable associations between patient-level factors for patients with 1 or more repeat adverse drug events compared with patients without any repeat events (*n* = 1178)

## Interpretation

We examined patients presenting to 3 emergency departments with adverse drug events associated with outpatient medications and found that almost one-third of the events were repeat events. The majority of the repeat events were due to preventable re-exposures that reviewers deemed inconsistent with best medical practice. All of the repeat adverse drug events identified led to at least an emergency department visit, indicating that they incurred health services utilization. The majority were rated as moderate or severe, indicating that they required a change in medical management or admission to hospital.

Our findings suggest that investigating causes for, and developing interventions targeting preventable re-exposures to medications that previously caused harm may reduce adverse drug events overall. This is particularly relevant for patient populations at higher risk of repeat adverse drug events, such as those with renal failure and mental illness. While medications will always carry an inherent degree of baseline risk when prescribed, patients who have a documented adverse event associated with a medication should probably only be re-exposed to a culprit medication if there is a clear indication for re-exposure and a strong contraindication is absent.

Although there is inherent subjectivity in assessing preventability, we rooted these assessments in best medical practice, as determined by 2 independent reviewers.26 Most re-exposures were deemed inconsistent with best medical practice and were therefore categorized as preventable. Many of these were to drugs that were not medically necessary or were even inappropriate (e.g., 2 benzodiazepines in an elderly patient), and others involved re-exposures to drugs that could have been replaced by an alternative drug with a lower risk of causing the same effect (e.g., glyburide in an elderly patient, despite hypoglycemia).

Adverse drug events resulting from errors in drug administration and dispensing have been the focus of costly medication safety interventions, including computerized provider order entry systems and medication reconciliation interventions. 10,28–31 In our study, repeat adverse drug events associated with outpatient medications were 100 times more common than events resulting from medication transcribing, dispensing and administration errors and 7 times more common than drug interactions, which are the focus of interaction-checking software. This indicates an urgent need to rethink current efforts to enhance safe medication use to address more common causes of preventable events.

Prior evidence on repeat events is scant. In a small single-centre Dutch study of elderly patients admitted to hospital for adverse drug reactions, 27% were represcribed the culprit drug that had been withdrawn in hospital within 6 months of discharge, irrespective of the severity of the reaction.8 This finding was mediated by poor communication between care providers and across health settings. A large administrative database study evaluated elderly Ontarians who were admitted to hospital with hypoglycemia while on glyburide or with a fall while on neuroleptics or benzodiazepines; these medications are considered inappropriate and high risk in this age group and were redispensed to 54.7% of patients within 6 months.7

We hypothesize that the high rate of repeat events in our study is due to a lack of standardized documentation of adverse drug events in medical records and to suboptimal communication between care providers who diagnose and treat serious adverse drug events to outpatient medications (typically hospital-based providers) and physicians who prescribe outpatient medications for chronic disease management (typically community-based providers).8,9,13,31 In 2014, the federal government amended the Food and Drugs Act to mandate the reporting of serious adverse drug reactions by health care institutions. However, no clinically useful reporting platform exists at present to achieve this aim, and the platforms that exist are not used to communicate clinically meaningful information between care providers.32 Previous medication safety and health information technology implementation evaluations have focused on single sectors of health (e.g., hospitals) and therefore have had limited ability to measure inappropriate re-exposures to harmful medications across health sectors.10,13,28–31 To date, to our knowledge there has been only 1 randomized trial evaluating the impact of an electronic decision support system on the represcription of contraindicated drugs.33 However, this was designed to prevent re-exposures in primary care and not to communicate information across heath sectors, where discontinuity in information about adverse drug events diagnosed by others is likely to be the greatest.

Our results highlight an urgent need to develop and evaluate health information technologies that can be used to communicate adverse drug event information between care providers and across health sectors. This could ensure that health care providers know about adverse drug events and medication contraindications diagnosed in other health settings before they prescribe and dispense medications, to ensure that patients are not unintentionally re-exposed to previously harmful medications without carefully considering associated risks and benefits, and whether safer alternatives exist.

### Limitations

Because we could only access the hospital records where the index event was diagnosed, patients re-presenting to other sites with adverse drug events were missed, rendering our estimates conservative. The results may also be subject to ascertainment bias in which adverse events may not have been properly documented; this too would have led to an underestimation of the proportion of repeat events. We did not aim to study the prevalence of primary adverse drug events or to look at their risk factors as this would have required a different study design. We had to exclude patients who had medical records that were illegible or inaccessible, and whose charts could not be accessed, limiting the generalizability of our findings.

Our interrater reliability for the rating of preventability of repeat events was moderate (κ = 0.53). Therefore, it is possible that the results misrepresent the proportion of repeat events deemed preventable in our sample. We recently published a paper that compares the methods of determining preventability and that details the challenges associated with obtaining good interrater reliability for preventability ratings.26

We limited our cohort to locations where high-quality prospectively collected data on adverse drug events exist, to ensure robust causality assessments between the drug and patient presentations. Unfortunately, these data were limited to urban acute care hospitals in 1 province, limiting the generalizability of our results. Our estimates may not apply to rural or nontertiary hospitals or to other jurisdictions. However, our findings are consistent with prior preliminary investigations elsewhere that showed that 27%–54% of patients were re-exposed to harmful medications and were at risk of a repeat event.7–9

### Conclusion

Repeat adverse drug events associated with outpatient medications are frequent, cause substantial health services utilization and are commonly preventable. Interventions to reduce repeat adverse drug events, particularly in high-risk patients such as those with renal failure and mental illness, are needed. These may include system-level interventions to ensure patients and care providers are aware of previously identified adverse drug events and medication contraindications when prescribing and dispensing medications to reduce unintentional re-exposures. Repeat adverse drug events should also be considered as an evaluation metric for quality improvement in medication safety.

## Footnotes

*   **Competing interests:** The authors’ research group is developing a software application called ActionADE, which will enable standardized documentation of adverse drug events by front-line clinicians and communication of this information to a central medication dispensing database. No other competing interests were declared.

*   This article has been peer reviewed.

*   **Contributors:** Corinne Hohl conceived the study idea; Corinne Hohl, Amber Cragg and Maeve Wickham contributed to study design and protocol development. Corinne Hohl, Amber Cragg and Maeve Wickham planned the analysis, and Amber Cragg and Maeve Wickham analyzed the data. Corinne Hohl, Amber Cragg, Maeve Wickham and Stephanie Woo planned the data collection, which was then carried out by Stephanie Woo, Christine Ackerley, Frank Scheuermeyer and Diane Villanyi. All authors drafted the manuscript, revised it for important intellectual content and approved the manuscript in its final form. All authors agree to act as guarantors of the work.

*   **Funding:** This study was funded by a peer-reviewed grant through the Drug Safety and Effectiveness Network of the Canadian Institutes of Health Research (CIHR), cofounded by CIHR and Health Canada (grant no. DES 147641).

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/7/3/E446/suppl/DC1](http://www.cmajopen.ca/content/7/3/E446/suppl/DC1).

## References

1.  Budnitz DS, Lovegrove MC, Shehab N, et al. (2011) Emergency hospitalizations for adverse drug events in older Americans. N Engl J Med 365:2002–12.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJMsa1103053&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22111719&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000297282600010&link_type=ISI) 

2.  Hohl CM, Dankoff J, Colacone A, et al. (2001) Polypharmacy, adverse drug-related events, and potential adverse drug interactions in elderly patients presenting to an emergency department. Ann Emerg Med 38:666–71.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1067/mem.2001.119456&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=11719747&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000172507700009&link_type=ISI) 

3.  Zed PJ, Abu-Laban RB, Balen RM, et al. (2008) Incidence, severity and preventability of medication-related visits to the emergency department: a prospective study. CMAJ 178:1563–9.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTc4LzEyLzE1NjMiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzcvMy9FNDQ2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  Research Priority Setting Working Group (2008) Global priorities for research in patient safety (World Health Organization, Geneva), 1st ed.
    
    

5.  (2017) Medication without harm: WHO Global Patient Safety Challenge (World Health Organization, Geneva).
    
    

6.  (2014) National action plan for adverse drug event prevention (U.S. Department of Health and Human Services, Office of Disease Prevention and Health Promotion, Washington (DC)).
    
    

7.  Western ICES (2016) Data Impact Challenge II Answer Submission (Canada Health Infoway, Toronto).
    
    

8.  van der Linden CM, Kerskes MC, Bijl AM, et al. (2006) Represcription after adverse drug reaction in the elderly: a descriptive study. Arch Intern Med 166:1666–7.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/archinte.166.15.1666&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16908802&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000239757700019&link_type=ISI) 

9.  van der Linden CM, Jansen PA, van Marum RJ, et al. (2010) Recurrence of adverse drug reactions following inappropriate re-prescription: better documentation, availability of information and monitoring are needed. Drug Saf 33:535–8.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20553055&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

10. Rinke ML, Bundy DG, Velasquez CA, et al. (2014) Interventions to reduce pediatric medication errors: a systematic review [published erratum in *Pediatrics* 2015;136:583]. Pediatrics 134:338–60.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6OToiMTM0LzIvMzM4IjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby83LzMvRTQ0Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

11. Carling CL, Kirkehei I, Dalsbø TK, et al. (2013) Risks to patient safety associated with implementation of electronic applications for medication management in ambulatory care — a systematic review. BMC Med Inform Decis Mak 13:133.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1472-6947-13-133&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24308799&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

12. Campanella P, Lovato E, Marone C, et al. (2016) The impact of electronic health records on healthcare quality: a systematic review and meta-analysis. Eur J Public Health 26:60–4.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/eurpub/ckv122&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26136462&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

13. van der Linden CM, Jansen PA, Grouls RJ, et al. (2013) Systems that prevent unwanted represcription of drugs withdrawn because of adverse drug events: a systematic review. Ther Adv Drug Saf 4:73–90.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1177/2042098613477125&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25083253&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

14. Hohl CM, Yu E, Hunte GS, et al. (2012) Clinical decision rules to improve the detection of adverse drug events in emergency department patients. Acad Emerg Med 19:640–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1553-2712.2012.01379.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22687179&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

15. Hohl CM, Badke K, Zhao A, et al. (2018) Prospective validation of clinical criteria to identify emergency department patients at high risk for adverse drug events. Acad Emerg Med 25:1015–26.
    
    

16. Hohl CM, Partovi N, Ghement I, et al. (2017) Impact of early in-hospital medication review by clinical pharmacists on health services utilization. PLoS One 12:e0170495.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

17. Hohl CM, McGrail K, Sobolev BG (2015) The effect of pharmacist-led medication review in high-risk emergency department patients: an evaluation protocol. CMAJ Open 3:E103–10.
    
    

18. Naranjo CA, Busto U, Sellers EM, et al. (1981) A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther 30:239–45.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1038/clpt.1981.154&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=7249508&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1981MC89700015&link_type=ISI) 

19. Hohl CM, Kuramoto L, Yu E, et al. (2013) Evaluating adverse drug event reporting in administrative data from emergency departments: a validation study. BMC Health Serv Res 13:473.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1472-6963-13-473&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24219303&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

20. Hohl CM, Karpov A, Reddekopp L, et al. (2014) ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc 21:547–57.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1136/amiajnl-2013-002116&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24222671&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

21. Nebeker JR, Barach P, Samore MH (2004) Clarifying adverse drug events: a clinician’s guide to terminology, documentation, and reporting. Ann Intern Med 140:795–801.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-140-10-200405180-00017&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15148066&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000221520900004&link_type=ISI) 

22. Adverse reaction information (Health Canada, Ottawa) modified 2012 Oct. 31. Available: [www.canada.ca/en/health-canada/services/drugs-health-products/medeffect-canada/adverse-reaction-information.html](http://www.canada.ca/en/health-canada/services/drugs-health-products/medeffect-canada/adverse-reaction-information.html). accessed 2016 July 26.
    
    

23. International drug monitoring (1969) The role of the hospital. World Health Organ Tech Rep Ser 425:5–24.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=4981079&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

24. Zed PJ (2005) Drug-related visits to the emergency department. J Pharm Pract 18:329–35, doi:10.1177/0897190005280049.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1177/0897190005280049&link_type=DOI) 

25. Hallas J, Harvald B, Gram LF, et al. (1990) Drug related hospital admissions: the role of definitions and intensity of data collection, and the possibility of prevention. J Intern Med 228:83–90.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1365-2796.1990.tb00199.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=2394974&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1990DU68600003&link_type=ISI) 

26. Woo SA, Cragg A, Wickham ME, et al. (2018) Methods for evaluating adverse drug event preventability in emergency department patients. BMC Med Res Methodol 18:160.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

27. Hosmer DW, Lemeshow S (2000) Applied logistic regression, second edition (John Wiley & Sons, New York).
    
    

28. Nuckols TK, Smith-Spangler C, Morton SC, et al. (2014) The effectiveness of computerized order entry at reducing preventable adverse drug events and medication errors in hospital settings: a systematic review and meta-analysis. Syst Rev 3:56.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/2046-4053-3-56&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24894078&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

29. Hemens BJ, Holbrook A, Tonkin M, et al., CCDSS Systematic Review Team (2011) Computerized clinical decision support systems for drug prescribing and management: a decision-maker-researcher partnership systematic review. Implement Sci 6:89.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1748-5908-6-89&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21824383&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

30. Carling CL, Kirkehei I, Dalsbø TK, et al. (2013) Risks to patient safety associated with implementation of electronic applications for medication management in ambulatory care — a systematic review. BMC Med Inform Decis Mak 13:133.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1472-6947-13-133&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24308799&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 

31. Chaudhry B, Wang J, Wu S, et al. (2006) Systematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med 144:742–52.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-144-10-200605160-00125&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16702590&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE446.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000237636400005&link_type=ISI) 

32. Hohl C, Lexchin JR, Balka E (2015) Can reporting of adverse drug reactions create safer systems while improving health data? CMAJ 187:789–90.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTg3LzExLzc4OSI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNy8zL0U0NDYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

33. Tamblyn R, Huang A, Perreault R, et al. (2003) The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ 169:549–56.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNjkvNi81NDkiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzcvMy9FNDQ2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

*   Copyright 2019, Joule Inc. or its licensors