Unnecessary antibiotic prescribing in a Canadian primary care setting: a descriptive analysis using routinely collected electronic medical record data
======================================================================================================================================================

* Kevin L. Schwartz
* Bradley J. Langford
* Nick Daneman
* Branson Chen
* Kevin A. Brown
* Warren McIsaac
* Karen Tu
* Elisa Candido
* Jennie Johnstone
* Valerie Leung
* Jeremiah Hwee
* Michael Silverman
* Julie H.C. Wu
* Gary Garber

## Abstract

**Background:** Unnecessary antibiotic use in the community in Canada is not well defined. Our objective was to quantify unnecessary antibiotic prescribing in a Canadian primary care setting.

**Methods:** We performed a descriptive analysis in Ontario from April 2011 to March 2016 using the Electronic Medical Records Primary Care database linked to other health administrative data sets at ICES. We determined antibiotic prescribing rates (per 100 patient–physician encounters) for 23 common conditions and estimated rates of unnecessary prescribing using predefined expected prescribing rates, both stratified by condition and patient age group.

**Results:** The study included 341 physicians, 204 313 patients and 499 570 encounters. The rate of unnecessary antibiotic prescribing for included conditions was 15.4% overall and was 17.6% for those less than 2 years of age, 18.6% for those aged 2–18, 14.5% for those aged 19–64 and 13.0% for those aged 65 or more. The highest unnecessary prescribing rates were observed for acute bronchitis (52.6%), acute sinusitis (48.4%) and acute otitis media (39.3%). The common cold, acute bronchitis, acute sinusitis and miscellaneous nonbacterial infections were responsible for 80% of the unnecessary antibiotic prescriptions. Of all antibiotics prescribed, 12.0% were for conditions for which they are never indicated, and 12.3% for conditions for which they are rarely indicated. In children, 25% of antibiotics were for conditions for which they are never indicated (e.g., common cold).

**Interpretation:** Antibiotics were prescribed unnecessarily for 15.4% of included encounters in a Canadian primary care setting. Almost one-quarter of antibiotics were prescribed for conditions for which they are rarely or never indicated. These findings should guide safe reductions in the use of antibiotics for the common cold, bronchitis and sinusitis.

Rising rates of antimicrobial resistance are an emerging public health crisis.1,2 Antibiotic use is associated with antimicrobial resistance at both the patient and population level.3–5 The largest modifiable driver of resistance is antibiotic use. In addition, antibiotics have important adverse effects, including up to a 30% risk of allergic reactions and gastrointestinal symptoms, including diarrhea associated with *Clostridium difficile*.6–9

Existing evidence suggests there is a considerable amount of inappropriate antibiotic use in ambulatory settings, where 92% of antibiotics are prescribed in Canada.10 In the United States, it is estimated that 30%–50% of antibiotics prescribed outside of hospitals are unnecessary.11,12 In the United Kingdom, overprescribing of antibiotics for respiratory infections is common, particularly for acute bronchitis, sinusitis and acute otitis media in children.13,14 Furthermore, an Ontario study identified that 46% of older adults with a presumed viral respiratory infection filled an antibiotic prescription.15 However, the degree of unnecessary antibiotic use in Canadian primary care settings is not well defined.

The US National Action Plan for Combatting Antibiotic-resistant Bacteria has set a goal to reduce inappropriate antibiotic prescribing by 50% by 2020.16 The UK’s 5-year plan is to reduce overall human antibiotic use by 15% by 2024.17 Canada has yet to articulate a similar plan to reduce overall antibiotic use in humans, partly because, to date, reasonable targets for safe reduction in antibiotic use in Canadian primary care settings have not been developed. This was highlighted as a major knowledge gap in Canada in a recent report from Canada’s chief medical officer of health.2

Studies from the US using administrative data to define unnecessary antibiotic prescribing have been criticized for misclassifying diagnostic codes and potentially overestimating unnecessary antibiotic use.18 Previous approaches using administrative codes to assess unnecessary antibiotic use have involved some arbitrary classifications, resulting in misclassification errors. As a result, we recently defined expected antibiotic prescribing rates using a modified Delphi method to enable less biased evaluation of unnecessary antibiotic prescribing in Canadian primary care settings.19

Our objective in the present study was to use these predefined expected rates of antibiotic prescribing to quantify the overall and the condition-specific rates of unnecessary antibiotic prescribing in a Canadian primary care setting. These data could be used to identify targets for antimicrobial stewardship efforts to reduce ambulatory antibiotic prescribing safely and appropriately in order to slow the emergence of antimicrobial resistance.

## Methods

### Setting and population

We conducted a descriptive analysis in Ontario between Apr. 1, 2011, and Mar. 31, 2016 using the Electronic Medical Records Primary Care database (EMRPC, also known as EMRALD) held at ICES. ICES is a not-for-profit research institute and a prescribed entity under section 45 of Ontario’s *Personal Health Information Protection Act*, which authorizes ICES to collect and store personal health information securely, without consent, for the purpose of evaluating or monitoring, planning for, or managing the health system.

### Data sources

The EMRPC is a repository of electronic medical records from a convenience sample of Ontario family physicians. Data for patients in the EMRPC were linked to the Registered Persons Database, Ontario Health Insurance Plan, National Ambulatory Care Reporting System and Canadian Institute for Health Information Discharge Abstract Database. These data sets were linked with the use of unique encoded identifiers and analyzed at ICES. Geographic data were obtained from census data. Physician characteristics were obtained from the ICES physician database and Client Agency Program Enrolment database.

For patients not in EMRPC practices, we obtained patient age and sex from the Registered Persons Database, neighbourhood income quintile and rurality from the census database, and comorbidities from the Discharge Abstract Database, Johns Hopkins’ Adjusted Clinical Groups System Version 10.0 Aggregated Diagnosis Groups and Resource Utilization Bands.

For family physicians not part of the EMRPC group, we obtained physician age, sex, country of medical graduation, rurality and years in practice from the ICES physician database. Primary care funding models were from the Client Agency Program Enrolment database.

### Definitions and inclusion and exclusion criteria

Cohort creation and exclusions are summarized in Figure 1.

![Figure 1:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/8/2/E360/F1.medium.gif)

[Figure 1:](http://www.cmajopen.ca/content/8/2/E360/F1)

Figure 1: 
Flow diagram showing cohort creation. Note: EMR = electronic medical record.

#### Physicians

Physicians had to have at least 1 year of experience using the electronic medical record system.20 Physicians were included for a fiscal year if they had seen 200 or more patients in that year and had prescribed at least 7 antibiotic prescriptions in that year. We used the latter exclusion criterion because physicians who prescribe very few antibiotics (< 1 per month) likely have quite different practices or very small volumes and are not relevant to antimicrobial stewardship efforts. Physicians were included for up to 5 years if they met inclusion criteria each year.

#### Encounters

The unit of analysis was a patient–physician encounter. An encounter was defined as an entry within the electronic medical record identified by the presence of both a physician billing claim and a progress note. We grouped multiple patient visits into 1 encounter if the visits were within 7 days of each other up to a maximum of 28 days. We did this because such encounters likely represent a single illness episode. We excluded encounters if there was any use of health care services for an infectious disease diagnosis in the prior 21 days. The rationale for this exclusion was to exclude encounters in which an antibiotic prescription may have originated from a non-EMRPC physician, as these antibiotics were not captured in our data.

#### Diagnosis

We used a predefined list of expected antibiotic prescribing rates for 23 common conditions based on point estimates that were previously established from a modified Delphi process.19 All other diagnoses were excluded, as appropriateness of antibiotic prescribing could not be assessed reliably. If there was more than 1 diagnosis for a particular encounter, we used a diagnostic hierarchy of assigning the diagnosis with the highest expected rate of antibiotic prescribing (Appendix 1, Supplemental Table S1, available at [www.cmajopen.ca/content/8/2/E360/suppl/DC1](http://www.cmajopen.ca/content/8/2/E360/suppl/DC1)).19 For example, if a patient was seen on day 1 and diagnosed with a common cold and then seen again on day 3 and diagnosed with pneumonia, these visits were grouped into a single encounter and assigned a diagnosis of pneumonia. We used physician diagnosis and did not incorporate other clinical, laboratory or diagnostic imaging results.

We defined the diagnosis of an encounter by the presence of a physician billing claim from the Ontario Health Insurance Plan database (Appendix 1, Supplemental Table S2). If there was no billing claim corresponding to 1 of the 23 conditions, we searched the electronic medical record clinical notes associated with the encounter for keywords associated with the 23 included conditions. Encounters with other diagnoses were excluded from the study. We developed and validated a keyword-searching algorithm, using manual chart review, to optimize the likelihood of assigning the correct diagnosis (Appendix 1, Supplemental Figure S1; Appendix 2, available at [www.cmajopen.ca/content/8/2/E360/suppl/DC1](http://www.cmajopen.ca/content/8/2/E360/suppl/DC1)).

We grouped diagnoses into tiers as previously defined (Appendix 1, Supplemental Table S2).19 Tier 1 included conditions for which antibiotics are always indicated (expected prescribing rate 100%), tier 2a included conditions for which antibiotics are frequently indicated (expected prescribing rate 51%–99%), tier 2b included conditions for which antibiotics are sometimes indicated (expected prescribing rate 21%–50%), tier 2c included conditions for which antibiotics are rarely indicated (expected prescribing rate 1%–20%), and tier 3 included conditions for which antibiotics are never indicated (expected prescribing rate 0%).

#### Antibiotic prescriptions

Using our previously validated algorithm,23 we defined an antibiotic prescription as related to an encounter if 1 or more antibiotic prescriptions were generated within 7 days from the end of the encounter (Appendix 1, Supplemental Table S3). We have previously validated the EMRPC in patients 65 years of age or more, compared to dispensed antibiotics, to reliably identify antibiotic prescriptions prescribed by EMRPC physicians (sensitivity 85%, 95% confidence interval [CI] 84%–85%; specificity 98%, 95% CI 98%–98%).23

### Statistical analysis

We compared our study populations of patients and family physicians to all patients and family physicians in Ontario. We calculated condition-specific rates of antibiotic prescribing per 100 encounters stratified by age group (< 2 yr, 2–18 yr, 19–64 yr and ≥ 65 yr). We calculated averages of antibiotic prescribing rates, weighted by the number of encounters, by age group. Using the results from the modified Delphi panel on expected antibiotic prescribing rates (Appendix 1, Supplemental Table S1),19 we calculated the percent of unnecessary antibiotic prescribing by subtracting the expected prescribing rate for a given condition and age group from the actual prescribing rate. Tier 1 conditions were constrained to having unnecessary antibiotic prescribing rates of 0%. We did not consider antibiotic prescribing rates that were lower than expected to be “inappropriate” for this study, and, therefore, any negative unnecessary prescribing rates were reported as 0%.

Canadian physician billing claims for respiratory infections have previously been shown to have good positive predictive value in both adults and children.21,22 However, we compared several different methods of assigning a diagnosis to encounters as sensitivity analyses (Appendix 2).

### Ethics approval

This study received ethics review board approval from Public Health Ontario and the Sunnybrook Health Sciences Centre.

## Results

A total of 499 570 encounters between Apr. 1, 2011, and Mar. 31, 2016 met the inclusion criteria for the 23 predefined conditions (Appendix 1, Supplemental Table S4). The study population included 204 313 patients and was overrepresented in rural areas and included more females compared to the Ontario population (Appendix 1, Supplemental Table S5). There were notable differences in geographic representation between the study patients and the Ontario population (Appendix 1, Supplemental Figure S3). The study included 341 physicians, who were more likely to be younger, female and trained in Canada compared to all Ontario family physicians (Table 1).

View this table:
[Table 1:](http://www.cmajopen.ca/content/8/2/E360/T1)

Table 1: 
Characteristics of study physicians and all Ontario primary care physicians

There were 310 141 antibiotics prescribed during the 5-year study period. Of the 310 141 prescriptions, 37 255 (12.0%) were for tier 3 conditions, and 38 290 (12.3%) were for tier 2c conditions. The percentages were higher for children (< 18 yr), in whom 25.0% of antibiotics were prescribed for tier 3 conditions (Figure 2).

![Figure 2:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/8/2/E360/F2.medium.gif)

[Figure 2:](http://www.cmajopen.ca/content/8/2/E360/F2)

Figure 2: 
Percentage of all antibiotics prescribed, by tier classification system.19 Tier 1 = conditions for which antibiotics are always indicated (expected prescribing rate 100%), tier 2a = conditions for which antibiotics are frequently indicated (expected prescribing rate 51%–99%), tier 2b = conditions for which antibiotics are sometimes indicated (expected prescribing rate 21%–50%), tier 2c = conditions for which antibiotics are rarely indicated (expected prescribing rate 1%–20%), tier 3 = conditions for which antibiotics are never indicated (expected prescribing rate 0%). *These antibiotics were not associated with an encounter included in the study.

Based on our exclusion criteria, the appropriateness of 82 408 antibiotic prescriptions could not be assessed, as the antibiotic was associated with an excluded condition or met another study exclusion criterion. The 5 most frequent excluded diagnoses associated with an antibiotic were abdominal complaints (*International Classification of Diseases, Ninth Revision* [Clinical Modification] code 787), acne (706), diabetes mellitus (250), eczema (691) and anxiety (300). An additional 75 546 antibiotic prescriptions had no documented encounter in the electronic medical record (Table 2). There were thus 152 187 antibiotic prescriptions included in the subsequent analyses (Figure 1).

View this table:
[Table 2:](http://www.cmajopen.ca/content/8/2/E360/T2)

Table 2: 
Antibiotics prescribed, stratified by age and tier classification system,19 to 204 313 patients in Ontario, 2011/12 to 2015/16*

The most common diagnoses among physician–patient encounters for all ages were the common cold, miscellaneous nonbacterial infections and urinary tract infections (Table 3). Overall, the antibiotic prescribing rate for these 23 included conditions was 30.6%. The unnecessary prescribing rate for included encounters was 15.4%. This varied by age: it was 17.6% for those less than 2 years, 18.6% for those aged 2–18, 14.5% for those aged 19–64 and 13.0% for those aged 65 or more. The conditions with the highest rates of unnecessary antibiotic prescribing were acute bronchitis (52.6%), acute sinusitis (48.4%), acute otitis media (39.3%) and acute pharyngitis (36.7%).

View this table:
[Table 3:](http://www.cmajopen.ca/content/8/2/E360/T3)

Table 3: 
Number of patient–physician encounters by age group and condition, with observed antibiotic prescribing rate

The conditions resulting in the highest number of unnecessary antibiotic prescriptions were the common cold, acute bronchitis, acute sinusitis and miscellaneous nonbacterial infections. Overall, 79.5% of the unnecessary antibiotics were prescribed for these 4 conditions (Figure 3). Children received relatively more unnecessary antibiotic prescriptions for miscellaneous nonbacterial infections and acute otitis media, whereas adults received more for acute bronchitis and acute sinusitis.

![Figure 3:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/8/2/E360/F3.medium.gif)

[Figure 3:](http://www.cmajopen.ca/content/8/2/E360/F3)

Figure 3: 
Contribution of common conditions to overall unnecessary antibiotic use by age. Note: COPD = chronic obstructive pulmonary disease. *Included herpes simplex, herpes zoster, infectious mononucleosis, warts, other viral illnesses, ringworm, candidiasis, stomatitis and serous otitis media.

In comparing the accuracy of different definitions of assigning a diagnosis to encounters, we found that the accuracy of the diagnostic algorithm ranged from 60% to 71% (67% for the primary analysis); however, about one-third of the errors had no clinical significance and two-thirds of the errors resulted in a condition with a higher-than-expected antibiotic prescribing rate. The impact of these differences on the study outcomes were minor (Appendix 1, Supplemental Figure S2).

## Interpretation

In this study from a Canadian primary care setting, 12.0% of antibiotics were prescribed for conditions for which antibiotics are never indicated, and another 12.3% for conditions for which antibiotics are rarely indicated. We observed a rate of unnecessary antibiotic prescribing of 15.4% for 23 common conditions. This rate was modestly higher for children. The conditions with the highest unnecessary prescribing rates, and therefore the most room for improvement, were acute bronchitis, acute sinusitis and acute otitis media. However, given the frequency of visits for the common cold, acute bronchitis, miscellaneous nonbacterial infections and acute sinusitis, these conditions were responsible for 80% of all unnecessary antibiotic prescriptions.

For context, the rate of antibiotic use in the US is 838 prescriptions per 1000 people annually,24 compared to 625 in Canada,25 621 in Ontario,26 626 in England27 and 325 in Sweden.28 We observed an antibiotic prescribing rate for the common cold of 18%, comparable to the rate of 25% observed in a similar study conducted in the UK.13 We observed antibiotic prescribing rates of 78% and 70% for acute exacerbation of chronic obstructive pulmonary disease and acute pharyngitis, respectively, compared to 73% and 59%, respectively, in the UK.13 However, compared to the UK,13 we observed lower rates of prescribing for acute bronchitis (60% v. 82%), acute otitis media in children aged 2–18 years (73% v. 88%), acute sinusitis (67% v. 88%) and asthma (8% v. 47%). In the US, 13% of antibiotics were prescribed for tier 1 conditions, 36% for tier 2 conditions, 23% for tier 3 conditions, and 29% with no diagnosis in adults less than 65 years and children.12 Fleming-Dutra and colleagues11 reported that 30% of antibiotics prescribed in ambulatory care in the US were likely unnecessary. However, it is notable that, in both the UK and US studies,11,12 diagnoses for noninfectious conditions were categorized as tier 3 conditions, whereas we excluded them. Furthermore, we developed specific definitions for the tier classification based on the percent of encounters in which antibiotics were recommended for each condition by experts.19 This may partially explain the lower rates of unnecessary antibiotic prescribing observed in our study, as some conditions may have been misclassified in previous studies.18

It is evident from these previous evaluations of outpatient antibiotic prescribing that antibiotics are overused in primary care. Unnecessary antibiotic use is associated with a 20%–30% risk of adverse medication effects, most commonly nausea, vomiting, diarrhea, rash, headache and vaginitis.29,30 Shehab and colleagues8 found that 16% of emergency department visits for adverse drug events were for antibiotic-associated effects, including allergic reactions and diarrhea associated with *C. difficile*. Furthermore, overuse of antibiotics is an important driver of antimicrobial resistance, incurring a fourfold increased risk in individual patients within 1 month of antibiotic exposure.5 There are additional important public health impacts of increasing antimicrobial resistance in populations with higher antibiotic use.3

However, decisions regarding antibiotic prescribing and drivers of unnecessary prescribing are complex. Physicians are required to balance salient factors such as diagnostic uncertainty, time constraints and perceived patient expectations for antibiotics against seemingly less tangible antibiotic-related harms and the public health impact of antimicrobial resistance.2,31

Our findings suggest that we can conservatively target a reduction of 15 antibiotic prescriptions per 100 infectious disease encounters. In children, the reduction could be larger: 18 prescriptions per 100 encounters. Acute pharyngitis, acute otitis media and acute bronchitis had the highest rate of unnecessary antibiotic prescribing in children and should be high-yield targets for intervention. Our study supports that we can safely reduce total antibiotic prescribing in primary care by 12%–24% overall. Reducing antibiotic prescribing for the common cold, acute bronchitis and acute sinusitis will have the greatest impact.

Reductions in antibiotic prescribing can be achieved by incorporating evidence-based antimicrobial stewardship interventions in the community that use principles of behavioural science.32,33 Strategies that have shown effectiveness have included public commitment posters in physician offices,34 peer-comparison audit and feedback,35,36 and communications training.37 Other effective interventions, in collaboration with 1 or more of the aforementioned strategies, include clinical decision support,38 and patient and clinician education.39 Any community antimicrobial stewardship strategy must include data collection and local champions to integrate quality improvement and appropriate antibiotic prescribing into clinical ambulatory practice.40

If antibiotic prescribing for conditions that never require antibiotics were eliminated, this would reduce the number of antibiotic prescriptions in Canada by 3 million annually (12% of the approximately 25 million outpatient antibiotics prescribed annually). Targeting upper respiratory tract infections and acute bronchitis, as well as acute otitis media in children, would have the greatest impact in reducing unnecessary antibiotic use.

### Limitations

We believe there is no way to evaluate unnecessary antibiotic prescriptions reasonably at a population level in the absence of established expected prescribing rates19 and a diagnosis by the physician. As a result, many antibiotics were excluded from our analysis. We think that the approach taken in previous studies resulted in some misclassification.11,12,18 However, even without classifying half of all antibiotic prescriptions, we observed that 24% of all antibiotics were prescribed for conditions that never or rarely require antibiotics. Our validation analysis identified some diagnostic errors; however, these resulted in no clinical significance or underestimated the rate of unnecessary antibiotic prescribing, making our estimates conservative. We were unable to account for delayed prescribing practices whereby antibiotic prescriptions may have been given to patients with instructions not to take the medication unless certain criteria were met; however, this practice is not recommended for tier 3 conditions.41,42

The study population was a convenience sample of family physicians included in EMRPC. We have previously validated EMRPC to identify antibiotics accurately; however, this validation was limited to patients aged 65 or more.23 Our study physicians were more likely to be younger, female and Canadian-trained compared to all Ontario family physicians. This may limit the generalizability of our findings, but our cohort of physicians were probably less likely than all Ontario family physicians to overprescribe antibiotics, which would result in underestimates of unnecessary antibiotic prescribing.15,43

We chose to use point estimates from our previous Delphi panel results for expected antibiotic prescribing rates19 to facilitate knowledge translation into practice. However, point estimates do not fully capture the degree of error, subjectivity or uncertainty in this process. Furthermore, they may not be applicable to all patient populations and regions.

Finally, this study quantified only unnecessary initiation of antibiotic treatment and did not address other features of inappropriate antibiotic prescribing, such as selection and duration.

### Conclusion

We observed that 15.4% of infectious disease encounters in a Canadian primary care setting resulted in an unnecessary antibiotic prescription. Almost one-quarter of all antibiotic prescriptions in primary care were written for conditions for which they are never or rarely indicated. These findings should guide targets for the safe reduction of antibiotic use in primary care.

## Footnotes

*   **Competing interests:** None declared.

*   This article has been peer reviewed.

*   **Contributors:** Kevin Schwartz conceived and designed the study, and drafted the manuscript. Bradley Langford, Nick Daneman, Kevin Brown, Warren McIsaac, Karen Tu, Jennie Johnstone, Valerie Leung, Julie Wu and Gary Garber contributed to the study conception and design, and data interpretation. Branson Chen analyzed the data. Elisa Candido, Julie Wu and Michael Silverman provided expertise on methodology, and data analysis and interpretation. All of the authors revised the manuscript critically for important intellectual content, approved the final version to be published and agreed to be accountable for all aspects of the work.

*   **Funding:** This project was funded by the Physician Services Incorporated Foundation. Karen Tu receives a Research Scholar Award from the Department of Family and Community Medicine at the University of Toronto.

*   **Data sharing:** The data set from this study is held securely in coded form at ICES. Although data-sharing agreements prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at [www.ices.on.ca/DAS](http://www.ices.on.ca/DAS). The full data set creation plan and underlying analytic code are available from the authors on request, with the understanding that the programs may rely on coding templates or macros that are unique to ICES.

*   **Disclaimer:** This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). Parts of this material are based on data or information, or both, compiled and provided by the Canadian Institute for Health Information (CIHI). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES, CIHI or the Ontario MOHLTC is intended or should be inferred.

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/8/2/E360/suppl/DC1](http://www.cmajopen.ca/content/8/2/E360/suppl/DC1).

## References

1.  (2016) Tackling drug-resistant infections globally: final report and recommendations, Review on Antimicrobial Resistance (Wellcome Trust & HM Government, London (UK)).
    
    

2.  (2019) Handle with care: preserving antibiotics now and into the future — Chief Public Health Officer of Canada’s 2019 spotlight report (Public Health Agency of Canada, Ottawa) Available: [www.canada.ca/content/dam/phac-aspc/documents/corporate/publications/chief-public-health-officer-reports-state-public-health-canada/preserving-antibiotics/Final\_CPHO\_Report\_EN\_June6\_2019.pdf](http://www.canada.ca/content/dam/phac-aspc/documents/corporate/publications/chief-public-health-officer-reports-state-public-health-canada/preserving-antibiotics/Final\_CPHO\_Report_EN_June6_2019.pdf). accessed 2019 July 3.
    
    

3.  Low M, Neuberger A, Hooton TM, et al. (2019) Association between urinary community-acquired fluoroquinolone-resistant *Escherichia coli* and neighbourhood antibiotic consumption: a population-based case–control study. Lancet Infect Dis 19:419–28.
    
    

4.  Bell BG, Schellevis F, Stobberingh E, et al. (2014) A systematic review and meta-analysis of the effects of antibiotic consumption on antibiotic resistance. BMC Infect Dis 14:13.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2334-14-13&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24405683&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

5.  Costelloe C, Metcalfe C, Lovering A, et al. (2010) Effect of antibiotic prescribing in primary care on antimicrobial resistance in individual patients: systematic review and meta-analysis. BMJ 340:c2096.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDAvbWF5MThfMi9jMjA5NiI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vOC8yL0UzNjAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

6.  Ahovuo-Saloranta A, Rautakorpi UM, Borisenko OV, et al. (2014) Antibiotics for acute maxillary sinusitis in adults [withdrawn in *Cochrane Database Syst Rev* 2015;(10):CD000243]. Cochrane Database Syst Rev (2):CD000243.
    
    

7.  Smith SM, Fahey T, Smucny J, et al. (2017) Antibiotics for acute bronchitis. Cochrane Database Syst Rev (6):CD000245.
    
    

8.  Shehab N, Lovegrove MC, Geller AI, et al. (2016) US emergency department visits for outpatient adverse drug events, 2013–2014. JAMA 316:2115–25.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2016.16201&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=27893129&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

9.  Brown KA, Khanafer N, Daneman N, et al. (2013) Meta-analysis of antibiotics and the risk of community-associated *Clostridium difficile* infection. Antimicrob Agents Chemother 57:2326–32.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYWFjIjtzOjU6InJlc2lkIjtzOjk6IjU3LzUvMjMyNiI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vOC8yL0UzNjAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

10. Canadian Antimicrobial Resistance Surveillance System (2018) 2017 report (Public Health Agency of Canada, Ottawa) Available: [www.canada.ca/en/public-health/services/publications/drugs-health-products/canadian-antimicrobial-resistance-surveillance-system-2017-report-executive-summary.html](http://www.canada.ca/en/public-health/services/publications/drugs-health-products/canadian-antimicrobial-resistance-surveillance-system-2017-report-executive-summary.html). accessed 2017 Nov 14.
    
    

11. Fleming-Dutra KE, Hersh AL, Shapiro DJ, et al. (2016) Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010–2011. JAMA 315:1864–73.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2016.4151&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=27139059&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

12. Chua KP, Fischer MA, Linder JA (2019) Appropriateness of outpatient antibiotic prescribing among privately insured US patients: ICD-10-CM based cross sectional study. BMJ 364:k5092.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNjQvamFuMTZfNC9rNTA5MiI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vOC8yL0UzNjAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

13. Pouwels KB, Dolk FCK, Smith DRM, et al. (2018) Actual versus ‘ideal’ antibiotic prescribing for common conditions in English primary care. J Antimicrob Chemother 73(Suppl 2):19–26.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/jac/dkx502&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=29490060&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

14. Smith DRM, Dolk FCK, Pouwels KB, et al. (2018) Defining the appropriateness and inappropriateness of antibiotic prescribing in primary care. J Antimicrob Chemother 73(Suppl 2):ii11–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/jac/dkx503&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=29490061&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

15. Silverman M, Povitz M, Sontrop JM, et al. (2017) Antibiotic prescribing for nonbacterial acute upper respiratory infections in elderly persons. Ann Intern Med 166:765–74.
    
    

16. (2015) National Action Plan for Combating Antibiotic-resistant Bacteria (Division of Healthcare Quality Promotion, National Center for Emerging and Zoonotic Infectious Disease, Centers for Disease Control and Prevention, Atlanta) Available: [www.cdc.gov/drugresistance/pdf/national\_action\_plan\_for\_combating\_antibotic-resistant\_bacteria.pdf](http://www.cdc.gov/drugresistance/pdf/national\_action\_plan\_for_combating_antibotic-resistant_bacteria.pdf). accessed 2019 July 3.
    
    

17. (2019) Tackling antimicrobial resistance 2019–2024 the UK’s five-year national action plan (HM Government, London (UK)) Available: [https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment\_data/file/784894/UK\_AMR\_5\_year\_national\_action\_plan.pdf](https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment\_data/file/784894/UK\_AMR\_5\_year_national_action_plan.pdf). accessed 2019 July 3.
    
    

18. Pouwels KB (2019) Rapid response to: Using irrational criteria to judge the appropriateness of antibiotic prescriptions. BMJ 367:l6461.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE4OiIzNjcvZGVjMTBfMTEvbDY0NjEiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzgvMi9FMzYwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

19. Wu J, Langford B, Ha R, et al. (Feb 17, 2020) Defining appropriate antibiotic prescribing in primary care – a modified Delphi panel approach. JAMMI doi:10.3138/jammi.2019-0023, [Epub ahead of print].
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.3138/jammi.2019-0023&link_type=DOI) 

20. Tu K, Widdifield J, Young J, et al. (2015) Are family physicians comprehensively using electronic medical records such that the data can be used for secondary purposes? A Canadian perspective. BMC Med Inform Decis Mak 15:67.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

21. Hwee J, Sung L, Kwong JC, et al. (2018) Use of physician billing claims to identify infections in children. PLoS One 13:e0207468.
    
    

22. Cadieux G, Tamblyn R (2008) Accuracy of physician billing claims for identifying acute respiratory infections in primary care. Health Serv Res 43:2223–38.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1475-6773.2008.00873.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18665858&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000260877700018&link_type=ISI) 

23. Schwartz KL, Wilton AS, Langford BJ, et al. (2019) Comparing prescribing and dispensing databases to study antibiotic use: a validation study of the Electronic Medical Record Administrative data Linked Database (EMRALD). J Antimicrob Chemother 74:2091–7.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

24. Outpatient antibiotic prescriptions — United States, 2015 (Centers for Disease Control and Prevention, Atlanta) reviewed 2017 Sept. 12. Available: [www.cdc.gov/antibiotic-use/community/pdfs/Annual-Report-2015.pdf](http://www.cdc.gov/antibiotic-use/community/pdfs/Annual-Report-2015.pdf). accessed 2019 July 3.
    
    

25. (2016) Canadian Antimicrobial Resistance Surveillance System — report 2016 (Public Health Agency of Canada, Ottawa) Available: [http://healthycanadians.gc.ca/publications/drugs-products-medicaments-produits/antibiotic-resistance-antibiotique/alt/pub-eng.pdf](http://healthycanadians.gc.ca/publications/drugs-products-medicaments-produits/antibiotic-resistance-antibiotique/alt/pub-eng.pdf). accessed 2017 May 3.
    
    

26. Schwartz KL, Achonu C, Brown KA, et al. (2018) Regional variability in outpatient antibiotic use in Ontario, Canada: a retrospective cross-sectional study. CMAJ Open 6:E445–52.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiY21ham8iO3M6NToicmVzaWQiO3M6ODoiNi80L0U0NDUiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzgvMi9FMzYwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

27. Dolk FCK, Pouwels KB, Smith DRM, et al. (2018) Antibiotics in primary care in England: Which antibiotics are prescribed and for which conditions? J Antimicrob Chemother 73(Suppl 2):ii2–10.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/jac/dkx504&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=29490062&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

28. Mölstad S, Löfmark S, Carlin K, et al. (2017) Lessons learnt during 20 years of the Swedish strategic programme against antibiotic resistance. Bull World Health Organ 95:764–73.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.2471/BLT.16.184374&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

29. Smith SM, Fahey T, Smucny J, et al. (2014) Antibiotics for acute bronchitis. Cochrane Database Syst Rev (3):CD000245.
    
    

30. Kenealy T, Arroll B (2013) Antibiotics for the common cold and acute purulent rhinitis. Cochrane Database Syst Rev (6):CD000247.
    
    

31. Mehrotra A, Linder JA (2016) Tipping the balance toward fewer antibiotics. JAMA Intern Med 176:1649–50.
    
    

32. Rowe TA, Linder JA (2019) Novel approaches to decrease inappropriate ambulatory antibiotic use. Expert Rev Anti Infect Ther 17:511–21.
    
    

33. Hicks LA, King LM, Fleming-Dutra KE (2019) Improving outpatient antibiotic prescribing. BMJ 364:l289.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE2OiIzNjQvZmViMTJfOC9sMjg5IjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby84LzIvRTM2MC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

34. Meeker D, Knight TK, Friedberg MW, et al. (2014) Nudging guideline-concordant antibiotic prescribing: a randomized clinical trial. JAMA Intern Med 174:425–31.
    
    

35. Meeker D, Linder JA, Fox CR, et al. (2016) Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA 315:562–70.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2016.0275&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26864410&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

36. Hallsworth M, Chadborn T, Sallis A, et al. (2016) Provision of social norm feedback to high prescribers of antibiotics in general practice: a pragmatic national randomised controlled trial. Lancet 387:1743–52.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0140-6736(16)00215-4&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26898856&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

37. Fleming-Dutra KE, Mangione-Smith R, Hicks LA (2016) How to prescribe fewer unnecessary antibiotics: talking points that work with patients and their families. Am Fam Physician 94:200–2.
    
    

38. Gulliford MC, Prevost AT, Charlton J, et al. (2019) Effectiveness and safety of electronically delivered prescribing feedback and decision support on antibiotic use for respiratory illness in primary care: REDUCE cluster randomised trial. BMJ 364:l236.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE2OiIzNjQvZmViMTJfOS9sMjM2IjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby84LzIvRTM2MC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

39. Tonkin-Crine SK, Tan PS, van Hecke O, et al. (2017) Clinician-targeted interventions to influence antibiotic prescribing behaviour for acute respiratory infections in primary care: an overview of systematic reviews. Cochrane Database Syst Rev (9):CD012252.
    
    

40. Sanchez GV (2016) Core elements of outpatient antibiotic stewardship. MMWR Recomm Rep 65:1–12.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.15585/mmwr.rr6501e1&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26987082&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

41. Little P, Moore M, Kelly J, et al., PIPS Investigators (2014) Delayed antibiotic prescribing strategies for respiratory tract infections in primary care: pragmatic, factorial, randomised controlled trial. BMJ 348:g1606.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDgvbWFyMDVfNC9nMTYwNiI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vOC8yL0UzNjAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

42. King LM, Fleming-Dutra KE, Hicks LA (2018) Advances in optimizing antibiotic prescribing in outpatient settings. BMJ 363:k3047.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNjMvbm92MTJfNC9rMzA0NyI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vOC8yL0UzNjAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

43. Jones BE, Sauer B, Jones MM, et al. (2015) Variation in outpatient antibiotic prescribing for acute respiratory infections in the veteran population: a cross-sectional study. Ann Intern Med 163:73–80.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/M14-1933&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26192562&link_type=MED&atom=%2Fcmajo%2F8%2F2%2FE360.atom) 

*   Copyright 2020, Joule Inc. or its licensors