Regional variability in outpatient antibiotic use in Ontario, Canada: a retrospective cross-sectional study
===========================================================================================================

* Kevin L. Schwartz
* Camille Achonu
* Kevin Antoine Brown
* Bradley Langford
* Nick Daneman
* Jennie Johnstone
* Gary Garber

## Abstract

**Background:** Regional variability in antibiotic use is associated with both antibiotic overuse and antimicrobial resistance. Our objectives were to benchmark outpatient antibiotic use and to evaluate geographic variability among health regions in the province of Ontario, Canada.

**Methods:** This was a cross-sectional study of antibiotics dispensed from outpatient retail pharmacies in Ontario between March 2016 and February 2017. We analyzed variability in the number of antibiotic prescriptions dispensed per 1000 population among Ontario’s 14 health regions with crude and adjusted Poisson regression models. Adjusted models controlled for rurality, 4 physician characteristics and 6 population characteristics.

**Results:** There were 8 352 578 antibiotics dispensed during the 1-year study period or 621 per 1000 population. The most commonly prescribed antibiotic classes were narrow-spectrum penicillins, macrolides, first-generation cephalosporins and second-generation fluoroquinolones, with adult women receiving the highest rate of prescriptions: 985 antibiotic prescriptions per 1000 population. There was geographic variability in total and class-specific antibiotic use. In the health region with the highest use 778 antibiotics were dispensed per 1000 population whereas in the health region with the lowest use 534 antibiotics were dispensed per 1000 population. The adjusted marginal standardized antibiotic prescription rates for the health regions with the highest and lowest use were 787 (95% confidence interval [CI] 658–934) and 546 (95% CI 494–606) antibiotic prescriptions per 1000 population, respectively.

**Interpretation:** We described baseline antibiotic usage in Ontario over a 12-month period, noting variability among some health regions. Our findings highlight the need for interventions to optimize antibiotic use and slow the emergence of antimicrobial resistance.

Antimicrobial resistance is an urgent public health threat. A study from the United Kingdom estimates that in the absence of significant intervention, deaths from drug-resistant infections will surpass deaths from cancer by 2050, resulting in 10 million deaths annually.1 Antibiotic use is the most important modifiable risk factor associated with the development of antimicrobial resistance at both the individual patient and population levels.2,3 In Canada, approximately 92% of antibiotics are used outside of the acute care hospital setting.4 In the United States it is estimated that 30% of all antibiotics prescribed in the community are unnecessary.5 A recent study from Canada found that almost 50% of Ontario seniors with upper respiratory infections inappropriately receive antibiotics, suggesting there are opportunities to reduce community antibiotic use.6

Antimicrobial stewardship is one of Accreditation Canada’s Required Organizational Practices for hospitals. The most effective antimicrobial stewardship interventions in hospitals have involved direct engagement with prescribers to promote behaviour change.7 There is no comparable requirement to promote appropriate antibiotic use in the community. It is challenging to implement antimicrobial stewardship strategies that are used in hospitals in a community setting. Most family physicians work in small groups or solo practices with no administrative oversight, minimal access to real-time consultations with infectious disease specialists or pharmacists for antimicrobial stewardship, and limited means to collect or analyze prescribing data. Despite an increasing focus on antimicrobial resistance and stewardship, antibiotic utilization rates have not declined over the last decade.4,8,9 The ability to measure antibiotic use in the community is a critical step in implementing effective stewardship interventions. Using a proprietary population-based Ontario data set, our objective was to describe the geographic variability of antibiotic use to inform future community-based interventions.

## Methods

### Setting and design

We performed a 12-month cross-sectional study analyzing outpatient antibiotic use across Ontario, Canada, between Mar. 1, 2016, and Feb. 28, 2017, at the level of the forward sortation area, which is a geographic unit defined by the first 3 characters in the Canadian postal code. We had a specific interest in comparing antibiotic variability across Ontario’s 14 health regions. The health regions are responsible for planning, integrating and distributing government funding for all public health care services in Ontario.

### Data source

We obtained data on antibiotics dispensed by outpatient Ontario pharmacies from the GPM database from IQVIA (formerly QuintilesIMS). The data set consists of aggregated antibiotic prescription counts at the level of the forward sortation area. Forward sortation areas were assigned to the health regions in which they were located. Approximately 95% of the forward sortation areas fell completely within the boundaries of a single health region. When forward sortation area boundaries overlapped health region boundaries, the forward sortation areas were assigned to the health region that included the main population centre. Forward sortation areas without pharmacies were combined with the largest neighbouring forward sortation area. IQVIA creates the GPM database by obtaining prescriptions directly from outpatient pharmacies as well as insurance claims and sales data. The Ontario portion of this database is derived from a sample of 65% of prescriptions from 4391 outpatient pharmacies.10 Insurance claim and sales databases are merged to supplement the pharmacy data. IQVIA then applies a geospatial projection algorithm so that the antibiotic prescription counts are representative of 100% of the population.11 Briefly, IQVIA projects antibiotic prescriptions from pharmacies not included in their data set on the basis of the volume of prescriptions from surrounding pharmacies and the geographic distance between the captured and non-captured stores. Their methodology is proprietary but routinely internally validated.12 Hospital sales data from IQVIA have been previously validated. 13 Outpatient antibiotic data from IQVIA represent the most complete population-based source of data in many jurisdictions, 4,12 but to our knowledge these data have not been externally validated.

Eligible antibiotics include oral systemic antibacterial agents from the J01 class of pharmacologic agents categorized according to the World Health Organization’s Anatomic Therapeutic Classification System.14 Total and 13 class-specific antibiotic prescription counts were grouped as penicillin without β-lactamase inhibitors, penicillin with β-lactamase inhibitors, first-generation cephalosporins, second- and third-generation cephalosporins, second-generation fluoroquinolones, third-generation fluoroquinolones, macrolides, trimethoprim and/or sulfonamides, tetracyclines, lincosamides, nitrofurantoin, metronidazole and others. The antibiotic classes were further organized by most common indication: respiratory, skin, urinary and other. Although there are overlapping indications for several of these agents, these classifications provide an approximation of antibiotic selection practices by indication (Supplementary Table A1, Appendix 1, available at [www.cmajopen.ca/content/6/4/E445/suppl/DC1](http://www.cmajopen.ca/content/6/4/E445/suppl/DC1)). The population denominator was obtained from 2016 Canadian census data.15

### Covariates

The Xponent database from IQVIA was used to measure physician characteristics within each forward sortation area: physician density (defined as the number of physicians per 1000 population), proportion of family physicians (defined as the number of family physicians divided by all physicians in that area), proportion of male to female physicians, proportion of physicians in early-career stage (defined as physicians in practice for < 11 years divided by all physicians), proportion of physicians in mid-career stage (physicians in practice for 11–24 years divided by all physicians) and proportion of physicians in late-career stage (physicians in practice for > 24 years divided by all physicians). We determined whether each dispensing pharmacy was in a rural or urban area by the second character of the pharmacy’s forward sortation area.16 Using 2016 Canadian census data, we incorporated population covariates on the proportion of 6 patient age and sex groupings (boys/girls age < 18 yr, men/women age 18–64 yr and men/women age ≥ 65 yr), the proportion of residents who were of Indigenous descent, the proportion of residents who were immigrants, the proportion of residents in low-income households and the proportion of residents without a postsecondary education. Detailed variable definitions are available from the Census Program of Statistics Canada.15

### Statistical analysis

Forward sortation area was the unit of analysis. We presented outpatient antibiotic use by number of dispensed prescriptions per 1000 population for total and class-specific antibiotics. Crude and adjusted models were constructed to test the hypothesis that variability between regions existed both before and after adjusting for important population and physician differences. For crude and adjusted estimates, we built Poisson regression models of antibiotic prescription counts at the level of forward sortation area with offsets corresponding to the logarithm of the population size. To account for clustering at the level of forward sortation area causing overdispersion, all Poisson models were estimated using generalized estimating equations with an exchangeable correlation structure. In crude models, we included just the region-level variable as an explanatory variable. In adjusted models, we also included, a priori, all of the aforementioned covariates of rurality, physician density, proportion of family physicians, proportion of male physicians, proportion of early-, mid- or late-career-stage physicians, population age and sex, and proportions of low-income, Indigenous and immigrant residents and residents lacking post-secondary education. Adjusted region-specific rates were estimated using the marginal standardization method, which estimates rates as if all the regions have the same covariate distribution, equal to that of the average of all regions.17 Crude and adjusted region-specific 95% confidence intervals (CIs) were calculated using bootstrapping. 18 We used type III tests (F statistics) with 13 degrees of freedom to assess the overall statistical significance of variability between the 14 health regions. Type III tests assess the statistical significance of each health region accounting for the other 13 health regions in the model. Statistical significance was defined as *p* < 0.05. Statistical analyses were performed in SAS version 9.3 (SAS Institute).

### Ethics approval

This study received research ethics board approval from Public Health Ontario.

## Results

There were 8 352 578 antibiotics dispensed during the 12-month study period, or 621 antibiotics per 1000 population. The population age and sex distributions were similar among the health regions (Table 1). Narrow-spectrum penicillins, macrolides, first-generation cephalosporins and second-generation fluoroquinolones were the most frequently prescribed antibiotic classes. There was significant variability by patient age and sex, with women aged 65 years and older receiving 985 antibiotics per 1000 population whereas men aged 18–64 years received 441 antibiotics per 1000 population (*p* < 0.001) (Table 2). Approximately 80% of all antibiotics prescribed to children were drugs most commonly used for respiratory indications (most commonly narrow-spectrum penicillins and macrolides); women ≥ 65 years of age more commonly received urinary agents (nitrofurantoin, second-generation fluoroquinolones and trimethoprim and/or sulfonamides). Fluoroquinolones comprised 17%, 20%, 9%, 10%, 0.4% and 0.2% of all antibiotics prescribed to women aged 65 years and older, men 65 years and older, women 18–64 years of age, men 18–64 years of age, girls less than 18 years of age and boys less than 18 years of age, respectively (Figure 1).

View this table:
[Table 1:](http://www.cmajopen.ca/content/6/4/E445/T1)

Table 1: 
Population demographics in Ontario by health region in 2016

View this table:
[Table 2:](http://www.cmajopen.ca/content/6/4/E445/T2)

Table 2: 
Crude regional rate of outpatient antibiotic use per 1000 population between Mar. 1, 2016, and Feb. 28, 2017, stratified by population age and sex

![Figure 1:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/6/4/E445/F1.medium.gif)

[Figure 1:](http://www.cmajopen.ca/content/6/4/E445/F1)

Figure 1: 
Crude rate of outpatient use of oral antibiotics per 1000 population by drug class between Mar. 1, 2016, and Feb. 28, 2017, in Ontario, stratified by patient age and sex.

We observed geographic variability in antibiotic use among some health regions. In the 3 regions with the highest use 778, 742 and 673 antibiotics were dispensed per 1000 population, whereas in the 3 regions with the lowest use 549, 537 and 534 antiobiotics were dispensed per 1000 population (Figure 2). There was also variability in the use of broader spectrum antibiotics, including lincosamides, fluoroquinolones and second-and third-generation cephalosporins (Figure 3).

![Figure 2:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/6/4/E445/F2.medium.gif)

[Figure 2:](http://www.cmajopen.ca/content/6/4/E445/F2)

Figure 2: 
Health region geographic variability in crude rate of antibiotic use per 1000 population between Mar. 1, 2016, and Feb. 28, 2017.

![Figure 3:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/6/4/E445/F3.medium.gif)

[Figure 3:](http://www.cmajopen.ca/content/6/4/E445/F3)

Figure 3: 
Crude rate of antibiotic use between Mar. 1, 2016, and Feb. 28, 2017, in each health region, separated by drug class.

After multivariable adjustment, there was minimal change in the marginal standardized rates of antibiotic use (Figure 4). The region with the highest use was in the southwest of the province (Erie St. Clair), with 787 (95% CI 658–934) antibiotic prescriptions per 1000 population per year, and the region with the lowest use was in the eastern part of the province (Champlain), with 546 (95% CI 494–606) antibiotic prescriptions per 1000 population. The type III test for overall health region variability was *p* = 0.0597. Full model results are in the Supplementary Table A2 in Appendix 1.

![Figure 4:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/6/4/E445/F4.medium.gif)

[Figure 4:](http://www.cmajopen.ca/content/6/4/E445/F4)

Figure 4: 
Crude and adjusted antibiotic prescription rates per 1000 population between Mar. 1, 2016, and Feb. 28, 2017, with bootstrapped 95% confidence intervals. The adjusted rates were calculated using the marginal standardization method adjusting for rurality, physician density, proportion of family physicians, proportion of male physicians, proportion of early-, mid- or late-career-stage physicians, population age and sex, and proportions of low-income residents, Indigenous residents, residents who were immigrants and residents with a post-secondary education.

## Interpretation

We defined baseline antibiotic use in Ontario and identified some geographic variability among the province’s 14 health regions. There was minimal change when the rates were adjusted for physician and population characteristics. The overall variability between regions was not statistically significant, but the large adjusted differences observed between the regions with the highest and lowest use support the need for antimicrobial stewardship interventions.

Addressing the rising rates of antimicrobial resistance through improved antibiotic use is a local and global priority. Data showed a modest decline in antibiotic use in Canada from 1995 to 2010, suggesting that efforts to raise public and provider awareness of rising antimicrobial resistance have met with some success.19,20 However, a recent analysis showed stable use of antibiotics by Ontario seniors over the past decade,9 despite evidence of substantial overuse in this population. 6 In the US similar data have been used to identify substantial inter-state variablity in antibiotic use. Overall the US uses 833 antibiotics per 1000 population, with some of the southern states using twice the amount used by the northwestern states. The amount of outpatient antibiotics used in Ontario is similar to that in the states with the lowest use21 but substantially higher than that of many European countries.22 We identified important differences in antibiotic use among patients in different age and sex strata. These differences were partially explained by the higher use of nitrofurantoin, second-generation fluoroquinolones, and trimethoprim and/or sulfonamides, probably reflecting antibiotic prescribing for urinary infections.

The explanations for the variability in antibiotic use observed in this and other studies are complex. Potential contributing factors include patient complexity, patient expectations, socioeconomic status, remoteness and physician access, as well as both patient and physician knowledge. In this study we incorporated some regional physician and population variables that could account for population differences and diffferences in health care access. A number of studies have suggested that prescriber factors are the key driver of higher antibiotic use. Several Canadian studies have identified physician practice type, practice volume, later career stage and prior prescribing patterns as significant predictors of high or inappropriate antibiotic use.23–25 Similarly, a study of a large Veteran Affairs population in the US showed that certain providers tended to prescribe antibiotics for acute respiratory infection while others did not, independent of patient and location factors.26 Therefore, the geographic variability we observed may be explained by physician prescribing behaviours being more similar among physicians in close proximity than among those in different health regions.

Overuse of antibiotics has multiple downstream and long-term public health implications, making antimicrobial stewardship an important patient safety initiative. Numerous studies have identified the association of antibiotic use with resistance, at both ecological and individual-patient levels.2,3 Studies from the US have identified the importance of the geographic variability of antibiotic use on resistance rates of *Streptococcus pneumoniae* as well as community-associated *Clostridium difficile* infections.27,28 Furthermore, it is estimated that there are 4 emergency department visits for adverse drug events for every 1000 individuals, with antibiotics representing one of the most common culprits.29 Inappropriate inpatient antibiotic use has been directly associated with adverse patient outcomes.30

The geographic variability in antibiotic use has implications for community stewardship interventions. A potential provincial strategy could be to initiate a stewardship program targeting the highest prescribers, as changing the behaviour of this group would be expected to have the greatest impact.31 However, it is probable that many prescribers in all regions overprescribe antibiotics, and therefore population-wide interventions would also be expected to produce benefits. Another potential strategy is to target specific high-risk agents such as fluoroquinolones. Health Canada recently issued a warning about serious adverse reactions to fluoroquinolones including tendinopathy, peripheral neuropathy and central nervous system disorders,32 and we observed some regional variability in fluoroquinolone use. However, if only certain drug classes are targeted, overall antibiotic use is unlikely to decline because of the squeezing-of-the-balloon effect, where use of alternative antibiotics tends to increase.33

Multiple studies have been published on a variety of interventions for outpatient antibiotic use. The best evidence is for communication skills training, point-of-care diagnostics and peer-comparison feedback.31,34–37 However, multifaceted strategies targeting both the general public and the prescriber are likely to optimize the impact of community antimicrobial stewardship programs.38 This study supports the need for a province-wide evidence-based antimicrobial stewardship program, with a strong surveillance component, to improve the appropriate use of outpatient antibiotics and slow the emergence of drug-resistant infections. Personalized peer-comparison feedback, combined with education of the general public, has the greatest potential to significantly reduce unnecessary antibiotic use; however, multiple studies have demonstrated the lack of sustained impacts, highlighting the fact that programs require ongoing data collection, education and feedback if they are to be successful.31,36,38,39

### Limitations

This study assessed population-level antibiotic use of more than 13 million people and demonstrated regional variability, but it has some limitations. The data are based on a subset of prescriptions from pharmacies, claims and sales. IQVIA then extrapolates the data to estimate population use using a proprietary geospatial algorithm. IQVIA data are the most complete source of outpatient population-based antibiotic data in many jurisdictions. 4,12,21,40 However, this database has not been externally validated and it is possible that the estimates of antibiotic use were less precise in certain regions with poorer coverage. Pharmacy dispensing data may not accurately represent antibiotic consumption as patient adherence can vary. We were unable to account for other potentially important predictors of antibiotic variability such as patient expectations and comorbidities. Our overall measure of regional variability was not statistically significant and further observation of regional trends in antibiotic use over time is warranted. We were also unable to assess the appropriateness of antibiotic prescribing from these data as they do not capture patient visits or diagnoses. Other sources of patient-level data will be required to assess appropriateness.

### Conclusion

We have identified some geographic variability in outpatient antibiotic use among health regions in Ontario. These data provide an important benchmark for expansion of a provincial outpatient antimicrobial stewardship program and highlight opportunities for interventions to optimize antibiotic use.

## Acknowledgements

The authors thank Steven Johnson and Sean Marshall from Public Health Ontario for creating Figure 2.

## Footnotes

*   **Competing interests:** None declared.

*   This article has been peer reviewed.

*   **Contributors:** All authors contributed substantially to the conception and design of the study. Camille Achonu performed the analysis and Kevin Schwartz drafted the manuscript. All authors critically revised the manuscript, provided approved the version to be published and agreed to act as guarantors for the work.

*   **Funding:** This study was funded by Public Health Ontario.

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/6/4/E445/suppl/DC1](http://www.cmajopen.ca/content/6/4/E445/suppl/DC1).

## References

1.  O’Neill J (2016) Tackling drug-resistant infections globally: final report and recommendations (Wellcome Trust & HM Government, London (UK)).
    
    

2.  Costelloe C, Metcalfe C, Lovering A, et al. (2010) Effect of antibiotic prescribing in primary care on antimicrobial resistance in individual patients: systematic review and meta-analysis. BMJ 340:c2096.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNDAvbWF5MThfMi9jMjA5NiI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNi80L0U0NDUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

3.  Bell BG, Schellevis F, Stobberingh E, et al. (2014) A systematic review and meta-analysis of the effects of antibiotic consumption on antibiotic resistance. BMC Infect Dis 14:13.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2334-14-13&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24405683&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

4.  (2017) Canadian Antimicrobial Resistance Surveillance System: 2017 report (Public Health Agency of Canada, Ottawa) Available: [www.canada.ca/en/public-health/services/publications/drugs-health-products/canadian-antimicrobial-resistance-surveillance-system-2017-report-executive-summary.html](http://www.canada.ca/en/public-health/services/publications/drugs-health-products/canadian-antimicrobial-resistance-surveillance-system-2017-report-executive-summary.html). accessed 2017 Nov 14.
    
    

5.  Fleming-Dutra KE, Hersh AL, Shapiro DJ, et al. (2016) Prevalence of inappropriate antibiotic prescriptions among us ambulatory care visits, 2010–2011. JAMA 315:1864–73.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2016.4151&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=27139059&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

6.  Silverman M, Povitz M, Sontrop JM, et al. (2017) Antibiotic prescribing for nonbacterial acute upper respiratory infections in elderly persons. Ann Intern Med 166:765–74.
    
    

7.  (2012) Policy statement on antimicrobial stewardship by the Society for Healthcare Epidemiology of America (SHEA), the Infectious Diseases Society of America (IDSA), and the Pediatric Infectious Diseases Society (PIDS). Infect Control Hosp Epidemiol 33:322–7.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1086/665010&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22418625&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

8.  Glass-Kaastra SK, Finley R, Hutchinson J, et al. (2014) Longitudinal surveillance of outpatient beta-lactam antimicrobial use in Canada, 1995 to 2010. Can J Infect Dis Med Microbiol 25:107–12.
    
    

9.  Tan C, Graves E, Lu H, et al. (2017) A decade of outpatient antimicrobial use in senior residents of Ontario. CMAJ Open 5:E878–85.
    
    

10. (2017) 2017 Annual report: putting patients first (Ontario College of Pharmacists, Toronto) Available: [http://ocpannualreport2017.ca/](http://ocpannualreport2017.ca/). accessed 2018 Aug 21.
    
    

11. Boardman C inventor, IMS Health Incorporated, assignee. (Feb 6, 2007) System and method for estimating product distribution using a product specific universe. US patent. 7,174,304.
    
    

12. Hicks LA, Bartoces MG, Roberts RM, et al. (2015) US outpatient antibiotic prescribing variation according to geography, patient population, and provider specialty in 2011. Clin Infect Dis 60:1308–16.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/cid/civ076&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25747410&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

13. Tan C, Ritchie M, Alldred J, et al. (2016) Validating hospital antibiotic purchasing data as a metric of inpatient antibiotic use. J Antimicrob Chemother 71:547–53.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/jac/dkv373&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26546668&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

14. WHO Collaborating Centre for Drug Statistics and Methodology (2012) Guidelines for ATC Classification and DDD Assignment 2013 (World Health Organization, Oslo) Available: [https://www.whocc.no/filearchive/publications/1_2013guidelines.pdf](https://www.whocc.no/filearchive/publications/1_2013guidelines.pdf). accessed 2017 June 21.
    
    

15. Data products, 2016 census (Statistics Canada, Ottawa) [modified 2018]. Available: [https://www12.statcan.gc.ca/census-recensement/2016/dp-pd/index-Eng.cfm](https://www12.statcan.gc.ca/census-recensement/2016/dp-pd/index-Eng.cfm). accessed 2018 May 23.
    
    

16. 1.  Halseth G, 
    2.  Halseth R
    
    du Plessis V, Beshiri R, Bollman R, et al. (2004) in Building for success: explorations of rural community and rural development, Definitions of rural, eds Halseth G, Halseth R (Brandon University: Rural Development Institute & Canadian Rural Revitalization Foundation), pp 51–80.
    
    

17. Muller CJ, MacLehose RF (2014) Estimating predicted probabilities from logistic regression: different methods correspond to different target populations. Int J Epidemiol 43:962–70.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/ije/dyu029&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24603316&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

18. Hesterberg T (2011) Bootstrap. Wiley Interdiscip Rev Comput Stat 3:497–526.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/wics.182&link_type=DOI) 

19. Finley R, Glass-Kaastra SK, Hutchinson J, et al. (2013) Declines in outpatient antimicrobial use in Canada (1995–2010). PLoS One 8:e76398.
    
    

20. McKay RM, Vrbova L, Fuertes E, et al. (2011) Evaluation of the Do Bugs Need Drugs? program in British Columbia: Can we curb antibiotic prescribing? Can J Infect Dis Med Microbiol 22:19–24.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22379484&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

21. Hicks LA, Taylor THJ, Hunkler RJUS (2013) Outpatient antibiotic prescribing, 2010. N Engl J Med 368:1461–2.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJMc1212055&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23574140&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000317333600031&link_type=ISI) 

22. Glass-Kaastra SK, Finley R, Hutchinson J, et al. (2014) Variation in outpatient oral antimicrobial use patterns among Canadian provinces, 2000 to 2010. Can J Infect Dis Med Microbiol 25:95–8.
    
    

23. Hutchinson JM, Foley RN (1999) Method of physician remuneration and rates of antibiotic prescription. CMAJ 160:1013–7.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTYwLzcvMTAxMyI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNi80L0U0NDUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

24. Silverman M, Povitz M, Sontrop JM, et al. (2017) Antibiotic prescribing for nonbacterial acute upper respiratory infections in elderly persons. Ann Intern Med 167:758–9.
    
    

25. Daneman N, Campitelli MA, Giannakeas V, et al. (2017) Influences on the start, selection and duration of treatment with antibiotics in long-term care facilities. CMAJ 189:E851–60.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTg5LzI1L0U4NTEiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzYvNC9FNDQ1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

26. Jones BE, Sauer B, Jones MM, et al. (2015) Variation in outpatient antibiotic prescribing for acute respiratory infections in the veteran population: a cross-sectional study. Ann Intern Med 163:73–80.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/M14-1933&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26192562&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

27. Hicks LA, Chien YW, Taylor TH Jr., et al. (2011) Outpatient antibiotic prescribing and nonsusceptible *Streptococcus pneumoniae* in the United States, 1996–2003. Clin Infect Dis 53:631–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/cid/cir443&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21890767&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

28. Dantes R, Mu Y, Hicks LA, et al. (2015) Association between outpatient antibiotic prescribing practices and community-associated *Clostridium difficile* infection. Open Forum Infect Dis 2:ofv113.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/ofid/ofv113&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26509182&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

29. Shehab N, Lovegrove MC, Geller AI, et al. (2016) US emergency department visits for outpatient adverse drug events, 2013–2014. JAMA 316:2115–25.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2016.16201&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=27893129&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

30. Tamma PD, Avdic E, Li DX, et al. (2017) Association of adverse events with antibiotic use in hospitalized patients. JAMA Intern Med 177:1308–15.
    
    

31. Hallsworth M, Chadborn T, Sallis A, et al. (2016) Provision of social norm feedback to high prescribers of antibiotics in general practice: a pragmatic national randomised controlled trial. Lancet 387:1743–52.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0140-6736(16)00215-4&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26898856&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

32. (2017) Fluoroquinolones — risk of disabling and persistent serious adverse reactions (Health Canada, Ottawa) Available: [www.healthycanadians.gc.ca/recall-alert-rappel-avis/hc-sc/2017/61900a-eng.php](http://www.healthycanadians.gc.ca/recall-alert-rappel-avis/hc-sc/2017/61900a-eng.php). accessed 2017 Oct 26.
    
    

33. Hernandez-Santiago V, Marwick CA, Patton A, et al. (2015) Time series analysis of the impact of an intervention in Tayside, Scotland to reduce primary care broad-spectrum antimicrobial use. J Antimicrob Chemother 70:2397–404.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/jac/dkv095&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25953807&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

34. Drekonja DM, Filice GA, Greer N, et al. (2015) Antimicrobial stewardship in outpatient settings: a systematic review. Infect Control Hosp Epidemiol 36:142–52.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1017/ice.2014.41&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25632996&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

35. Meeker D, Linder JA, Fox CR, et al. (2016) Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial. JAMA 315:562–70.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2016.0275&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26864410&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

36. Linder JA, Meeker D, Fox CR, et al. (2017) Effects of behavioral interventions on inappropriate antibiotic prescribing in primary care 12 months after stopping interventions. JAMA 318:1391–2.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2017.11152&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=29049577&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

37. Tonkin-Crine SK, Tan PS, van Hecke O, et al. (2017) Clinician-targeted interventions to influence antibiotic prescribing behaviour for acute respiratory infections in primary care: an overview of systematic reviews. Cochrane Database Syst Rev (9):CD012252.
    
    

38. Cross EL, Tolfree R, Kipping R (2017) Systematic review of public-targeted communication interventions to improve antibiotic use. J Antimicrob Chemother 72:975–87.
    
    

39. Gerber JS, Prasad PA, Fiks AG, et al. (2014) Durability of benefits of an outpatient antimicrobial stewardship intervention after discontinuation of audit and feedback. JAMA 312:2569–70.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2014.14042&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25317759&link_type=MED&atom=%2Fcmajo%2F6%2F4%2FE445.atom) 

40. Fleming-Dutra KE, Bartoces M, Roberts RM, et al. (2018) Characteristics of primary care physicians associated with high outpatient antibiotic prescribing volume. Open Forum Infect Dis 5:ofx279.
    
    

*   Copyright 2018, Joule Inc. or its licensors