The role of hospital characteristics in patient safety: a protocol for a national cohort study
==============================================================================================

* Khara M. Sauro
* G. Ross Baker
* George Tomlinson
* Christopher Parshuram

## Abstract

**Background:** Substantial expenditures on health care safety programs have been justified by their goal of reducing health care associated–harm (adverse events), but adverse event rates have not changed over the past 4 decades. The objective of this study is to describe hospital-level factors that are relevant to safety in Canadian hospitals and the impact of these factors on hospital adverse events.

**Methods:** This is a protocol for a national cohort study to describe the association between hospital-level factors and adverse events. We will survey at least 90 (35%) Canadian hospitals to describe 4 safety-relevant domains, chosen based on the literature and expert consultation, namely patient safety culture, safety strategies, staffing, and volume and capacity. We will retrospectively identify hospital adverse events from a national data source. We will evaluate organization-level factors using established scales and a survey, codesigned by the study team and hospital leaders. Hospital leaders, clinical unit leaders and front-line staff will complete the surveys once a year for 3 years, with an anticipated start date of winter 2022. We will use national health administrative data to estimate the rate and type of hospital adverse events corresponding to each 1-year survey period.

**Interpretation:** Analysis of data from this project will describe hospital organizational factors that are relevant to safety and help identify organizational initiatives that improve hospital patient safety. In addition to biyearly reports to the leaders of the participating hospitals, we have a multifaceted and tailored dissemination strategy that includes integrating the knowledge users into the study team to increase the likelihood that our study will lead to improved hospital patient safety.

Hospital-associated harm occurs frequently; it is estimated that nearly 1 in 10 hospitalized patients experience hospital-associated harm, which contributes to 2.8% of hospital deaths and the loss of 4.7 million years of life in good health, globally.1–3 Health care–associated harm is often measured as adverse events (AE), defined as unintended, negative consequences of health care. Adverse events increase lengths of stay for patients, their chances of being admitted to intensive care units (ICUs) and dying in hospital, and they use more health care resources.3–14 This provides a compelling rationale for the deployment of extensive, resource-intensive initiatives to improve patient safety. Unfortunately, the best available data do not support the notion that these efforts are improving patient safety.15–17 The reasons patient safety efforts have not had a desirable return on investment are complex and multifaceted, and have not been disentangled or comprehensively explored. Patient characteristics have been extensively evaluated as risk factors for safety incidents because of their important association with AEs. Patients having surgical procedures, those with multimorbidity, those who have greater severity of illness and patients who are older are each at increased risk of experiencing AEs.3,12–14,18 However, patient characteristics constitute only 1 domain of potential risk factors for AEs and are often not modifiable.

Organization-level characteristics have been studied less frequently as risk factors for hospital AEs, but the limited evidence that exists suggests they play an important role in patient safety.19–26 Hospital-level factors that may be relevant to safety can be conceptually grouped into 4 broad domains, namely patient safety culture, patient safety strategies, staffing, and hospital volume and capacity.19–21,27,28 Evidence suggests that these domains, and the factors within, each contribute to patient safety and may be better predictors of patient safety than patient characteristics. 19–21,23,29–34 Furthermore, organization-level factors present a modifiable target to improve hospital patient safety.

To date, most safety improvement strategies have had a narrow focus on specific patient characteristics, AEs or clinical practices, and have not resulted in a meaningful change in the rate of AEs.3,15 Effective strategies to prevent hospital AEs are needed to ensure that hospital care is safe. Hospital-level organization factors present a unique and innovative target for safety improvement initiatives, but their association with the rate of hospital AEs is not well understood. Studies have been small, have used single, cross-sectional designs or have studied only a few organization or hospital factors in specific clinical areas (e.g., ICU or emergency departments).19,22,23,35–37 Therefore, high-quality evidence from appropriately powered, high-quality studies is needed to inform organizational decision-making about hospital care to improve patient safety.

The objective of this study is to describe hospital-level factors that are relevant to safety in Canadian hospitals and the impact of these factors on patient safety in hospitals. Based on existing evidence, we hypothesize that patient safety culture, staffing, and hospital capacity and volume will be related to hospital AEs. More specifically, hospitals with poor patient safety culture that are stressed, from an organization perspective (e.g., low staff-to-patient ratios that are consistently at capacity), will have more AEs than hospitals with good patient safety culture that are not stressed.

## Methods

### Study design and setting

We will survey at least 90 hospitals, which represents 35% of eligible Canadian acute care hospitals.38 We will survey hospitals once a year for 3 years, beginning in winter 2022. We will use this cohort of hospitals to describe safety-relevant domains and AEs within each hospital.

In Canada, hospital care is provided through a universally accessible health care system that is publicly funded by provincial and federal governments.39,40

### Participants and recruitment

Eligible hospitals are those that provide inpatient acute care to adults (> 18 yr), have at least 50 nonpsychiatric hospital beds and provide patient-level data to the Canadian Institute for Health Information (CIHI). National data for 2017–2018 suggest there are 256 such hospitals (Figure 1).38 We will purposively sample eligible hospitals from all Canadian provinces, (except hospitals in Quebec, as they do not provide Discharge Abstract Database data to CIHI), based on hospital characteristics, to include a mix of urban, tertiary academic and community hospitals.

![Figure 1:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/9/4/E1041/F1.medium.gif)

[Figure 1:](http://www.cmajopen.ca/content/9/4/E1041/F1)

Figure 1: 
Frequency of hospitals with adult, nonpsychiatric acute care beds in Canada (excluding Quebec).38

We have secured letters of intent to participate from more than 72 hospitals. If a hospital refuses to participate, we will invite another hospital with similar characteristics to participate. The most responsible, hospital-level decision-maker of eligible hospitals will be invited to consent to participate on behalf of their hospital. Within each consented hospital, we will include 3 distinct groups of participants, namely hospital-level decision-makers or their delegates, leaders of clinical areas within each hospital (e.g., department or division heads, medical director, unit executive directors) or their delegates (e.g., quality improvement leads, clinical unit managers), and front-line staff.

A research team member will screen the organizational charts of the eligible acute care hospitals to identify the most responsible, hospital-level decision-maker who will be invited to participate in the study. The hospital-level decision-maker will then nominate the most appropriate leader within each clinical area to participate. The clinical area leader will facilitate recruitment of front-line staff by sending emails to all front-line staff within their clinical area, as well as by posting recruitment materials on websites and in physical spaces within hospitals. This recruitment strategy will ensure that multiprofessional, front-line staff (e.g., doctors, nurses, respiratory technicians, dietitians, pharmacists) working on different shifts or schedules are able to participate and complete surveys. This approach has resulted in a 91% response rate in previous studies.41 We will obtain written consent for each participant. The consent form will describe what participation entails, how their data will be stored and safety measures for ensuring their data are secure.

Given that this is a longitudinal study, we will try to minimize risk of bias from attrition. We will invite the same participants to complete the survey every year; however, if they do not, they will be replaced by others in the same role. To ensure continued engagement, we will enter participants into a draw for $5–$10 coffee gift cards for each survey completed, which will be drawn within 3 months of survey completion at each hospital. Biyearly, we will also provide tailored audit and feedback reports and newsletters to participants.

We will obtain data for all adult patients admitted to the participating hospitals from CIHI to measure hospital AEs, volume and capacity during the study period. Figure 2 shows our plan for participant recruitment and data collection, using surveys and CIHI data.

![Figure 2:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/9/4/E1041/F2.medium.gif)

[Figure 2:](http://www.cmajopen.ca/content/9/4/E1041/F2)

Figure 2: 
Overview of study methods and design.

### Exposure

We will use 4 broad categories to describe organization-level factors; these are safety culture, safety strategies, staffing, and hospital volume and capacity. These organization-level factors will describe hospitals and relevant clinical areas, which may include ICUs, general medical units, specialty medical units, general surgical units, specialty surgical units, operating rooms and the medical response teams, as determined by the hospital leaders.

We identified the 4 exposure variables through a narrative review of the literature (Appendix 1, available at [www.cmajopen.ca/content/9/4/E1041/suppl/DC1](http://www.cmajopen.ca/content/9/4/E1041/suppl/DC1)) and through an iterative consultation process (via meetings and email for over a year) with our steering committee, which includes hospital leaders and researchers with expertise in the area of patient safety. We will use inventories and validated surveys to measure our exposure variables.

We will measure organization-level variables once a year for 3 years in each hospital, with the study start date unique for each hospital. We will assign front-line staff consenting to participate a unique study identification number and ask them to complete the surveys assessing patient safety culture and staff well-being every year for the duration of the study (3 time points). Similar methods have been used to evaluate patient safety.36 We will ask clinical area leaders or delegates to complete the survey regarding patient safety strategies yearly for the duration of the study (3 time points). We will collect staffing measures (e.g., nurse-to-patient ratios for day shift and night shift) intermittently, as guided by clinical area leaders, but on at least 2 randomly selected days each month of each year for the 3 years of the study.

#### Patient safety culture

We will use the validated Canadian Patient Safety Culture Survey Tool (Can-PSCS) to assess patient safety culture.42,43 The Can-PSCS is a 23-item survey that asks participants to rate their perceptions and opinions of patient safety using a 5-point Likert scale (strongly agree to strongly disagree). The Can-PSCS was developed based on the United States’ Agency for Health Research and Quality,44 the Patient Safety Climate in Healthcare Organizations survey37 and the Error Climate Scale.35 We chose the Can-PSCS because of its sound psychometric properties, based on validation across several settings, and because it is theory-based and specifically tailored to the Canadian context. As such, it has been adopted by Accreditation Canada, the accrediting body for health care organizations in Canada.42 We will use the Can-PSCS overall score as the primary descriptor of patient safety culture.

#### Patient safety strategies

There are no established measures of safety strategies; therefore, a de novo survey to measure patient safety strategies is needed. Three authors (K.M.S., G.R.B., C.P.) codeveloped a safety strategy survey with hospital leaders, based on a narrative review of the evidence, using standard survey development methodology, informed by existing materials (Appendix 2, available at [www.cmajopen.ca/content/9/4/E1041/suppl/DC1](http://www.cmajopen.ca/content/9/4/E1041/suppl/DC1)).41,45,46 We will ask participants to rate the adoption of and fidelity to evidence-based safety strategies within their hospital. The survey will also include questions regarding the number of dedicated staff and full-time equivalents acting in safety jobs; safety budget; organizational membership in safety organizations (e.g., accreditation, other); the existence of audit and feedback systems for safety incidents (e.g., reporting and learning systems); and the number of ongoing patient safety strategies, according to a list of safety strategies recommended for implementation47 and refined in collaboration with hospital leaders. We will test the face validity of this survey with experts in patient safety and quality improvement before distribution.

#### Staffing

Staffing variables will include nurse-to-patient ratios, total numbers of staff and full-time equivalents, staff turnover, physician-to-patient ratios, overnight physician staffing and type, and availability of medical emergency team (i.e., dedicated team to respond to codes or to calls from unit staff). We will capture these data elements from clinical area leaders using an electronic standardized data collection form. We will use the established Moral Distress Survey-Revised (MDS-R) to measure staff well-being.48

#### Volume and capacity

We will use national data, available through CIHI, to capture the number of funded beds, occupancy, the number and type of specialty units, the number of admissions, the number of deferred patients and the resource intensity used to treat patients within each hospital. The abstracted CIHI data will correspond to the year of the survey data collection so we can link data for all 4 exposure variables.

### Outcome

We will link exposure data with AE data from CIHI, hospital characteristics and patient characteristics from a retrospective cohort of patients admitted to participating hospitals. Using CIHI data, we will measure the rate of AEs in participating hospitals using algorithms for identifying AEs with the validated Canadian version of the *International Statistical Classification of Diseases and Related Health Problems, 10th Revision* (ICD-10-CA).17,49

### Predictor variables

We will collect hospital-level data from CIHI to evaluate hospital characteristics, such as geographic location, teaching status, type of hospital and urban or rural hospital. We will also use these data to compare participating hospitals to non-responding hospitals.

### Data sources and measurement

Eligible participants will sign a web-based consent form before being able to complete surveys. Survey data collection will be electronic. Participants will receive email notifications with unique links to the surveys for each of the 3 response periods. Participant-specific survey links will remain open for a 2-week period, during which time the participant may repeatedly access the online questionnaires and may save partially completed responses before submitting their completed responses. Up to 4 reminder emails will be sent to participants who have not responded. The web forms used for each questionnaire will include input guidance and completeness review before submission is accepted. Once submitted, data will not be modifiable. Communication will be to the preferred email provided by the participant and will use the preferred name, indicated by the participant at the start of the study, as we are doing in our national clinical trial ([NCT04176094](http://www.cmajopen.ca/lookup/external-ref?link_type=CLINTRIALGOV&access_num=NCT04176094&atom=%2Fcmajo%2F9%2F4%2FE1041.atom)).

Data will be managed and stored centrally at the Center for Safety Research coordinating centre at the SickKids Research Institute. The underpinning Oracle (Austin, TX) platform is housed on a secure server with current patches that are managed by the information technology services at the SickKids Research Institute.

We will link CIHI data to the survey data using the hospital identification number. We will link only aggregate data, rather than patient-level data, for each hospital.

### Statistical analysis

We will include a sample of at least 90 hospitals, based on the commitment of 72 hospitals to participate and on the representation of 35% of eligible Canadian acute care hospitals. Our previous work found that, with an AE rate of 10% (standard deviation 5%), a sample size of 90 will provide 79% power to detect a 20% decrease in the rate of AEs.4,14

We will calculate response rates using the number of responses divided by the number of participants invited to participate. Each participant can only complete the survey once; therefore, there will not be any duplicate responses from participants. The number of participants that were invited to participate will be calculated using the number of unique emails distributed.

To describe the organizational-level variables and the AEs, the unit of analysis will be the hospital. We will use means (standard deviations) and proportions (interquartile ranges) to describe the domains for each of the 3 cross-sectional outcome measurements within the study. We will use composite scores for each of the safety-relevant domains to distill multiple questions within each domain using principal component analysis and multiple correspondence analysis. We will measure differences between measurement periods to evaluate temporal trends and stability over time. Despite the use of electronic survey data completion, which includes checks for completeness, in cases where data items are missing from surveys, we will use a case-wise approach, whereby we will include the information for complete items, but will exclude the overall questionnaire variable. We will not employ statistical correction for nonrepresentative responses. We will stratify the response rates by the type of responder (i.e., hospital leader, clinical area leader, front-line staff).

We will describe the incidence of AEs as a proportion of hospital admissions and patients, and as a rate (per patient days), including the overall AE incidence and the incidence of each type of AE. We will explore the association between AE incidence and organization-level variables using logistic regression. To account for possible effect measure modifiers, we will include type of respondent and hospital characteristics, such as geographic location, teaching status, type of hospital and urban or rural hospital in our model.

We will also report response rates of front-line staff for each hospital, with the number of survey responses as the numerator and the number of staff for each of the clinical areas as the denominator.

### Ethics approval

This study has been approved by the SickKids Research Ethics Board (REB # 1000073064).

## Interpretation

In this study, we will describe organization-level factors relevant to hospital patient safety in Canada. There has been little improvement in the rate of hospital AEs over time, despite a call to arms by the Institute of Medicine and a surge in evidence around patient safety.15 Although the reason for the stagnant rates of AEs requires further investigation, a potential factor might be that, to date, patient safety initiatives have largely focused on the same, nonmodifiable, patient-level variables and limited or poorly implemented safety strategies. Regardless of the reason, it is clear that innovative approaches to improve safety among hospitalized patients are needed if we are to improve care. Organization-level factors represent an untapped, potentially modifiable means to improve the safety of hospital care.

We have developed a multifaceted and tailored dissemination strategy. We have integrated the knowledge users into the study team by establishing a steering committee to ensure the findings of our study will be useful and actionable. We will also provide biyearly reports to hospital leaders to increase the value added to participating hospitals and to minimize attrition over the 3 years of the study period. We will disseminate our findings through traditional academic avenues, namely presentations at conferences and peer-reviewed publication. This knowledge translation strategy will increase the likelihood that our study will lead to improved hospital patient safety.

### Limitations

Although this study provides a considered approach to evaluating hospital patient safety, includes a large number of hospitals and evaluates a comprehensive number of organization-level factors, it is not without limitations. We are not able to include hospitals in one of the largest provinces in Canada, Quebec. Quebec does not contribute data from its hospitals to the CIHI data set, meaning we cannot ascertain data on AEs from Quebec and, consequently, cannot evaluate the association between organization-level factors and AEs in that province. Although we have increased the rigour of our study by employing validated measures of both our exposures and outcomes, there is a risk of measurement bias. We do not anticipate any potential measurement biases differentially affecting any particular group or hospital. Finally, our recruitment strategy purposely aims to include a diverse group of hospitals that are representative of Canadian hospitals; however, there is a risk of selection bias (i.e., self-selection from hospitals with certain characteristics declining to participate). To mitigate this risk, we have engaged leaders from hospitals across the country.

### Conclusion

This study will provide a robust profile of hospital-level factors that are relevant to safety and a catalogue of safety activities adopted in Canadian hospitals. The findings of this study can help prioritize implementation of effective safety strategies and discontinuation of resource-intense, yet ineffective, programs. The rich, high-quality data from this study will inform key decisions influencing the safety of care in our hospitals and future research aimed at designing, implementing and evaluating hospital-level safety initiatives, including those safety strategies currently being used in Canadian hospitals.

## Footnotes

*   **Competing interests:** Christopher Parshuram reports holding shares in Bedside Clinical Systems. No other competing interests were declared.

*   This article has been peer reviewed.

*   **Contributors:** All authors contributed equally to the conception and design of the study. Christopher Parshuram is responsible for the study management, with substantial contributions from all remaining authors. Khara Sauro drafted the manuscript. All authors revised it critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.

*   **Data sharing:** Data will be shared upon reasonable request to the corresponding author after data collection has been completed.

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/9/4/E1041/suppl/DC1](http://www.cmajopen.ca/content/9/4/E1041/suppl/DC1)

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: [https://creativecommons.org/licenses/by-nc-nd/4.0/](https://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  GBD 2017 DALYs and HALE Collaborators (2018) Global, regional, and national disability-adjusted life-years (DALYs) for 359 diseases and injuries and healthy life expectancy (HALE) for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet 392:1859–922.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0140-6736(18)32335-3&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=30415748&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

2.  Sunshine JE, Meo N, Kassebaum NJ, et al. (2019) Association of adverse effects of medical treatment with mortality in the United States: a secondary analysis of the Global Burden of Diseases, Injuries, and Risk Factors Study. JAMA Netw Open 2:e187041.
    
    

3.  de Vries EN, Ramrattan MA, Smorenburg SM, et al. (2008) The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 17:216–23.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjE3LzMvMjE2IjtzOjQ6ImF0b20iO3M6MjE6Ii9jbWFqby85LzQvRTEwNDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

4.  Baker GR, Norton PG, Flintoft V, et al. (2004) The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ 170:1678–86.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTcwLzExLzE2NzgiO3M6NDoiYXRvbSI7czoyMToiL2NtYWpvLzkvNC9FMTA0MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

5.  Brennan TA, Leape LL, Laird NM, et al. (1991) Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 324:370–6.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJM199102073240604&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=1987460&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1991EW29600004&link_type=ISI) 

6.  Wilson RM, Runciman WB, Gibberd RW, et al. (1995) The quality in Australian Health Care Study. Med J Aust 163:458–71.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.5694/j.1326-5377.1995.tb124691.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=7476634&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1995TM31000004&link_type=ISI) 

7.  1.  Kohn LT, 
    2.  Corrigan JM, 
    3.  Donaldson MS
    
    Institute of Medicine (US) Committee on Quality of Health Care in America (2000) To err is human: building a safer health system, eds Kohn LT, Corrigan JM, Donaldson MS (The National Academies Press (US), Washington (DC)), pp 1–312.
    
    

8.  Forster AJ, Asmis TR, Clark HD, et al., Ottawa Hospital Patient Safety Study (2004) Ottawa hospital patient safety study: incidence and timing of adverse events in patients admitted to a Canadian teaching hospital. CMAJ 170:1235–40.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTcwLzgvMTIzNSI7czo0OiJhdG9tIjtzOjIxOiIvY21ham8vOS80L0UxMDQxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

9.  Forster AJ, Kyeremanteng K, Hooper J, et al. (2008) The impact of adverse events in the intensive care unit on hospital mortality and length of stay. BMC Health Serv Res 8:259.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1472-6963-8-259&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19091089&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

10. Forster AJ, Murff HJ, Peterson JF, et al. (2003) The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med 138:161–7.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-138-3-200302040-00007&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=12558354&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000181018600001&link_type=ISI) 

11. Hoonhout LH, de Bruijne MC, Wagner C, et al. (2009) Direct medical costs of adverse events in Dutch hospitals. BMC Health Serv Res 9:27.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1472-6963-9-27&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19203365&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

12. Sauro KM, Quan H, Sikdar KC, et al. (2017) Hospital safety among neurologic patients: a population-based cohort study of adverse events. Neurology 89:284–90.
    
    

13. Sauro KM, Soo A, de Grood C, et al. (2020) Adverse events after transition from ICU to hospital ward: a multicenter cohort study. Crit Care Med 48:946–53.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

14. Sauro KM, Soo A, Quan H, et al. (2020) Adverse events among hospitalized critically ill patients: a retrospective cohort study using administrative data. Med Care 58:38–44.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

15. Sauro KM, Machan M, Whalen-Browne L, et al. (Aug 30, 2021) Evolving factors in hospital safety? A systematic reivew and meta-analysis of hospital adverse events. J Patient Saf doi:10.1097/PTS.0000000000000889, [Epub ahead of print].
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/PTS.0000000000000889&link_type=DOI) 

16. Sauro KM, Ghali WA, Stelfox HT (2020) Measuring safety of healthcare: an exercise in futility? BMJ Qual Saf 29:341–4.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI5LzQvMzQxIjtzOjQ6ImF0b20iO3M6MjE6Ii9jbWFqby85LzQvRTEwNDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

17. Southern DA, Burnand B, Droesler SE, et al. (2017) Deriving ICD-10 codes for patient safety indicatiors for large-scale surveillance using administrative hospital data. Med Care 55:252–60.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

18. Zegers M, De Bruijne MC, Spreeuwenberg P, et al. (2011) Variation in the rates of adverse events between hospitals and hospital departments. Int J Qual Health Care 23:126–33.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/intqhc/mzq086&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21227956&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000288250600004&link_type=ISI) 

19. Curry LA, Brault MA, Linnander EL, et al. (2018) Influencing organisational culture to improve hospital performance in care of patients with acute myocardial infarction: a mixed-methods intervention study. BMJ Qual Saf 27:207–17.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjI3LzMvMjA3IjtzOjQ6ImF0b20iO3M6MjE6Ii9jbWFqby85LzQvRTEwNDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

20. Jacobs R, Mannion R, Davies HTO, et al. (2013) The relationship between organizational culture and performance in acute hospitals. Soc Sci Med 76:115–25.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.socscimed.2012.10.014&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23159305&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

21. Mannion R, Davies HTO, Marshall MN (2005) Cultural characteristics of “high” and “low” performing hospitals. J Health Organ Manag 19:431–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1108/14777260510629689&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16375066&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

22. Mardon RE, Khanna K, Sorra J, et al. (2010) Exploring relationships between hospital patient safety culture and adverse events. J Patient Saf 6:226–32.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/PTS.0b013e3181fd1a00&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21099551&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

23. Kline TJB, Willness C, Ghali WA (2008) Determinants of adverse events in hospitals: the potential role of patient safety culture. J Healthc Qual 30:11–7.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18257452&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

24. Singer S, Lin S, Falwell A, et al. (2009) Relationship of safety climate and safety performance in hospitals. Health Serv Res 44:399–421.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1475-6773.2008.00918.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19178583&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000264164400006&link_type=ISI) 

25. Wang X, Liu K, You L-m, et al. (2014) The relationship between patient safety culture and adverse events: a questionnaire survey. Int J Nurs Stud 51:1114–22.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.ijnurstu.2013.12.007&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24418106&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

26. Weaver SJ, Lubomksi LH, Wilson RF, et al. (2013) Promoting a culture of safety as a patient safety strategy: a systematic review. Ann Intern Med 158:369–74.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-158-5-201303051-00002&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23460092&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000316058600002&link_type=ISI) 

27. Gagliardi AR, Majewski C, Victor JC, et al. (2010) Quality improvement capacity: a survey of hospital quality managers. Qual Saf Health Care 19:27–30.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjc6IjE5LzEvMjciO3M6NDoiYXRvbSI7czoyMToiL2NtYWpvLzkvNC9FMTA0MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

28. Baker RG (2014) Governance, policy and system-level efforts to support safer healthcare. Healthc Q 17:21–6.
    
    

29. Silvera GA (2017) The moderating role of hospital size on the relationship between patient experience and patient safety. Qual Manag Health Care 26:210–7.
    
    

30. Sanchez McCutcheon A, MacPhee M, Davidson JM, et al. (2006) Staffing for safety: a synthesis of the evidence on nurse staffing and patient safety (Canadian Health Services Research Foundation, Ottawa).
    
    

31. MacPhee M, Ellis J, Sanchez McCutcheon A (2006) Nurse staffing and patient safety. Can Nurse 102:18–23.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17094364&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

32. Kumar K, Zarychanski R, Bell DD, et al., Cardiovascular Health Research in Manitoba Investigator Group (2009) Impact of 24-hour in-house intensivists on a dedicated cardiac surgery intensive care unit. Ann Thorac Surg 88:1153–61.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.athoracsur.2009.04.070&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19766800&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000270388500016&link_type=ISI) 

33. Lee A, Cheung YSL, Joynt GM, et al. (2017) Are high nurse workload/staffing ratios associated with decreased survival in critically ill patients? A cohort study. Ann Intensive Care 7:46.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/s13613-017-0269-2&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 

34. Pronovost PJ, Angus DC, Dorman T, et al. (2002) Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review. JAMA 288:2151–62.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.288.17.2151&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=12413375&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000179048100026&link_type=ISI) 

35. Hofmann DA, Mark B (2006) An investigation of the relationship between safety climate and medication errors as well as other nurse and patient outcomes. Person Psychol 59:847–69.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1744-6570.2006.00056.x&link_type=DOI) 

36. Leonard S, O’Donovan A (2018) Measuring safety culture: application of the Hospital Survey on Patient Safety Culture to radiation therapy departments worldwide. Pract Radiat Oncol 8:e17–26.
    
    

37. Singer SJ, Gaba DM, Geppert JJ, et al. (2003) The culture of safety: results of an organization-wide survey in 15 California hospitals. Qual Saf Health Care 12:112–8.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjEyLzIvMTEyIjtzOjQ6ImF0b20iO3M6MjE6Ii9jbWFqby85LzQvRTEwNDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

38. (2018) CIHI hospital beds staffed and in operation 2017–2018 (Canadian Institute for Health Information, Ottawa) Available: [https://www.cihi.ca/sites/default/files/document/beds-staffed-and-in-operation-2017-2018-en-web-revmay3.xlsx](https://www.cihi.ca/sites/default/files/document/beds-staffed-and-in-operation-2017-2018-en-web-revmay3.xlsx). accessed 2021 June 21.
    
    

39. Martin D, Miller AP, Quesnel-Vallée A, et al. (2018) Canada’s universal health-care system: achieving its potential. Lancet 391:1718–35.
    
    

40. Canada’s health care system (Health Canada, Ottawa) modified 2019 Sept 17. Available: [https://www.canada.ca/en/health-canada/services/health-care-system/reports-publications/health-care-system/canada.html](https://www.canada.ca/en/health-canada/services/health-care-system/reports-publications/health-care-system/canada.html). accessed 2021 June 21.
    
    

41. Bradley EH, Curry LA, Spatz ES, et al. (2012) Hospital strategies for reducing risk-standardized mortality rates in acute myocardial infarction. Ann Intern Med 156:618–26.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-156-9-201205010-00003&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22547471&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000303403700002&link_type=ISI) 

42. Ginsburg LR, Tregunno D, Norton PG, et al. (2014) ‘Not another safety culture survey’: using the Canadian patient safety climate survey (Can-PSCS) to measure provider perceptions of PSC across health settings. BMJ Qual Saf 23:162–70.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjIzLzIvMTYyIjtzOjQ6ImF0b20iO3M6MjE6Ii9jbWFqby85LzQvRTEwNDEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

43. Ginsburg L, Gilin D, Tregunno D, et al. (2009) Advancing measurement of patient safety culture. Health Serv Res 44:205–24.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1475-6773.2008.00908.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18823446&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000262470800013&link_type=ISI) 

44. ACSNI Human Factors Study Group (1993) Third report, Organizing for safety (Her Majesty’s Stationery Office, London (UK)).
    
    

45. Dillman D (2000) Mail and telephone surveys: the Total Design Method (John Wiley & Sons, Inc, Hoboken (NJ)), 2nd ed.
    
    

46. Lunney M, Wahby S, Sauro KM, et al. (2018) Patient satisfaction with epilepsy surgery: What is important to patients? Epileptic Disord 20:364–73.
    
    

47. Shekelle PG, Pronovost PJ, Wachter RM, et al. (2013) The top patient safety strategies that can be encouraged for adoption now. Ann Intern Med 158:365–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-158-5-201303051-00001&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23460091&link_type=MED&atom=%2Fcmajo%2F9%2F4%2FE1041.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000316058600001&link_type=ISI) 

48. Schaefer R, Zoboli EL, Vieira MM (2019) Psychometric evaluation of the Moral Distress Risk Scale: a methodological study. Nurs Ethics 26:434–42.
    
    

49. CIHI; Canadian Patient Safety Institute (2016) Measuring patient harm in Canadian hospitals: With what can be done to improve patient safety? (Canadian Institute for Health Information (CIHI), Ottawa).
    
    

*   © 2021 CMA Joule Inc. or its licensors