Emergency department interventions that could be conducted in subacute care settings for patients with nonemergent conditions transported by paramedics: a modified Delphi study
================================================================================================================================================================================

* Ryan P. Strum
* Walter Tavares
* Andrew Worster
* Lauren E. Griffith
* Andrew P. Costa

## Abstract

**Background:** As the number of patients with nonemergent conditions who are transported by paramedics continues to increase in Ontario, redirecting specific patients to subacute settings may be more beneficial and suitable for both patients and emergency departments. We aimed to evaluate whether emergency department interventions conducted on patients with nonemergent conditions who are transported by paramedics could be conducted in subacute health centres.

**Methods:** We conducted a RAND/UCLA modified Delphi study in Ontario between Oct. 13 and Dec. 19, 2020. We used purposive sampling to recruit practising emergency and primary care physicians for an expert panel. We abstracted interventions given to adult patients with nonemergent conditions (18 yr of age or older) who were transported by paramedics to an emergency department from the National Ambulatory Care Reporting System (NACRS) database (Jan. 1, 2014, to Mar. 31, 2018). Participants in the expert panel rated the suitability of the 150 most frequently recorded emergency department interventions from the NACRS database, for completion in subacute health care centres. We set consensus at 70% agreement.

**Results:** We invited 25 physician experts, 21 of whom consented to participate; 20 physicians completed round 1, and 18 physicians completed both rounds. After 2 rounds, consensus was reached on 146 (97.3%) interventions; 103 interventions (68.7%) were suitable for subacute centres, 43 (28.7%) for only the emergency department and 4 (2.6%) did not receive consensus. For subacute centres, all 103 interventions were rated for urgent care centres; walk-in medical centres were applicable for 46 (30.6%) interventions and clinics led by nurse practitioners for 47 (31.3%) interventions.

**Interpretation:** Most interventions provided to patients with nonemergent conditions transported by paramedics to emergency departments were identified as suitable for urgent care clinics, with one-third being suitable for either walk-in medical centres or clinics led by nurse practitioners. This study has potential to inform a patient classification model for paramedic-initiated redirection of patients from emergency departments, although further contextualization is required for this to be implemented in clinical practice.

**Study registration:** ID [ISRCTN22901977](http://www.cmajopen.ca/external-ref?link_type=ISRCTN&access_num=ISRCTN22901977).

Paramedic services are increasingly transporting patients with nonemergent conditions (i.e., nonemergent patients) to the emergency department when primary health care facilities may be more beneficial for their care.1,2 In Ontario, patients with nonemergent conditions account for 60% of all patients transported by ambulance, of which 74% are discharged the same day.3 Initiatives by paramedic services,4 government5 and researchers6 have not decreased paramedic transports for nonemergent visits to emergency departments: from 2014 to 2017, usage has increased by 12% (456 510 to 511 801) in Ontario.7 Increasing visits to the emergency department have outpaced population growth in Ontario by more than double (13.6% v. 6.2%, respectively),8 which suggests that use of emergency departments has broadened. Broadened use of paramedic services by nonemergent patients and a legislative requirement to transport all patients to the emergency department regardless of acuity are exacerbating the problem.9,10

Redirecting nonemergent patients to subacute care centres instead of emergency departments may offer a feasible solution to prevent some nonemergent visits.11 Patient redirection has been successful in Canada; a computer algorithm to direct nonemergent visits from emergency departments to primary care centres not only left patients satisfied with the care they received (84%), but was also described as a safe strategy (only 5.9% of 980 diverted patients had unexpected health care visits to the emergency department, none for severe complications).12 Internationally, subacute centres, such as urgent care clinics, have reduced the likelihood of visits to the emergency department for lower acuity conditions and have shown that they can perform treatments equivalent to emergency departments for minor illnesses and traumatic injuries at a lower cost.13–16 Redirection to subacute care centres by paramedics may have beneficial long-term implications by reducing paramedic transport consumption and can have higher cost-effectiveness than transport to an acute centre emergency department.17–20

Evidence to support redirecting patients transported by paramedics to subacute centres is inconclusive, and international findings may not be generalizable across Canada. 21 Part of the difficulty arises from an absence of a suitable patient classification for evaluating which patients transported by paramedic services could have been redirected. 20 Identification of which interventions patients receive in an emergency department visit could be an omitted fundamental characteristic to classify patient suitability for redirection to the emergency department. Inclusion of the main emergency department intervention could support the organization of where patients who are potentially redirectable could be transported. Although interventions alone cannot dictate patient eligibility for redirection, scaffolding emergency department interventions that are suitable for subacute centres with additional patient contextualization (i.e., age, acuity and presenting complaint) could construct a novel patient classification specific for paramedic redirection. Therefore, our objective was to establish consensus on a set of emergency department interventions received by nonemergent patients transported by paramedics that could be conducted in subacute health care centres.

## Methods

### Study design

We used a RAND/UCLA modified Delphi study design from Oct. 13, 2020, to Dec. 19, 2020, to evaluate consensus on physician interventions in the emergency department that could be conducted in alternative subacute health centres. 22–24 This methodology allowed us to assess a collective group’s judgments on patient procedures and facilitate group discussion between rounds.24 From the National Ambulatory Care Reporting System (NACRS) emergency department database, we generated a list of the 150 most frequently recorded *Canadian Classification of Health Interventions* (CCI) main intervention codes on nonemergent adult patients (18 yr or older) who were transported to hospital by paramedics in Ontario from Jan. 1, 2014, to Mar. 31, 2018, to be evaluated through 2 rounds of ratings.3,25

### Participants

We used purposive sampling to select 25 primary care and emergency physicians who were currently or recently practising in Ontario, Canada, for our Delphi expert committee.3 We sought physicians who had either extensive medical experience, academic experience or a leadership role in oversight of paramedic practice to ensure they could offer high-quality comprehension when evaluating emergency department interventions. All selected experts were emailed a study information package (objective, purpose and contribution), and those who participated gave informed consent before beginning the modified Delphi.

We only recruited physicians to participate because all interventions included in our study were performed by physicians. We excluded all other types of practitioners (including paramedics) to reduce any potential bias of experts evaluating interventions that may not be within the practitioner’s scope of practice. We determined a priori that the Delphi expert committee must be composed of at least 10 physicians, with representation from both emergency and primary care disciplines to increase the reliability of group judgments.26 Once an expert was recruited, they were asked to complete a demographic questionnaire. Only physicians who completed at least 1 round were included in the Delphi expert committee and were provided a $75 e-gift card for participation. We recruited experts during the period from Oct. 13 to Nov. 5, 2020.

### Data source

The 150 most frequently recorded emergency department interventions that we included represented 95.5% (1 259 998/1 319 388) of all interventions recorded in NACRS during the study period. We determined a priori that our intervention list should encompass at least 95% of total interventions in the study cohort to increase face validity. The NACRS database contains a population-level collection of hospital administrative records for Ontario.

We considered nonemergent patients as having a Canadian Triage and Acuity Score (CTAS, an ordinal scale that ranges from 1 to 5, with a score of 1 representing the most emergent [resuscitation] and 5 the least urgent [nonurgent]) of 3 (urgent) to 5 (nonurgent) based on clinical judgment (R.P.S. and A.W.).27 All recorded CTAS scores were assigned upon entry to the emergency department by an emergency department or triage nurse.

### Delphi process

The RAND/UCLA modified Delphi method is a strategy that analyzes collective expert judgments to produce superior results than any one expert would, which results in increased content validity.28 We used the secure and encrypted Check-Market software tool program to develop and administer the study questionnaires (described below) to the experts. All data were stored by the investigators on encrypted servers.

In the questionnaires, interventions were presented in 6 subsections based on their section of the CCI Tabular List (2018, volume 3 categorization): physical or physiologic therapeutic interventions (Section 1); diagnostic interventions (Section 2); diagnostic imaging interventions (Section 3); cognitive, psychosocial and sensory therapeutic interventions (Section 6); other health care interventions (Section 7); and therapeutic interventions strengthening the immune system (Section 8).25 We did not include obstetrical and fetal interventions (Section 5) because we did not identify any in the study cohort.

For each intervention, experts were asked to rate whether the intervention should be conducted exclusively in the emergency department or if it could be conducted in a subacute health care centre. If an expert indicated an intervention suitable for a subacute centre, they were asked if it could be conducted in an urgent care centre, a walk-in medical centre or a clinic led by nurse practitioners (multiple selections were permitted). We selected these subacute centres because they represented the most feasible centres that patients could be redirected to when transported by paramedic services, their services target nonemergent events, they are abundant in Ontario and, at present, they do not receive patients by ambulance.

Standardized definitions of each destination were provided to minimize any heterogeneity in expert interpretation of a function of the health care centre. In addition, we provided descriptions of staffing, imaging and nonclinical specialty service abilities at each type of subacute centre to increase intermember consistency.

We conducted 2 modified Delphi rounds between Nov. 6 and Dec. 19, 2020. Between the 2 rounds, we (R.P.S. and A.P.C.) hosted a videoconference debrief (Dec. 9, 2020) with the Delphi expert committee to share the results of round 1 and facilitate a discussion on the interventions that did not reach consensus.24 The videoconference was not recorded, but study investigators were permitted to take notes.3

### Statistical analysis

We considered consensus as any intervention receiving 70% or greater agreement among all experts for an individual health care centre (either emergency department or subacute centre). We collected all expert ratings from round 1, extracted the data of individual reports (CheckMarket software, extracted by R.P.S.) and composed a general feedback form that contained aggregate percentage agreement of all interventions reaching consensus and those that did not. Only 1 submission per expert was accepted.

Round 2 of the modified Delphi included all emergency department interventions that did not receive consensus in round 1. The expert ratings from round 2 served as the final consensus level on the residual interventions.

### Ethics approval

This study received a research ethics board exemption waiver from the Hamilton Integrated Research Ethics Board (review reference 2020-11451-GRA).

## Results

We invited 25 physicians who met the selection criteria to participate in the study; 21 accepted and consented to participate. Twenty experts completed round 1 and comprised the Delphi expert committee for our study. Figure 1 shows the flow of recruitment and modified Delphi rounds in the study.

![Figure 1:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/10/1/E1/F1.medium.gif)

[Figure 1:](http://www.cmajopen.ca/content/10/1/E1/F1)

Figure 1: 
Study course of recruitment and 2 rounds of the RAND/UCLA modified Delphi consensus survey.

Most of the experts were male (*n* = 14, 70%) and reported their primary medical practice as emergency medicine (*n* = 16, 80%), with the remaining experts practising in family medicine (*n* = 3, 15%) or both (*n* = 1, 5%). The characteristics of the Delphi expert committee are shown in Table 1.

View this table:
[Table 1:](http://www.cmajopen.ca/content/10/1/E1/T1)

Table 1: 
Demographic characteristics of the Delphi expert committee in the RAND/UCLA modified Delphi consensus process

In round 1, 139 (92.7%) interventions achieved a 70% consensus agreement among all experts for use in at least a single subacute health care centre. All interventions included in round 1 were considered and rated by all 20 participating experts. The remaining 11 interventions that did not achieve consensus were included in round 2, all of which were physical or physiologic therapeutic interventions. In round 2, 7 additional interventions reached consensus from experts for use in at least a single subacute health care centre, with the remaining 4 not reaching consensus. Two experts who completed round 1 did not complete round 2. Consensus results in the RAND/UCLA modified Delphi exercise are shown in summary by round in Table 2.

View this table:
[Table 2:](http://www.cmajopen.ca/content/10/1/E1/T2)

Table 2: 
Emergency department interventions receiving consensus through each round of the RAND/UCLA modified Delphi survey

Of emergency department interventions that achieved overall consensus, 103 (68.7%) were rated as suitable for a subacute health care centre (Table 3). Of the remaining 47 interventions, 43 (28.7%) were rated as appropriate only for the emergency department, and 4 interventions did not reach consensus (2.6%). All 103 interventions were deemed suitable for an urgent care centre, of which 46 were suitable for a walk-in medical centre and 47 for a clinic led by nurse practitioners.

View this table:
[Table 3:](http://www.cmajopen.ca/content/10/1/E1/T3)

Table 3: 
Health care centre that could conduct emergency department interventions receiving consensus in the RAND/UCLA modified Delphi process

Of interventions requiring diagnostic imaging (Section 3), all magnetic resonance imaging (MRI) and computed tomography (CT) interventions were identified as only suitable for the emergency department, whereas the remaining 2 imaging categories (radiography and ultrasonography) were rated appropriate for urgent care centres. All interventions of CCI Sections 7 (other health care interventions) and 8 (therapeutic interventions strengthening the immune system) were determined to be appropriate for subacute health care centres and nearly all interventions of Section 2 (diagnostic interventions) and 6 (cognitive, psychosocial and sensory therapeutic interventions). The 4 interventions that did not receive consensus ranged in ratings of 50%–66% among experts. All interventions receiving consensus for any of the 3 subacute health care centres are shown in Appendix 1, available at [www.cmajopen.ca/content/10/1/E1/suppl/DC1](http://www.cmajopen.ca/content/10/1/E1/suppl/DC1), and results of the RAND/UCLA modified Delphi agreement ratings for all interventions are shown in Appendix 2, available at [www.cmajopen.ca/content/10/1/E1/suppl/DC1](http://www.cmajopen.ca/content/10/1/E1/suppl/DC1).

## Interpretation

Our study showed consensus by primary and emergency care physicians on what clinical interventions commonly performed in the emergency department for nonemergent patients transported by paramedics are suitable for alternative settings of subacute health care. Specifically, 68.7% of included emergency department interventions were rated as suitable for conducting in urgent care centres, 30.7% in walk-in medical centres and 31.3% for clinics led by nurse practitioners, whereas 2.6% did not receive consensus.

Our results are consistent with previous research that suggested that urgent care centres and similar subacute centres can be reasonable avenues for treatment of nonemergent conditions in patients who would otherwise be directed to the emergency department.14,29 There is an absence of evidence that measures the appropriateness of which emergency department interventions could be conducted in subacute settings, because most articles analyze patient conditions, diagnostics and medication administration.14,15,30 A 2010 study using data from retail medical clinics, urgent care centres and emergency departments in the United States reported that 13.7% to 27.1% of all patients in the emergency department could be safely managed by urgent care; however, the types of interventions were not reported.14 Using subacute centres to offset visits to the emergency department has been evaluated amply but has focused heavily on outcomes of patient satisfaction and cost avoidances;14,29 quality of care, care received and simulation modelling may be more important indicators for supporting models for paramedic redirection of patients.31,32

That most of the included interventions were found to be appropriate for subacute centres acknowledges the confidence that study experts have in the ability of subacute centres to provide emergency department interventions. Of interventions that were rated for the emergency department only, many required sedation practices, intensive monitoring or advanced emergency physician skills. The 4 interventions that did not receive consensus all shared the same intervention procedure of using a reduction technique to treat an injury. Of Section 3 interventions involving diagnostic imaging, equipment was determined as the limiting factor (not injury site or physician interpretation).

An overarching goal of our study was to determine if we could achieve consensus on which emergency department interventions could be performed elsewhere, such that an epidemiologic patient classification could be constructed to inform redirection by paramedics. We recognize that interventions alone are insufficient considerations for such redirection programs. However, in combination with other indicators (e.g., contextualized patient features) and supports (e.g., education), knowledge of interventions suitable for subacute health care centres has the potential to support the construction of a patient classification model for paramedic-initiated redirection from the emergency department. Future research is required to incorporate additional patient and administrative information into such a classification to provide contextualization before evaluating its validity for clinical guidance. Our findings contribute toward informing the circumstances in which paramedic service–based programs intended to support redirecting patients bound for the emergency department may be feasible and appropriate.

### Limitations

Our Delphi expert panel of emergency physicians was mostly male, a limitation of purposive sampling, although we do not believe this influenced the study’s results. An inherent limitation of using secondary administrative data sets is the completeness of the procedural fields. Our data set was 63.7% complete for the main interventions field in NACRS (1 319 388/2 070 260); this was expected because patients admitted to hospital may have their emergency department interventions recorded in the Discharge Abstract Database instead of NACRS. In other instances, there was no intervention completed during the visit, or the intervention was not recorded. Our cohort size remained large and is trustworthy based on our study objectives.

Individual judgments may be subjective given an expert’s own evaluation of safety in selecting health care centres. This limitation was minimized in the study design to include only physicians with adequate knowledge of emergency and primary care practices in Ontario; the Delphi committee contained a high number of experts; and a detailed description of each health care centre was provided.

Our findings may not be generalizable in settings where payment structures for health care, accessibility to subacute care or ambulance availability are different. In addition, our research was specific in terms of population (i.e., adult, nonemergent and transported by ambulance), only included emergency department interventions and did not take into consideration any additional clinical details.

### Conclusion

With a continued increase in the proportion of patients with nonemergent or low-acuity conditions transported to emergency departments by paramedic services, it is important to explore features that support redirection programs so that their effect on outcomes for patients and use of emergency departments can be evaluated. Most interventions provided by physicians in the emergency department to patients with nonemergent conditions transported by paramedic services were identified as suitable for subacute health care centres (i.e., urgent care centres, walk-in medical centres and clinics led by nurse practitioners). Although focusing on interventions alone has limitations, our results suggest there may be a patient population suitable for redirection programs by paramedic services in Ontario as a way of countering the emergency department crisis.

Our findings may help to inform construction of a patient classification system for patients with nonemergent conditions for use by paramedic services that could be used to prevent visits to the emergency department and to align paramedic services with patient needs better. Future research is required to augment our findings with additional patient and hospital contextualization toward such a classification system.

## Acknowledgements

The authors thank all 20 physicians of the expert committee who participated in the RAND/UCLA modified Delphi exercise for their valuable contributions.

## Footnotes

*   **Competing interests:** None declared.

*   This article has been peer reviewed.

*   **Contributors:** Ryan Strum and Andrew Costa led the conceptualization of the study methodology. Ryan Strum designed the study, conducted the RAND/UCLA modified Delphi exercise, and drafted and revised the manuscript. Walter Tavares, Andrew Worster and Lauren Griffith contributed to the design of the study and interpretation of the data. All of the authors revised the manuscript critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.

*   **Funding:** The authors received no specific funding for this study.

*   **Data sharing:** All aggregate data herein are accessible to other interested parties by application to the corresponding author.

*   **Editor’s note:** This article appears as a preprint (medRxiv): [https://doi.org/10.1101/2021.06.01.21258191](https://doi.org/10.1101/2021.06.01.21258191).

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/10/1/E1/suppl/DC1](http://www.cmajopen.ca/content/10/1/E1/suppl/DC1).

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: [https://creativecommons.org/licenses/by-nc-nd/4.0/](https://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  Gill JM (1994) Nonurgent use of the emergency department: Appropriate or not? Ann Emerg Med 24:953–7.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0196-0644(94)70213-6&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=7978571&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1994PP22500018&link_type=ISI) 

2.  Drummond AJ (2002) No room at the inn: overcrowding in Ontario’s emergency departments. CJEM 4:91–7.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17612426&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

3.  Strum RP, Tavares W, Worster A, et al. (2021) Development of the PriCARE classification for potentially preventable emergency department visits by ambulance: a RAND/UCLA modified Delphi study protocol. BMJ Open 11:e045351.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTEvMS9lMDQ1MzUxIjtzOjQ6ImF0b20iO3M6MTk6Ii9jbWFqby8xMC8xL0UxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  Reason C (Nov 17, 2014) Toronto.com, Toronto paramedics hope to educate people on when they should call 911. Available: [https://www.toronto.com/news-story/5042205-toronto-paramedics-hope-to-educate-people-on-when-they-should-call-911/](https://www.toronto.com/news-story/5042205-toronto-paramedics-hope-to-educate-people-on-when-they-should-call-911/). accessed 2021 June 30.
    
    

5.  (2005) Improving access to emergency services: a system commitment (Ministry of Health, Ministry of Long-Term Care, Toronto) Available: [https://www.health.gov.on.ca/en/common/ministry/publications/reports/emerg\_dept\_05/emerg\_dept\_05.aspx](https://www.health.gov.on.ca/en/common/ministry/publications/reports/emerg\_dept_05/emerg_dept_05.aspx). (accessed 2021 June 30).
    
    

6.  Dejean D, Giacomini M, Welsford M, et al. (2016) Inappropriate ambulance use: a qualitative study of paramedics’ views. Healthc Policy 11:67–79.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

7.  National Ambulatory Care Reporting System (ICES, Toronto).
    
    

8.  (2016) Under pressure: emergency department performance in Ontario (Health Quality Ontario, Toronto) Available: [http://underpressure.hqontario.ca/](http://underpressure.hqontario.ca/). (accessed 2020 July 23).
    
    

9.  Krebs LD, Kirkland SW, Chetram R, et al. (2017) Low-acuity presentations to the emergency department in Canada: exploring the alternative attempts to avoid presentation. Emerg Med J 34:249–55.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzNC80LzI0OSI7czo0OiJhdG9tIjtzOjE5OiIvY21ham8vMTAvMS9FMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

10. Vertesi L (2004) Does the Canadian emergency department triage and acuity scale identify non-urgent patients who can be triaged away from the emergency department? CJEM 6:337–42.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17381991&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

11. Morgan SR, Chang AM, Alqatari M, et al. (2013) Non–emergency department (ED) interventions to reduce ED utilization: a systematic review. Acad Emerg Med 20:969–85.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24127700&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

12. Morris J, Daoust R, Cournoyer A, et al. (2018) LO65: Safety and satisfaction of a new program redirecting low-acuity emergency department patients to medical clinic: a prospective cohort study. CJEM 20(S1):S29–30.
    
    

13. Carlson LC, Raja AS, Dworkis DA, et al. (2020) Impact of urgent care openings on emergency department visits to two academic medical centers within an integrated health care system. Ann Emerg Med 75:382–91.
    
    

14. Weinick RM, Burns RM, Mehrotra A (2010) Many emergency department visits could be managed at urgent care centers and retail clinics. Health Aff (Millwood) 29:1630–6.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjk6IjI5LzkvMTYzMCI7czo0OiJhdG9tIjtzOjE5OiIvY21ham8vMTAvMS9FMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

15. Llovera I, Loscalzo K, Gao J, et al. (2019) Increased access to urgent care centers decreases low acuity diagnoses in a nearby hospital emergency department. Am J Emerg Med 37:486–8.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

16. Hutchison B, Østbye T, Barnsley J, et al. (2003) Patient satisfaction and quality of care in walk-in clinics, family practices and emergency departments: the Ontario Walk-In Clinic Study. CMAJ 168:977–83.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNjgvOC85NzciO3M6NDoiYXRvbSI7czoxOToiL2NtYWpvLzEwLzEvRTEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

17. van Weel C, Kidd MR (2018) Why strengthening primary health care is essential to achieving universal health coverage. CMAJ 190:E463–6.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTkwLzE1L0U0NjMiO3M6NDoiYXRvbSI7czoxOToiL2NtYWpvLzEwLzEvRTEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

18. Gonçalves-Bradley D, Khangura JK, Flodgren G, et al. (2018) Primary care professionals providing non-urgent care in hospital emergency departments. Cochrane Database Syst Rev 2:CD002097.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/14651858.CD002097.pub4&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=29438575&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

19. Silow-Carroll S, Alteras T, Stepnick L (2006) Patient-centered care for underserved populations: definition and best practices (Economic and Social Research Institute, Washington (D.C.)).
    
    

20. Tavares W, Drennan I, Van Diepen K, et al. (2017) Building capacity in healthcare by reexamining clinical services in paramedicine. Prehosp Emerg Care 21:652–61.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

21. Kirkland SW, Soleimani A, Rowe BH, et al. (2019) A systematic review examining the impact of redirecting low-acuity patients seeking emergency department care: Is the juice worth the squeeze? Emerg Med J 36:97–106.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo3OiIzNi8yLzk3IjtzOjQ6ImF0b20iO3M6MTk6Ii9jbWFqby8xMC8xL0UxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

22. Okoli C, Pawlowski SD (2004) The Delphi method as a research tool: an example, design considerations and applications. Inf Manage 42:15–29.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.im.2003.11.002&link_type=DOI) 

23. Boulkedid R, Abdoul H, Loustau M, et al. (2011) Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One 6:e20476.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1371/journal.pone.0020476&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21694759&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

24. 1.  Fitch K
    
    , ed (2001) The Rand/UCLA appropriateness method user’s manual (Rand, Santa Monica (CA)), p 109.
    
    

25. (2018) Canadian coding standards for version 2018 ICD-10-CA and CCI (Canadian Institute for Health Information, Ottawa).
    
    

26. Murphy MK, Black NA, Lamping DL, et al. (1998) Consensus development methods, and their use in clinical guideline development. Health Technol Assess 2(3):i–iv, 1–88.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=9561893&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

27. Bullard MJ, Unger B, Spence J, et al., CTAS National Working Group (2008) Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) adult guidelines. CJEM 10:136–51.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1017/S1481803500009854&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18371252&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

28. Sullivan GM (2011) A primer on the validity of assessment instruments. J Grad Med Educ 3:119–20.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22655129&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

29. Sancton K, Sloss L, Berkowitz J, et al. (2018) Low-acuity presentations to the emergency department. Can Fam Physician 64:e354–e360.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiY2ZwIjtzOjU6InJlc2lkIjtzOjk6IjY0LzgvZTM1NCI7czo0OiJhdG9tIjtzOjE5OiIvY21ham8vMTAvMS9FMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

30. Campbell MK, Silver RW, Hoch JS, et al. (2005) Re-utilization outcomes and costs of minor acute illness treated at family physician offices, walk-in clinics, and emergency departments. Can Fam Physician 51:82–3.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiY2ZwIjtzOjU6InJlc2lkIjtzOjc6IjUxLzEvODIiO3M6NDoiYXRvbSI7czoxOToiL2NtYWpvLzEwLzEvRTEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

31. Laskowski M, McLeod RD, Friesen MR, et al. (2009) Models of emergency departments for reducing patient waiting times. PLoS One 4:e6127.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19572015&link_type=MED&atom=%2Fcmajo%2F10%2F1%2FE1.atom) 

32. 1.  Weerasinghe D
    
    Laskowski M, Mukhi S (2009) in Electronic healthcare [lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering], Agent-based simulation of emergency departments with patient diversion, ed Weerasinghe D (Springer, Berlin (Germany)), pp 25–37.
    
    

*   © 2022 CMA Impact Inc. or its licensors