Research
Open Access

Inclusion of patient-level emergency department characteristics to classify potentially redirectable visits to subacute care: a modified Delphi consensus study

Ryan P. Strum, Walter Tavares, Andrew Worster, Lauren E. Griffith and Andrew P. Costa
January 24, 2023 11 (1) E70-E76; DOI: https://doi.org/10.9778/cmajo.20220062

Abstract

Background: Most patients transported by Ontario paramedics to the emergency department have non-emergent conditions and may be more appropriately served by subacute community-based care centres. We sought to determine consensus on a set of patient characteristics that could be useful to classify retrospective emergency department visits that had a high probability of being primary care–like and potentially redirectable to a subacute care centre by paramedics.

Methods: We conducted a modified Delphi study to assess expert consensus on characteristics of patients transported by paramedics to the emergency department from August to October 2021. An expert Delphi committee was constructed of emergency and family physicians in Ontario using purposive sampling. Experts rated whether each characteristic was useful to be included in a classification to identify potentially redirectable visits retrospectively, as well as characteristic details (e.g., upper and lower bounds). Consensus was considered 75% agreement.

Results: Sixteen experts participated in the study; the experts were mostly male (75%) and evenly divided between emergency and family medicine. After 2 rounds, consensus was achieved on 8 of 9 characteristics (89%). Four characteristics were determined as useful to classify potentially redirectable emergency department visits: age (81%), triage acuity (100%), specialist consult in the emergency department (94%) and emergency department visit outcome (81%). Specifications of each characteristic were refined as follows: young and middle-aged adults with a non-emergent triage acuity, did not receive a specialist physician consult in the emergency department and discharged from the emergency department.

Interpretation: Strong consensus was achieved to specify a classification system for potentially redirectable emergency department visits. These results will be combined with knowledge of which subacute care centres could conduct the main physician interventions to retrospectively identify emergency department visits that could have been suitable for paramedic redirection for further research.

Study registration: ID ISRCTN22901977.

Most patients transported by Ontario paramedics to the emergency department have non-emergent conditions.1 Despite efforts to improve emergency department throughput (such as instituting ambulatory care divisions in the emergency department), challenges to provide timely and high-quality care remain.2,3 Subacute community-based care centres (i.e., urgent care or specialized community services that manage chronic illnesses) may be appropriate alternatives to the emergency department when patients do not require emergency health care. These centres have been associated with increased continuity of care, shorter wait times and lower health care costs per visit.4–6 Moreover, redirection of specific patient cohorts could be an important strategy to reduce emergency department overcrowding, while providing equivalent patient-centred care.7–9 Paramedic redirection has been difficult to implement; there is a lack of validated patient classifications to identify redirection suitability in the prehospital field reliably. 7,8,10 Various epidemiological classifications describe emergency department visits that could have been potentially preventable, but their translation to paramedic practices is problematic; paramedic redirection is not incorporated, and large heterogeneity exits in objectives, inclusion criteria or clinician scope of practice.1,11–13 Lastly, identification of patient cohorts potentially suitable for emergency department redirection is challenging to determine before the visit to the emergency department, when diagnostics, services rendered and outcome of the visit are unknown.7,8,10

To inform prospective research on paramedic redirection, a retrospective epidemiological classification is needed first to identify and examine emergency department visits for which redirection could have been appropriate. The specific parameters of which clinical and nonclinical features are useful to retrospectively identify potentially redirectable patient cohorts in emergency department data are not known.14 Knowledge of useful patient characteristics to classify potentially redirectable emergency department visits could be helpful to provide a concise depiction of patients for further investigation regarding care needs, services required and redirection feasibility. Particularly, emergency department visits determined to have been potentially suitable for redirection could permit study of their prehospital clinical presentations after linkage between paramedic and emergency department data resources.15–17

Our objective was to determine consensus on a set of emergency department visit patient characteristics that could be useful to retrospectively identify visits that had a high probability of being primary care–like and could have been potentially redirectable to subacute centres by paramedics.

Methods

We used a modified Delphi study design to establish and examine consensus on which patient characteristics are useful to consider when determining primary care–like emergency department visits that are potentially redirectable by paramedics to subacute care. All patient characteristics included are routinely collected in each emergency department visit and stored in the National Ambulatory Care Reporting System (NACRS) emergency department database.18 Ontario paramedics are legislatively mandated to transport all patients to the emergency department, independent of care needs or acuity.19 However, Ontario’s Ministry of Health has amended paramedic regulation to incorporate new models of care, including transport to subacute non-emergency department alternatives.20 Combining useful emergency department patient characteristics identified in this study could identify a retrospective patient cohort of potentially redirectable emergency department visits with high internal validity and inform future research to support paramedic redirection initiatives. This study occurred between August and October 2021 in Ontario, Canada.

Participants

Emergency and family care physicians were recruited to participate from differing regions across Ontario between Aug. 1 and Aug. 11, 2021. Selection of experts was based on their participation in a parallel modified Delphi study that determined appropriateness of an emergency department physician intervention to be conducted in specified subacute care centres instead of the emergency department.14 These physicians were previously screened for their expertise and met the inclusion criteria of currently practising, practising in Ontario, and exhibiting at least 1 of (a) leadership role in paramedic practice oversight or paramedic medical director, (b) extensive medical experience (≥ 15 yr) or (c) holding an academic faculty appointment.14 Ontario medical directors are physicians who authorize paramedics to perform controlled medical acts in the prehospital field and are responsible for each paramedic’s quality of care.21 Physicians were originally recruited using purposive sampling and were balanced between emergency and primary care medicine.14 Recruited physicians were sent a study package describing this study’s objectives and methods when invited to participate. We determined a priori that the Delphi expert committee should be composed of at least 14 physicians, greater than the generally accepted minimum of 12 participants. 22,23 Consent was obtained in writing before any data collection. Physicians were provided an electronic gift card at the completion of the study for their participation.

Characteristic selection

We generated a list of all available patient characteristics in the NACRS emergency department database for inclusion consideration in the exercise. The NACRS is a hospital- and community-based ambulatory care administrative database that collects data on every patient’s emergency department visit at the time of service in Ontario.24 All Ontario emergency departments provide administrative reports to the NACRS quarterly, making it a population database with minimal incompleteness.25 All potential characteristics were initially screened by 2 authors (R.P.S. and A.W.) and excluded if they had greater than 50% missingness in the NACRS database or did not have clinical relevancy to the study’s objective. Overall, we selected 9 characteristics for inclusion in the modified Delphi rounds based on scientific literature, inclusion as a variable in other emergency department patient classification systems, clinical judgment and availability of data in the NACRS.11–13,18,26 The included characteristics were age, sex, triage acuity, main diagnostic category, comorbidities, specialist consult performed in the emergency department, outcome of emergency department visit, time from triage to emergency department outcome and return to emergency department within 30 days or less.

Patient age parameters were collapsed into 5-year ordinal levels, after age 20 years. Triage acuity is categorized by the Canadian Triage and Acuity Scale (CTAS), an ordinal scale that ranges from 1 (most emergent; resuscitation) to 5 (least emergent; nonurgent).27 The main diagnostic category was assigned by the emergency department physician and recorded using the International Statistical Classification of Diseases and Related Health Problems, 10th Revision (ICD-10).28 Comorbidities were defined as pre-existing diagnoses at time of emergency department visit and included hypertension, diabetes, chronic obstructive pulmonary disease, asthma, rheumatoid arthritis, congestive heart failure, bowel disease and cancer. Only these 8 comorbidities could be included in the study, as only these are readily collected in the administrative data of the NACRS.18

Delphi process

The modified Delphi method is a consensus strategy to systematically analyze the judgments of experts in a specified field.29 Modified Delphi studies provide practical applications of questionnaires to engage experts individually while yielding results as aggregate consensus that is greater than any expert individually.23 Iterations (or rounds) often involve inviting input, tabulating consensus and re-presenting items that reached consensus and remain discrepant for additional consideration. In each round, experts were posed questions related to the usefulness of patient characteristics to classify emergency department visits that are primary care–like and potentially redirectable to subacute care centres by paramedics. When experts rated a patient characteristic as useful, each was asked to rate which parameters of the characteristic would specify the emergency department with a high specificity to the study’s objective. Standardized definitions of each patient characteristic were provided to minimize unintended or heterogeneous interpretations. Consensus was evaluated through 2 rounds of expert ratings, as further rounds are unlikely to result in differing ratings when items of the subsequent rounds are minimal and could not be modified.30 The structure of the questions posed to experts is shown in Box 1, and the content of plausible questionnaire responses is shown in Appendix 1, available at www.cmajopen.ca/content/11/1/E70/suppl/DC1. Participants selected their parameters from a list, shown in Appendix 1; no free text was accepted.

Box 1:

Questions posed to expert physicians in the modified Delphi questionnaire consensus exercise to evaluate suitability of patient characteristics that indicate an emergency department visit was potentially redirectable to subacute centres by paramedics

1. Determining consensus on patient characteristics

Do you think (patient characteristic) is a useful characteristic to consider when determining if an emergency department visit is both primary care–like and potentially redirectable to subacute care centres by paramedics?

2. Determining specific parameters of a useful patient characteristic*

If so, what specific parameters of (patient characteristic) suggest a visit is primary care–like and potentially redirectable to subacute care centres by paramedics?

  • ↵* Parameter specification posed only to experts who answered “Yes” to the consensus question.

The modified Delphi rounds occurred between Aug. 11 and Sept. 21, 2021. Round 1 was distributed to all experts for their individual ratings. After completion of Round 1, data were extracted to analyze consensus of each characteristic and parameter. A general feedback form was distributed to all experts reporting the aggregated results of Round 1 to aid their considerations for characteristic ratings in Round 2 (i.e., reported percent agreement).29 Round 2 was constructed with only the characteristics that did not receive consensus in Round 1, and distributed for a second round of ratings. We determined that a priori ratings of Round 2 would serve as the final consensus results, as further rounds are unlikely to change consensus and participation rates diminish when items on subsequent rounds are minimal. We used CheckMarket, an electronic survey software, to collect all data. All data were stored with the investigators via CheckMarket’s secure and encrypted program.

Data analysis

We determined a priori that a characteristic must receive 75% agreement or greater to achieve consensus.31 Each patient characteristic was considered independent from one another during ratings. All parameter specifications used in this study were taken directly from the NACRS; we did not categorize the parameters of each characteristic. Demographic statistics of the expert Delphi committee were reported using frequency and proportion.

Ethics approval

This study received a research ethics board exemption waiver from the Hamilton Integrated Research Ethics Board; review reference 2020-11451-GRA. All participants provided written consent before study initiation.

Results

Sixteen physicians agreed to participate in this study. Three declined owing to current workload, and 1 did not respond. The expert Delphi committee was mostly male (75%) and acknowledged their primary practising field as emergency medicine (81%). Medical training of the committee was split evenly between family and emergency medicine. Length of practice was spread evenly throughout the Delphi committee with a range from less than 5 years to 30 years or more; the largest groups had 5 to 9 years (25%) and 3 or more years (25%) in practice. A total of 31% of the expert committee were medical directors responsible for medical oversight of paramedic practices in Ontario. Characteristics of the committee are shown in Table 1.

View this table:
Table 1:

Demographic characteristics of the expert committee in the modified Delphi consensus exercise

Table 2 shows the results of the consensus modified Delphi exercise. Overall, 4 patient characteristics achieved consensus in the Delphi exercise. In Round 1 of the modified Delphi exercise, 7 of 9 patient characteristics achieved consensus. The 2 characteristics that did not receive consensus were reposed in Round 2, resulting in 1 characteristic reaching consensus and 1 characteristic not. All 16 experts completed the Round 1 questionnaire, and 15 completed Round 2. The patient characteristics identified as useful characteristics to consider in a classification were age (81%), triage acuity (100%), specialist consult performed in the emergency department (94%) and outcome of the emergency department visit (81%). Patient characteristics determined not useful to classify emergency department visits were sex (100%), comorbidities (75%), time parameter from triage to emergency department outcome (88%) and return to emergency department within 30 days or less (80%). The characteristic “main diagnostic category” did not receive consensus after 2 rounds. Overall, the disagreement on characteristics were spread evenly among the physicians trained in emergency and family medicine.

View this table:
Table 2:

Results of a 2-round modified Delphi exercise to establish which patient characteristics are important to consider when classifying whether an emergency department visit was primary care–like and potentially redirectable to subacute care centres by paramedics

Parameters of useful characteristics that achieved consensus were refined as follows: young and middle-aged adults with a non-emergent triage acuity, did not receive a specialist physician consult in the emergency department and discharged from the emergency department. When experts rated patient characteristics useful to classify emergency department visits that are primary care–like and potentially redirectable by paramedics, each supplied a parameter specification to constrain a characteristics range based on their expert judgment. Of the 13 experts who rated age as a useful patient characteristic, all rated the lowest age provided (18 yr) appropriate for the lower boundary, and the upper boundary ranged from 50 years to “no upper limit.” The largest selection for the upper age was 70 years. Triage acuity was rated by all as useful, with all rating CTAS level 5 as the lowest acuity for the lower boundary. All experts selected an upper acuity boundary as CTAS level 4 (100%), with CTAS level 3 rated as the upper boundary from a smaller proportion (37.5%). Nearly all found specialist consult in the emergency department a useful characteristic to include, with all rating that only emergency department visits that did not receive an emergency department specialist physician consult as useful to consider for the classification. Lastly, most of the experts who rated the outcome of the emergency department visit as useful selected discharged from emergency department as useful for this classification, with the other options not suitable (admitted to hospital, transfer to another acute care facility directly from emergency department and left after triage with no medical assessment).

Interpretation

Strong consensus was found by a committee of emergency and primary care physicians for a set of patient characteristics that may be useful to determine emergency department visits that were primary care–like and potentially redirectable by paramedics to subacute care centres. Patient age, triage acuity, specialty consult in the emergency department and outcome of the emergency department visit are useful characteristics to specify inclusion criteria in an epidemiologic classification system.

Our results were fairly consistent with similar published classifications that propose to identify preventable emergency department visits retrospectively. Our lowest triage acuity parameter was consistent with similar classifications, though triage acuity was not included as a parameter in many classifications. 12 Age was incorporated into only a small number of similar classifications, but when included had an upper boundary of 75 years or younger.11,32 Classifications that incorporated the outcome of an emergency department visit included only patients who were discharged from the emergency department, and must not have been hospitalized, been admitted or died in emergency department, a finding consistent with our study.12 Some classifications included the main diagnostic or presenting complaint as an identifier, though our study results could not achieve consensus on whether this is an important determining factor.11,13,33,34 Some classifications cited only emergency department visits arriving by self-referral or walk-in as eligible, which is incompatible with the objective of our study to construct a classification specifically to examine patients transported by paramedics.35 Given the abundant exclusion of paramedic-transported patients from published classifications, our research contributes a focus on an underintegrated patient cohort of potentially preventable emergency department classifications.

Most useful patient characteristics had large agreement among the physician committee on parameter specification. The upper age limit differed among experts, showing their hesitancy to include the geriatric cohort in plausibly redirecting patients from the emergency department. High agreement was recognized in the lower age limit (18 yr), indicating experts were content with redirecting patients who likely will not have as numerous or complex conditions as older age groups. Specifying triage acuity was largely consistent by including the most non-emergent acuities (CTAS levels 4 and 5), with relatively a third of experts indicating urgent (CTAS level 3) acuities could be appropriate. The low number of experts rating urgently triaged patients as suitable for redirection to subacute care may exemplify a conservative approach to withhold making clinical judgments on emergency department visits that potentially could require acute care, although most urgently triaged emergency department visits do not.36 High agreement was also observed in specifying discharge as the emergency department outcome, and not having received a specialist consult during the visit. These specifications acknowledge that patients should be stable to be discharged home or to their place of residence, and the condition treated should not exceed the scope of practice of an attending emergency physician. The main diagnostic category did not receive consensus, an important finding that implies some experts desired to understand the condition of the emergency department visit before making a judgment on redirection suitability. A plausible explanation for this result is that the diagnostic categories of ICD-10 are too broad to make generalizations on appropriateness as determined by the physicians. Given that broad diagnostic categories do not predict acuity or severity of illness, the indecision of its utility to incorporate into a classification remains in question. Though knowledge of diagnostic categories was important to some experts, comprehension of the main intervention received during the emergency department visit ought to contribute more beneficial evidence to make a judgment on redirection.

While this study contributes to evolving conceptual frameworks intended to comprehensively categorize patients potentially suitable for redirection, the absence of a validated patient classification remains. For instance, patient characteristics alone may not be sufficient classifying features to make determinations on which patients could have been potentially redirected. Inclusion of the physician intervention is an important element missing from our study that is core to developing a redirection patient classification, though this is minimally included in published classifications.11,12,14,34 Further understanding and inclusion of a subacute centre’s capacity to provide medical care that is equivalent to the care provided in the emergency department may also be needed, and should be incorporated into any patient classification aiming to identify redirectable emergency department visits retrospectively.

The results of this study will support the construction of an epidemiologic patient classification to retrospectively identify paramedic-transported emergency department visits that may have been suitable for redirection in emergency department databases for further study. This patient classification will be constructed using the results of this study and knowledge of which subacute care centres could conduct a specified primary care intervention.1,14 Data linkage of paramedic medical reports to emergency department data resources for emergency department visits that meet this classification’s inclusion criterion will identify which patients to examine before hospital arrival, an important component to inform prospective redirection research and direct clinical guideline development. 1 Our results inform our epidemiologic understanding of which patients could have been potentially suitable for paramedic redirection, and cannot be readily incorporated into clinical practice before thorough validation is undertaken. Future research is required to experimentally validate our results and determine the generalizability of our study in differing emergency department data sets.

Limitations

The expert Delphi committee was composed mostly of emergency physicians, a limitation of purposive sampling. Recruitment was balanced between emergency and family medicine physicians, but enrolment rates left a slight imbalance, with a higher proportion of physicians practising in an emergency department. However, emergency physicians were well equipped to make determinations in this study, and we do not anticipate this affecting the results when the committee’s medical training was split evenly between emergency and family medicine. Individual judgments may be subjective based on an expert’s own clinical experiences, formal training or approach to patient care, though this limitation was minimized by providing detailed definitions of each patient characteristic and using a robust Delphi methodology. Only patient characteristics that were available in the NACRS had the potential to be included in this study. Using emergency department visit characteristics retrospectively limits the translation of our results to direct paramedic practices; however, translation of these findings into paramedic reports would make a useful application for prehospital translation.

Conclusion

Patient characteristics were identified to assist classifying emergency department visits that may have been primary care–like and potentially redirectable by paramedics to subacute care centres. Though patient features alone cannot make determinations on patient suitability for a redirection classification, these variables contribute to ongoing efforts to identifying eligibility criteria of patients encountered by paramedics for further epidemiologic study. Combining the results of this research along with knowledge of where the main emergency department intervention could be conducted may be helpful to analyze when patients could potentially be redirected from the emergency department, and where and to which providers they could be transported for care.

Acknowledgement

The authors thank the 16 physicians of the expert committee who participated in the modified Delphi exercise for their valuable contributions.

Footnotes

  • Competing interests: Ryan Strum declares that McMaster University paid the publication fee for this article and supported the writing of the manuscript through a PhD student stipend. He also reports that the data cohort creation was prepared by an independent ICES analyst for the study authors. Lauren Griffith’s salary is supported by the McLaughlin Foundation Professorship in Population and Public Health. No other competing interests were declared.

  • This article has been peer reviewed.

  • Contributors: Ryan Strum and Andrew Costa led the conceptualization of the study methodology. Ryan Strum conducted the modified Delphi exercise, analyzed and interpreted results, and drafted and revised the manuscript. All authors made contributions to the design of the study, methods, interpretation and manuscript. All authors gave final approval of the version to be published and agreed to be accountable for all aspects of the work.

  • Funding: This work was substantially supported by Schlegel Chair in Clinical Epidemiology and Aging at McMaster University.

  • Data sharing: All aggregate data herein are accessible to other interested parties by application to the corresponding author.

  • Supplemental information: For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/11/1/E70/suppl/DC1.

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/

References

  1. ↵
    1. Strum RP,
    2. Tavares W,
    3. Worster A,
    4. et al.
    (2021) Development of the PriCARE classification for potentially preventable emergency department visits by ambulance: a RAND/UCLA modified Delphi study protocol. BMJ Open 11:e045351.
    OpenUrlAbstract/FREE Full Text
  2. ↵
    1. Copeland J,
    2. Gray A
    (2015) A daytime fast track improves throughput in a single physician coverage emergency department. CJEM 17:648–55.
    OpenUrl
  3. ↵
    1. Oliver D
    (2019) David Oliver: Do public campaigns relieve pressure on emergency departments? BMJ 367:l6542.
    OpenUrlFREE Full Text
  4. ↵
    (2012) Specialized community-based care: an evidence-based analysis (Health Quality Ontario, Toronto) Available: https://www.hqontario.ca/Portals/0/Documents/evidence/reports/full-report-specialized-care.pdf, 12, 20, pp 1–60. (accessed 2022 June 25). Ontario Health Technology Assessment Series.
    OpenUrl
    1. Mehta N,
    2. Kpelitse KA,
    3. Devlin RA,
    4. et al.
    (2017) Working Paper no 170005, Primary care access and emergency department utilization: theory and evidence from Canada (Canadian Centre for Health Economics, Toronto).
  6. ↵
    1. Ly O,
    2. Price D,
    3. Saskin R,
    4. et al.
    (2021) Low-acuity emergency department use among patients in different primary care models in Hamilton and Ontario. Healthc Manage Forum 34:234–9.
    OpenUrl
  7. ↵
    1. Morris J,
    2. Daoust R,
    3. Cournoyer A,
    4. et al.
    (2018) LO65: safety and satisfaction of a new program redirecting low-acuity emergency department patients to medical clinic: a prospective cohort study. CJEM 20(Suppl 1):S29–30.
    OpenUrl
  8. ↵
    1. Kirkland SW,
    2. Soleimani A,
    3. Rowe BH,
    4. et al.
    (2019) A systematic review examining the impact of redirecting low-acuity patients seeking emergency department care: Is the juice worth the squeeze? Emerg Med J 36:97–106.
    OpenUrlAbstract/FREE Full Text
  9. ↵
    1. Affleck A,
    2. Parks P,
    3. Drummond A,
    4. et al.
    (2013) Emergency department overcrowding and access block. CJEM 15:359–84.
    OpenUrlPubMed
  10. ↵
    1. Clesham K,
    2. Mason S,
    3. Gray J,
    4. et al.
    (2008) Can emergency medical service staff predict the disposition of patients they are transporting? Emerg Med J 25:691–4.
    OpenUrlAbstract/FREE Full Text
  11. ↵
    1. Purdy S,
    2. Griffin T,
    3. Salisbury C,
    4. et al.
    (2009) Ambulatory care sensitive conditions: terminology and disease coding need to be more specific to aid policy makers and clinicians. Public Health 123:169–73.
    OpenUrlCrossRefPubMed
  12. ↵
    1. Nagree Y,
    2. Camarda VJ,
    3. Fatovich DM,
    4. et al.
    (2013) Quantifying the proportion of general practice and low-acuity patients in the emergency department. Med J Aust 198:612–5.
    OpenUrlCrossRefPubMed
  13. ↵
    1. Gruneir A,
    2. Bell CM,
    3. Bronskill SE,
    4. et al.
    (2010) Frequency and pattern of emergency department visits by long-term care residents: a population-based study. J Am Geriatr Soc 58:510–7.
    OpenUrlCrossRefPubMed
  14. ↵
    1. Strum RP,
    2. Tavares W,
    3. Worster A,
    4. et al.
    (2022) Emergency department interventions that could be conducted in subacute care settings for patients with non-emergent conditions transported by paramedics: a modified Delphi study. CMAJ Open 10:E1–7.
    OpenUrlAbstract/FREE Full Text
  15. ↵
    1. Crow R,
    2. Spicer J
    (1995) Categorisation of the patient’s medical condition: an analysis of nursing judgement. Int J Nurs Stud 32:413–22.
    OpenUrlPubMed
    1. Li C-Y,
    2. Karmarkar A,
    3. Kuo Y-F,
    4. et al.
    (2020) A comparison of three methods in categorizing functional status to predict hospital readmission across post-acute care. PLoS One 15:e0232017.
    OpenUrl
  17. ↵
    1. Woodbury ML,
    2. Velozo CA,
    3. Richards LG,
    4. et al.
    (2013) Rasch analysis staging methodology to classify upper extremity movement impairment after stroke. Arch Phys Med Rehabil 94:1527–33.
    OpenUrlCrossRefPubMed
  18. ↵
    National Ambulatory Care Reporting System metadata (NACRS) (Canadian Institute for Health Information, Ottawa) Available: https://www.cihi.ca/en/national-ambulatory-care-reporting-system-metadata-nacrs. (accessed 2020 Nov. 14).
  19. ↵
    Ambulance Act, R.S.O. 1990, c. A.19, Available: https://www.ontario.ca/laws/statute/90a19. (accessed 2022 June 25).
  20. ↵
    (2019) Ontario’s Regulatory Registry, Enabling new models of care for select 9-1-1 patients. Available: https://www.ontariocanada.com/registry/view.do?postingId=30268&language=en. (accessed 2020 July 23).
  21. ↵
    (1999) Delegation of controlled acts (College of Physicians and Surgeons of Ontario, Toronto) updated March 2021. Available: https://www.cpso.on.ca/Physicians/Policies-Guidance/Policies/Delegation-of-Controlled-Acts. (accessed 2022 June 26).
  22. ↵
    1. Vogel C,
    2. Zwolinsky S,
    3. Griffiths C,
    4. et al.
    (2019) A Delphi study to build consensus on the definition and use of big data in obesity research. Int J Obes (Lond) 43:2573–86.
    OpenUrl
  23. ↵
    1. Murphy MK,
    2. Black NA,
    3. Lamping DL,
    4. et al.
    (1998) Consensus development methods, and their use in clinical guideline development. Health Technol Assess 2:i–iv, 1–88.
    OpenUrlPubMed
  24. ↵
    1. Li G,
    2. Lau JT,
    3. McCarthy ML,
    4. et al.
    (2007) Emergency department utilization in the United States and Ontario, Canada. Acad Emerg Med 14:582–4.
    OpenUrlCrossRefPubMed
  25. ↵
    (2012) Data quality documentation, National Ambulatory Care Reporting System: multi-year information (Canadian Institute for Health Information, Ottawa) Available: https://www.cihi.ca/sites/default/files/nacrs_multi-year_info_en_0.pdf. (accessed 2022 June 14).
  26. ↵
    (2022) Data availability for CIHI data holdings (as of June 30, 2022) (Canadian Institute for Health Information (CIHI), Ottawa) Available: https://www.cihi.ca/sites/default/files/document/data-availability-and-jurisdictional-coverage-en.xlsx. (accessed 2022 Feb. 28).
  27. ↵
    1. Bullard MJ,
    2. Unger B,
    3. Spence J,
    4. et al.,
    5. CTAS National Working Group
    (2008) Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) adult guidelines. CJEM 10:136–51.
    OpenUrlCrossRefPubMed
  28. ↵
    (2018) Canadian coding standards for version 2018 ICD-10-CA and CCI (Canadian Institute for Health Information, Ottawa).
  29. ↵
    1. Fitch K,
    2. Bernstein SJ,
    3. Aguilar MD,
    4. et al.
    (2001) The Rand/UCLA appropriateness method user’s manual (RAND, Santa Monica (CA)), pp 1–109.
  30. ↵
    1. Niederberger M,
    2. Spranger J
    (2020) Delphi technique in health sciences: a map. Front Public Health 8:457.
    OpenUrlPubMed
  31. ↵
    1. Diamond IR,
    2. Grant RC,
    3. Feldman BM,
    4. et al.
    (2014) Defining consensus: a systematic review recommends methodologic criteria for reporting of Delphi studies. J Clin Epidemiol 67:401–9.
    OpenUrlCrossRefPubMed
  32. ↵
    (Sept 12, 2013) (2017) Hospitalizations for ambulatory care sensitive conditions (ACSC) (Ministry of Health and Long-Term Care, Toronto) Available: https://www.health.gov.on.ca/en/pro/programs/ris/docs/hosp_for_ambulatory_care_en.pdf, pp 1–4. (accessed 2021 Nov. 16).
  33. ↵
    (2011) Family practice sensitive conditions (Alberta Health Services, Edmonton) Available: https://www.albertahealthservices.ca/Publications/ahs-pub-pr-def-FPSC.pdf, pp 1–8. (accessed 2021 Dec. 20).
  34. ↵
    1. van der Pol M,
    2. Olajide D,
    3. Dusheiko M,
    4. et al.
    (2019) The impact of quality and accessibility of primary care on emergency admissions for a range of chronic ambulatory care sensitive conditions (ACSCs) in Scotland: longitudinal analysis. BMC Fam Pract 20:32.
    OpenUrlPubMed
  35. ↵
    1. Nagree Y,
    2. Gosbell A,
    3. McCarthy S,
    4. et al.
    (2013) Determining the true burden of general practice patients in the emergency department: getting closer. Emerg Med Australas 25:487–90.
    OpenUrlCrossRefPubMed
  36. ↵
    1. Strum RP,
    2. Mowbray FI,
    3. Worster A,
    4. et al.
    (2021) Examining the association between paramedic transport to the emergency department and hospital admission: a population-based cohort study. BMC Emerg Med 21:117.
    OpenUrl
Back to top

In this issue

CMAJ Open: 11 (1)
Vol. 11, Issue 1
1 Jan 2023

Article tools

Respond to this article
Print
Download PDF
Article Alerts
Email Article
Citation Tools
Inclusion of patient-level emergency department characteristics to classify potentially redirectable visits to subacute care: a modified Delphi consensus study
Ryan P. Strum, Walter Tavares, Andrew Worster, Lauren E. Griffith, Andrew P. Costa
Jan 2023, 11 (1) E70-E76; DOI: 10.9778/cmajo.20220062
Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo

Related Articles

Cited By...

  • No citing articles found.

Similar Articles