Article Text
PDF
Abstract
Objective To examine the process of implementing an electronic consultation (eConsult) service and evaluate its impact along key metrics outlined by the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework.
Design Cross-sectional study.
Setting Clinics using eConsult in four provinces across Canada: Alberta, Manitoba, Quebec and Newfoundland and Labrador.
Participants All eConsult cases submitted in four participating provinces were included.
Intervention The eConsult service is a secure online application that allows primary care providers and specialists to communicate regarding a patient’s care. We measured the impact using system utilisation data and mandatory close-out surveys completed at the end of each eConsult.
Main outcome measures Implementation progress and impact were examined using the five categories outlined by the RE-AIM framework: reach, effectiveness, adoption, implementation and maintenance.
Results Four provinces provided data from different periods, ranging from 4 years (Alberta) to 10 months (Manitoba). Total cases completed ranged from 96 (Manitoba) to 6885 (Alberta). Newfoundland had the largest menu of available specialties (n=35), while Alberta and Quebec had the smallest (n=22). The most frequently requested groups varied across provinces, with only endocrinology appearing in the top five for all provinces. The average specialist response time ranged from 3 days (Manitoba) to 16.7 days (Alberta). Between 54% (Newfoundland) and 66% (Manitoba) of cases resulted in new or additional information. Primary care providers avoided completing referrals they had originally considered in 36% (Newfoundland) to 53% of cases (Manitoba), while only between 27 % (Quebec) and 29% (Newfoundland) of cases resulted in a referral. In every province, services demonstrated higher rates of usage in their last quarter of data than their first.
Conclusions eConsult was successfully implemented in four new provinces across Canada. Implementation strategies and scope varied, but services demonstrated substantial consistency on several key metrics, most notably on whether new information was learnt and impact on decision to refer.
- primary care
- eConsult
- referral
- access to care
- wait times.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Study data span four provinces and multiple regions, allowing for a robust examination of eConsult’s generalisability and scalability.
The data included were observational and clinician based, which do not allow for a direct patient perspective.
Differences in service structure and data collection meant some metrics could not be captured across all participating services.
Introduction
Excessive wait times for specialist care are a serious issue across Canada.1 2 In the 2016 Commonwealth Fund Survey, Canada placed last on the measure of specialist access among the 11 countries surveyed, with 56% of Canadians reporting wait times of more than 4 weeks for a specialist appointment versus an average of 36% across all participating countries.1 Studies assessing specialist wait times across Canada have reported median wait times ranging from 5 to 11 weeks,3–6 with median wait for some high-demand specialties (eg, infectious diseases) reaching up to 24 weeks.6 Poor access to specialist advice has serious consequences, reducing patients’ ability to carry out day-to-day activities, increasing anxiety and potentially causing an overall deterioration in health.2 On a health service level, long wait times result in delayed diagnoses, duplicated testing and dissatisfaction among healthcare providers—factors that increase costs while reducing quality of care.7
In an effort to address this issue, the Champlain Building Access to Specialists through eConsultation (BASE) eConsult service was launched in 2010. The BASE model of care is a method of care delivery designed to improve access to specialist advice by allowing primary care providers (PCP) to send questions concerning patient care to specialists. It is not a specific technology and can be adopted on any digital platform capable of facilitating secure communication between PCPs and specialists. In the BASE model of care, PCPs seeking specialist advice on a patient’s care log onto their platform and select a specialty group (as opposed to an individual specialist). A case assigner allocates the eConsult to an appropriate specialist based on availability. The specialist responds to the PCP’s question within 1 week by providing advice on how to manage the patient, recommending the patient receive a face-to-face referral (not necessarily with them) or requesting additional information. PCPs can ask additional questions. Specialties are added to the service based on PCP requests, and the service undergoes continual evaluation to ensure quality and seek user feedback.
Launched as a small proof of concept and soon expanded to a full pilot in the Champlain health region of Eastern Ontario, Canada, the eConsult service has completed over 50 000 eConsults, enrolled more than 1400 PCPs and provides access to 114 specialty groups. Specialists respond to cases in a median of 1.9 days, and over two-thirds of cases are resolved without the patient requiring a face-to-face specialist visit.8 Given its success on a regional level, the eConsult team engaged in efforts to expand the service to new jurisdictions. However, Canada faces a number of barriers to successful scale-up, resulting in many projects being unable to expand beyond their pilot phase.9 10 To avoid these pitfalls, we successfully sought grants from the Canadian Institutes of Health Research. Through this process, the eConsult team formed key partnerships with provincial and national organisations to support its expansion to new jurisdictions within Ontario11 and across provinces. As part of this initiative, multiple eConsult teams based in their own jurisdictions worked closely with the Canadian Foundation for Healthcare Improvement, which supported eConsult’s spread and scale through a two-phase Connected Medicine Collaborative (figure 1).
Timeline of the Canadian Foundation for Healthcare Improvement’s two-phase Connected Medicine Collaborative.
Phase I of the Collaborative recruited 10 teams from across Canada and internationally to participate in a 9-month Access to Specialist eConsult Collaborative, in which teams aimed to develop business cases and strategies to implement one of two remote consult services in their jurisdictions: the Champlain BASE eConsult service and the telephone-based Rapid Access to Consultative Expertise service.12 The Canadian Foundation for Healthcare Improvement (CFHI) supported teams by facilitating sharing of information and hosting multiple online and face-to-face touchpoints with the innovators and new implementation sites. Phase II was built on the previous initiative to launch a 15-month quality improvement Collaborative, with 11 teams in 7 provinces participating.13 As a result of these initiatives, eConsult services have been launched in Alberta, Manitoba, Quebec, Newfoundland and Labrador and New Brunswick. Throughout this process, the teams have encountered various challenges and learnt a number of important lessons, which will be relevant to those seeking to spread, scale and sustain healthcare innovations.
In this study, we examine the process of implementing eConsult in four Canadian provinces and evaluate their impact along key metrics outlined by the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework.
Methods
Design
This study involves a cross-sectional analysis of data from eConsult services implemented in four provinces across Canada.
Setting
To evaluate the impact of eConsult’s replication, this study draws data from eConsult services in Alberta (AB), Manitoba (MB), Quebec (QC) and Newfoundland and Labrador (NL). Services in three of these provinces (MB, QC, NL) operate using a platform identical to the BASE model of care, which was first developed in Ottawa, Ontario in 2009. The remaining province (AB) incorporated an eReferral service with eConsult capabilities similar principles to the BASE model into its provincial electronic health record, called Alberta Netcare, in 2014. While an eConsult service has been implemented in New Brunswick using the BASE model, the service had only minimal data at the time of this study and was thus excluded.
The Canadian healthcare system is publicly funded and provides universal access to a host of clinical services, including primary care, specialty care and emergency medicine. Other elements of healthcare, such as pharmaceuticals and allied health services, are not universally funded. While the federal government provides funding, each province and territory is responsible for overseeing the administration of healthcare in its jurisdiction, with the exception of some specialty populations where care is managed federally (eg, First Nations communities, members of the military and inmates of federal penitentiaries). As such, the exact healthcare context varies slightly between the provinces participating in this study.
RE-AIM framework
As part of our Canadian Institutes of Health Research-funded activities to explore the factors critical to eConsult’s successful adoption, we used the RE-AIM framework to assess eConsult’s impact based on five criteria: (1) reach into the target population; (2) effectiveness or efficacy; (3) adoption by target settings, institutions and staff; (4) implementation, including its consistency and costs of delivery and (5) maintenance of intervention effects in individuals and settings over time.14 A description of each criterion and its associated metrics is included in the supplementary appendix 1.
Supplementary file 1
Participants
This study includes aggregated information from eConsult cases completed in the four participating provincial services.
Analysis
Each participating province contributed data collected by the service. This includes the use of data collected automatically (eg, specialty group submitted to and response time) and responses to surveys completed at the conclusion of each case. The AB service does not include a closeout survey, so was exempted from metrics that relied on survey data. Data reporting periods varied between provinces out of necessity, as provinces implemented eConsult at different points in time. Where possible, at least 1 year of data were included. A complete list of data metrics available from each service is presented in table 1.
Data available for analysis from each of the four participating services
Patient and public involvement
Patient representatives are an important part of eConsult’s decision-making team, and inclusion of patient voices has been a cornerstone of the service’s process of evaluation and dissemination of knowledge.15 However, this study did not draw on direct patient data but instead relied on the aggregate use of data and survey responses from PCPs. Patient involvement in this particular study was therefore limited.
Results
Reach
The reporting period for the study varied between provinces. MB had the shortest reporting period, with data from December 2017 to September 2018, while AB had the longest, with data from August 2014 to July 2018. The total number of cases completed ranged from 96 cases (MB) to 6885 cases (AB). When expressed as population rates in the first year of implementation, PCPs completed 1.14 eConsults per 1000 people in NL, 0.04 eConsults per 1000 people in Quebec, 0.07 eConsults per 1000 people in Manitoba and 0.0035 eConsults per 1000 people in Alberta. Data on the number of cases completed, population rates and providers enrolled is listed in table 2.
The data reporting period, number of cases completed and specialties available for eConsult services across provinces
Services offered a range of specialties, with NL offering the greatest variety (n=35), and AB and QC the least (n=22). The most frequently requested groups varied across provinces. Only endocrinology appeared in the top-five specialties across all provinces (table 1). The top 10 most frequently referred to specialties across services are displayed in figure 2.
The top-10 most frequently requested specialties across services. AB, Alberta; MB, Manitoba; NL, Newfoundland and Labrador; QC, Quebec.
Effectiveness
The average specialist response time ranged from 3 days (MB) to 16.7 days (AB). Between 54% (NL) and 66% (MB) of cases resulted in the PCP getting new or additional information to use in their patient’s treatment (figure 3). PCPs avoided completing referrals they had originally considered in 36% (NL) to 53% of cases (MB), while only between 24% (MB) and 29% (NL) of cases resulted in patients still requiring a referral (figure 4). Furthermore, in a small number of cases (QC, NL: 3%; MB: 1%), PCPs had not originally considered a referral but implemented one based on specialist advice.
Which of the following best describes the outcome of this eConsult for your patient? NL, Newfoundland and Labrador; QC, Quebec.
Impact of eConsult on referral based on PCP response to closeout survey. MB, Manitoba; NL, Newfoundland and Labrador; QC, Quebec.
Across all services where PCP survey data were collected, PCPs expressed high levels of satisfaction with the eConsult service. PCPs ranked specialists’ responses as high or very high value for their patients in 96% (QC), 95% (NL) and 96% (MB) of cases (figure 5).
Primary care provider-reported value of eConsult for their patients on a 5-point Likert scale. MB, Manitoba; NL, Newfoundland and Labrador; QC, Quebec .
Adoption
Services from each province varied in scope. As the most recently implemented service (launched in December 2017), the MB service was also the smallest, with 93 PCPs enrolled across 18 clinics in 9 municipalities and 31 practising specialists. The QC service enrolled 139 PCPs, who practised in 11 clinics across three different regions (Outaouais, Abitibi and Mauricie), as well as 55 specialists practising in 3 specialty clinics. The NL service demonstrated the broadest scope of adoption among services using the BASE model, with 252 enrolled PCPs practicing in 33 municipalities across the province and 56 participating specialists. The AB service was the largest overall, with 1446 participating PCPs across 46 municipalities. PCP enrolment over time is presented in figure 6.
Primary care provider enrolment in eConsult services by month.
Of the 93 PCPs enrolled in the MB service, 32% (n=30) were active users, meaning they had submitted at least one case. QC demonstrated a similar rate of active users, with 33% (n=44) of the 135 enrolled PCPs submitting at least one case. NL showed a higher ratio of active users, with 56% (n=140) of enrolled PCPs deemed active.
Implementation
The implementation strategy for each province varied based on the needs of its population and the infrastructure already in place. Each service partnered with regional and provincial groups capable of hosting the service and expanding it to new jurisdictions (table 3). In MB and NL, the BASE platform was replicated directly and provided as part of CIHR funded grants, following the template laid in the Champlain region and using the same software (Microsoft SharePoint 2010). By contrast, QC adopted the BASE model of managed care but built the service on the Enterprise Telehealth Platform already in operation on the Quebec Healthcare Network to leverage existing infrastructure. AB incorporated eConsult into Alberta Netcare, the electronic health record responsible for storing patients’ health information province wide.
Details on the platform, host organisation and payment models for participating eConsult services
Methods for payment vary between provinces. In AB, providers are compensated by submitting fee codes in the same manner as other services, with separate codes and rates established for referring ($32.43) and responding ($76.27) providers. In the other provinces, fee codes are either not available (QC, NL) or too low to adequately support the service (MB). Payment is thus provided through the service, with specialists earning $200/hour prorated to the length of time spent answering the case.
The provincial services have engaged in various activities supporting eConsult’s implementation in their jurisdictions. AB Netcare eReferral engaged in a number of promotional activities aimed at physicians and clinical support staff, including presentation at local and provincial conferences, publication through regional authorities (eg, AB Medical Association, AB College of Physicians and Surgeons) and service-affiliated websites (eg, AHS, AB Netcare eReferral and Calgary Zone Specialist LINK), in-person training and webinars. The AB team worked collaboratively with the primary care networks and various specialty groups in the province to engage physicians to facilitate adoption of eReferral. In MB, activities included presentations (eg, Manitoba College of Family Physicians Annual Scientific Assembly, the University of Manitoba Academic Days), publications (eg, Manitoba College of Family Physician’s MCFP E_News Update, Nine Circles Community Health Centers Annual Report) and outreach to local PCPs and specialists. In QC, activities included outreach to provincial and national organisations (eg, Quebec College of Family Physicians, Canada Health Infoway), presentation at conferences (eg, Centre intégré de santé et des services sociaux de l’Outaouais (CISSSCO) Research Day16) and publications highlighting eConsult in professional and popular media (eg, Réseau-1 Québec,17 CISSSCO newsletter,18 Le Droit19). In NL, promotional activities included presentations (eg, NL Medical Association Annual General Meeting, Nurse Practitioner’s Professional Practice Group NL, Primary Healthcare Partnership Forum, NL College of Family Physicians Annual Scientific Assembly), publications disseminated through the NL Medical Association (eg, president letters, eUpdates, page on the NL Medical Association website dedicated to eConsult) and outreach to local PCPs and specialists.
Maintenance
All participating services demonstrated growth of usage during the study period, though case volumes varied based on the number of users and length of time the service had been established. Monthly case volumes for MB, QC and NL are shown in figure 7. Given the substantial difference in scale between AB and other services, monthly case volumes for AB are charted separately in figure 8. In every case, services demonstrated higher rates of usage in their last quarter of data than their first. In NL, the service completed an average of 16.3 cases/month in its first quarter of available data versus 121.7 cases/month in its last quarter. In QC, there were 4.7 cases/month in the first quarter versus 83 cases/month in the last quarter. MB’s data spanned less than 4 full quarters, with 2 cases/month in the first 3-month period and 13.6 cases/month in the last. AB completed 2.3 cases/month in its first quarter and 527 cases/month in its last quarter.
Monthly case volume of eConsult by province for Newfoundland and Labrador, Quebec and Manitoba.
Monthly case volume of eConsult in Alberta (Newfoundland and Labrador, Quebec and Manitoba services included for scale).
The participating services are at different stages of implementation. AB is a fully sustained service, integrated into the provincial EMR platform and funded directly by the province. NL has also entered the sustainability phase, with provincial expansion underway. MB and QC remain regional services at present, though in both cases discussions regarding provincial expansion and sustainment are ongoing.
Discussion
eConsult is a promising technology tool designed to reduce wait times to access specialist advice. Our findings demonstrated eConsult’s spread and scale in four provinces across Canada, with a growing number of Canadians benefiting from rapid access to specialist advice through eConsult. The implementation process varied between regions based on existing services, local needs and clinical champions, allowing us to take a tailored approach that fit each community. However, once implementation began, the services demonstrated a common pattern of growth, reported similar response times and case outcomes and delivered similarly high levels of provider satisfaction. In all cases, the median response time was far shorter than the 5–11 week-long median waits for specialist appointments reported in the literature. Using the RE-AIM framework allowed us to paint a broad comparative picture of the service in each region.
Barriers to spread and scale have limited the adoption of many promising healthcare innovations. Canada has a reputation of being a ‘land of perpetual pilot projects’, where programmes are regularly initiated but often fail to expand or sustain themselves beyond their initial implementation period.9 10 In their 2015 report, Naylor et al highlight several barriers to an innovation’s spread, including a lack of funding tailored to scaling up pilots, too little focus on patient-centred care, aversion to deploying digital technology and a fragmented healthcare system that inadvertently promotes regional siloing.10
Our team has worked to overcome these barriers through strong and fruitful partnerships with provincial and national organisations. These groups can provide vital sources for funding and support interjurisdictional coordination and knowledge sharing to ensure successful ideas do not get lost but have the opportunity to be tested and replicated in new environments. The CFHI Connected Medicine Collaborative, cited in the introduction of this paper, is a good example. Launched in 2015, the programme has led to the successful implementation of 11 programmes across 7 provinces.12 Much of its success stemmed from taking programmes that had demonstrated effectiveness along key metrics and allowing motivated groups in other provinces to replicate them while drawing on the expertise of previous implementers. Beyond the Collaborative, our team has endeavoured to promote knowledge sharing through our Think Tank and National Forum, annual events that draw clinicians, patients and decision makers from across Canada to discuss issues pertinent to eConsult’s expansion.15 The third meeting was held in November 2018, with a focus on developing best practices to support the spread of eConsult and eReferral nationwide. Patient partners have consistently been well represented at these events, reflecting our team’s commitment to maintaining a patient-centred approach to care.
Some factors, such as the number of cases completed, PCPs enrolled and specialty services available, varied substantially between provinces and are reflected in the provincial population rates of completed eConsults per 1000 people during the first year of implementation. These variations are to be expected, given differences in implementation strategy, local needs and provider availability, scope and the fact that some services had been implemented for a longer period than others—for instance, the AB service included cases from August 2014 to July 2018, while the MB service was first launched in December 2017 and thus could only report cases over a 10-month period. The population rates observed in the present study ranged from 0.0035 to 1.14 eConsults per 1000 people. In the cases of MB, QC and NL, this level of usage exceeds that exhibited in the service’s first year in the Champlain region (0.16 eConsults per 1000 people) and the Mississauga Halton region (0.36 eConsults per 1000 people).11 This increased uptake likely stems from the fact that these services were able to build on an established model of care and leverage the network of support originally generated in Ontario. AB is the outlier in this regard, as it showed an irregular pattern of usage growth reflecting its origins as an eReferral platform on which eConsult capabilities were only fully used after several years of implementation. Furthermore, it is worth noting that the AB service was implemented within a pre-established platform (ie, Netcare) that already had a substantial user base in the province. As such, recruitment was less burdensome, as it entailed getting PCPs to use a new function within a familiar platform rather than encouraging them to learn an entirely new application, which potentially explains the rapid growth in usage in 2017 when the service’s eConsult capabilities were first emphasised.
More consistency between provinces was seen in PCPs’ responses to closeout surveys among the three services where these surveys were used (MB, QC, NL), particularly between QC and NL. MB demonstrated results that varied from the other two provinces, likely due to QC and NL having a larger number of completed cases, (450 and 1656, respectively), allowing for a more robust sample. This assumption is supported by the latest numbers reported by the Champlain BASE eConsult service. In a recent study assessing 14105 Champlain BASE eConsult cases completed over a 5-year period, PCPs’ survey responses closely aligned with those from the QC and NL services: PCPs reported new/additional advice in 57% of cases (vs 58% in QC and 54% in NL), and 32% of cases resulted in a face-to-face referral (vs 27% in QC and 29% in NL).8 Given time, we anticipate that survey responses for the MB service will draw closer to this range.
The main strength of this study is the breadth of its data, which spans four provinces and multiple regions. By collating measures of eConsult’s impact across multiple jurisdictions, our findings make a strong case for eConsult’s generalisability and scalability. However, our study also has several limitations. The data included were observational and clinician based, which do not allow for a direct patient perspective. Differences in service structure and data collection meant some metrics could not be captured across all participating services. This was most notable for the AB service, which does not incorporate a closeout survey into its process and as such could not provide direct provider feedback. Furthermore, differences in structure and delivery between the AB service and the others using the BASE model present a challenge to direct comparison. Future studies should explore these issues using a standard survey across all jurisdictions. Additionally, patient participants should be sought directly to provide further insight from the patient perspective.
Conclusion
The eConsult service has been successfully implemented in four new provinces across Canada, three using the BASE model (MB, QC, NL) and one incorporating eConsult capabilities into an existing eReferral platform (AB). Implementation strategies and scope varied, but services demonstrated consistency on several key metrics, most notably case outcomes. Further time and research is needed to assess the long-term sustainability of these services and their impact on outcomes affecting patient health.
Acknowledgments
The authors wish to thank the PCPs and specialists who use the service and Justin Joschko for his assistance in editing the manuscript and preparing it for publication.
Footnotes
Contributors CL and EK conceived of and designed the study, and contributed to the data analysis and drafting of the publication. AB, JC, MDP, GF, JG, LI, LO, RM, VN and AS contributed data to the study and were involved with its conception, conduct, analysis and reporting. ND contributed to its data analysis. All authors helped write and edit the manuscript and approved the final draft.
Funding Funding for this project was provided through the Canadian Institutes of Health Research. The authors affirm their independence from these funders. The funders played no part in the study design, collection,analysis or interpretation of the data, in the writing of the report or in the decision to submit the article for publication. All authors had full access to all the data in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis.
Competing interests None declared.
Ethics approval This study was approved by the Ottawa Health Science Network Research Ethics Board (Protocol #2009848-01H).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Data from the study are available from the corresponding author on reasonable request.
Patient consent for publication Not required.