Abstract
Background: The COVID-19 pandemic resulted in a rapid shift from in-person to virtual care delivery for many medical specialties across Canada. The purpose of this study was to explore the lived experiences of resident physicians and faculty related to teaching, learning and assessment during ambulatory virtual care encounters within the competency-based medical education model.
Methods: In this qualitative phenomenological study, we recruited resident physicians (postgraduate year [PGY] 1–5 trainees) and faculty from the Departments of Surgery and Medicine at Queen’s University, Ontario, via purposive sampling. Participants were not required to have exposure to virtual care. Interviews were conducted from September 2020 to March 2021 by 1 researcher, and 2 researchers conducted focus groups via Zoom to explore participants’ experiences with the transition to virtual care. These were audio-recorded and transcribed verbatim; qualitative data were analyzed thematically.
Results: There were 18 male and 19 female participants; 20 were resident physicians and 17 were faculty; 19 were from the Department of Surgery and 18 from the Department of Medicine. All faculty participants had participated in virtual care during ambulatory care; 2 PGY-1 residents in surgery had not actively participated in virtual care, although they had participated in clinics where faculty were using virtual care. The mean age of faculty participants was 38 (standard deviation [SD] 8.6) years, and the mean age of resident physicians was 29 (SD 5.4) years. Overall, 28 interviews and 4 focus groups (range 2–3 participants per group) were conducted, and 4 themes emerged: teaching and learning, assessment, logistical considerations, and suggestions. Barriers to teaching included the lack of direct observations and teaching time, and barriers to assessment included an absence of specific Entrustable Professional Activities (EPAs) and feedback focused on virtual care–related competencies. Logistical challenges included lack of technological infrastructure, insufficient private office space and administrative burdens. Both resident physicians and faculty did not foresee virtual care limiting resident physicians’ ability to progress within competency-based medical education. Benefits of virtual care included increased accessibility to patients for follow-up visits, for disclosing patients’ results and for out-of-town visits. Suggestions included faculty development, improved access to technology and space, educational guidelines for conducting virtual care encounters, and development of virtual care–specific competencies and EPAs.
Interpretation: In the postgraduate program we studied, virtual care imposed substantial barriers on teaching, learning and assessment during the first year of the COVID-19 pandemic. Adapting to new circumstances such as virtual care with suggestions from resident physicians and faculty may help to ensure the continuity of postgraduate medical education throughout the COVID-19 pandemic.
The COVID-19 pandemic has placed an unprecedented strain on health care systems worldwide, affecting all aspects of care and necessitating a rapid shift to virtual care for many medical specialties across Canada. 1,2 Beyond altering how outpatient care is delivered, virtual care changed how faculty teach and modified resident physicians’ learning experiences.3
Various frameworks have explored dimensions of satisfaction with virtual care from both patient4 and provider perspectives. 5 The provider satisfaction framework is particularly relevant to our study, as it explores components of health care providers’ overall satisfaction with virtual care: demographic characteristics, care settings, motivations and experiences (Figure 1).5 Although prior studies have investigated the perspectives of patients and providers during transitions to virtual care, few have investigated the impact of virtual care on resident physicians’ education.6 Understanding this impact within competency-based medical education (CBME), which emphasizes outcome-based learning and feedback,7 is critical. Our objective for this study was to understand faculty and resident physicians’ experiences with teaching, learning and assessment during the transition to virtual care during the COVID-19 pandemic and to identify suggestions for improvement.
Methods
We used a qualitative phenomenological study design with interviews and focus groups to explore the perspectives of resident physicians and faculty as they transitioned to virtual care in ambulatory care. The shift to virtual ambulatory patient care at Queen’s University (Kingston, Ontario) began in March 2020, when the World Health Organization declared COVID-19 a pandemic,8 and virtual care was quickly adopted into ambulatory practice without prior training for providers. The clinical catchment area for Queen’s University includes about 495 000 people living in Southeastern Ontario.9 For this study, we defined virtual care as any form of telemedicine, including phone calls or videoconferencing using regulated platforms.10
A phenomenological approach was taken, focusing on individuals’ lived experiences,11 to understand the experiences of resident physicians and faculty with the transition to virtual ambulatory care. Details on the study design and researcher reflexivity can be found in Appendix 1, available at www.cmajopen.ca/content/10/3/E762/suppl/DC1. Consolidated Criteria for Reporting Qualitative Research (COREQ)12 was used to guide the reporting of this qualitative research.12 Recruitment took place between August 2020 and March 2021, and interviews were conducted from September 2020 to March 2021. Interviewers (J.S.S.H. and R.L.) were second-year medical students at Queen’s University trained to conduct interviews by H.B., a health education researcher with previous experience in conducting phenomenological research. The research team also consisted of faculty from the Departments of Surgery and Medicine (S.A., S.M., R.A. and B.Z.), as well as 2 resident physicians (E.K. and S.B.).
Participants
Participants were invited via email by a research team member (J.B.), provided a letter of information and consent form, and emailed to schedule interviews. Clinical members of the research team (E.K., S.B., S.A., S.M., R.A. and B.Z.) assisted in recruitment. Fellows were not included in the recruitment process.
All faculty and resident physicians within the Departments of Surgery and Medicine were contacted by email. All individuals who consented to participate in our study were recruited from this convenience sample. All faculty and resident physicians (all postgraduate years [PGYs]) participate in ambulatory care as part of their clinical practice or training. Ambulatory care consisted of onsite hospital clinics, where resident physicians were involved. All faculty participants were affiliated with Queen’s University, and none had personal offices outside of hospital. There was no requirement for participants to have extensive exposure to virtual care, as we hoped to get a good representation of the state of the transition to virtual care within both departments.
Interview guides
Two interview guides (1 for faculty and 1 for resident physicians; Appendix 2, available at www.cmajopen.ca/content/10/3/E762/suppl/DC1 and Appendix 3, available at www.cmajopen.ca/content/10/3/E762/suppl/DC1) were developed by an interprofessional team that included clinicians and medical education experts (H.B., B.Z., R.A., S.A. and S.M.), some of whom were involved in virtual care before the pandemic. The interview guides were created to reflect 4 topics of interest: the experience of using virtual care in ambulatory settings; how teaching, learning and assessments evolved with the transition to virtual care; barriers to and facilitators of conducting virtual care; and suggestions for peers and/or institutions adopting virtual care. A question relating to the ability to assess the Canadian Medical Education Directives for Specialists (CanMEDS) roles was also included, as the CBME curriculum is broadly based on these core competencies. 13 We consulted the literature on the patient–physician interaction, engagement, teaching, learning and assessment, 14–18 and literature on the CBME model to develop the guides.13 The preliminary interview guide was piloted using a think-aloud protocol with faculty members (B.Z., S.A., R.A. and S.M.) and was revised by the research team (J.S.S.H., R.L., H.B., S.A., S.M., R.A. and B.Z.) through a consensus process; however, no data from these pilots were included in the study findings. This process consisted of asking each of the interview questions, having the faculty respond to each interview question and thinking about how each question was being interpreted. At the end of this session, there was a discussion about the flow, content and wording of the questions. This process resulted in minor modifications to the questions, such as rewording to enhance clarity and revising the question order within the interview guide. In addition to the piloting process, the 3 interviewers met after conducting 3 interviews to discuss the effectiveness of the interview guide. This resulted in the addition of some prompts and reordering a few questions to enhance the flow.
Data collection
Demographic information, such as sex, age, program and level of training, was collected via the secure consent form (Qualtrics) and can be found in Appendix 4, available at www.cmajopen.ca/content/10/3/E762/suppl/DC1. Gender and race were also collected, with the option to leave the space blank if the participant preferred not to disclose. Years in practice for faculty were collected via the College of Physicians and Surgeons of Ontario website.19 Participants were asked to consent to having their Entrustable Professional Activities (EPAs) data collected from the electronic education management platform (Elentra)20 by program administrators at the time of consent. Entrustable Professional Activities are a set of specialty-specific milestones, which are tagged to standardized national competency requirements. They are triggered after clinical encounters to be assessed by faculty. Resident physicians must be deemed entrustable before moving on to subsequent stages of training.17
Although resident physicians were given the option to participate in focus groups instead of one-on-one interviews to allow for increased discussion and participation, booking several resident physicians into a single time slot was not always possible, and resident physicians with scheduling conflicts were given the option to participate in one-on-one interviews. Participants were identified by study ID codes: faculty (Fx) and resident physicians interviewed individually (Rx), and residents interviewed in focus groups (FGx). Interviews lasting 30–45 minutes and focus groups lasting 1 hour were conducted via Zoom, audio-recorded and transcribed verbatim. Two interviewers conducted the focus groups, and 1 interviewer conducted one-on-one interviews. A form of member checking was completed during the interview process, whereby the interviewers summarized back key topics to the participants. No additional follow-up interviews were conducted.
Data analysis
A phenomenological approach to data analysis was guided by 3 methodological principles: maintaining an openness to the phenomenon of interest, identifying and exploring preconceptions, and maintaining an ongoing reflective attitude.21 Virtual care consisted of both video- and telephone-based clinical interactions.10 This approach centres on both “what” the experience was and “how” it was experienced by the interviewees. 22 In particular, a hermeneutic phenomenological approach allowed us to understand the themes that were identified from the experiences and to interpret these iteratively to provide insight into the experience of teaching and learning in a virtual clinical environment.23
Three independent coders (J.S.S.H., R.L. and H.B.) analyzed the qualitative data using open coding in QSR International’s NVivo software (Version 12).24 Throughout the data analysis process, the researchers most closely involved with data collection and analysis (J.S.S.H., R.L. and H.B.) met with members of the wider research team (S.A., S.M., R.A. and B.Z.) to discuss themes, subthemes, emerging codes, interpretations and potential new lines of enquiry, thereby drawing on the combined insights of those closely handling the data and members of the team with the wider perspective of clinical and educational experience. A diverse sample of transcripts from interviews and focus groups across participant groups and specialties was selected for comparison to ensure intercoder reliability (3 transcripts from the Department of Surgery and 3 transcripts from the Department of Medicine). The data were analyzed inductively using thematic analysis and completed using an iterative approach in 3 steps: preliminary coding, line-by-line coding and generation of themes.25 In preliminary coding, 3 researchers coded half of an interview together and coded the other half independently for intercoder reliability. Interviews were analyzed as they were conducted, which then informed subsequent interviews until theoretical saturation occurred. Subsequently, a codebook was created to guide the coding of the remaining transcripts. Details on the coding process can be found in Appendix 1.
Ethics approval
Ethics approval was obtained from the Queen’s University Health Sciences and Affiliated Teaching Hospitals Research Ethics Board (HSREB #6030474).
Results
A total of 175 individuals were contacted regarding the study, and 42 individuals consented to participate. However, 2 faculty and 3 resident physicians did not respond to multiple requests from the research team to schedule their interview or focus group. As such, a total of 37 participants were included in the analysis, with 20 resident physicians and 17 faculty (Table 1). Faculty participated in 17 semistructured individual interviews, and resident physicians participated in 11 semistructured individual interviews and 4 focus groups (range 2–3 participants per group) between September 2020 and March 2021.
Participants were evenly distributed across the Departments of Surgery (n = 19) and Medicine (n = 18), and across sex (male n = 18, female n = 19). The mean age among faculty was 38 (standard deviation [SD] 8.6) years and 29 (SD 5.4) years among resident physicians. The mean number of years in practice for faculty was 10.1 (SD 7.8) years.
Exposure to virtual care
All faculty participants in our study had participated in virtual care in some capacity during their ambulatory clinics. Two PGY-1 residents in surgery did not actively participate in virtual care, although they did participate in clinics where faculty were using virtual care. All other residents had exposure to virtual care. There were differences among resident physicians at different stages of training, with more senior residents (PGY-4 and PGY-5) having more autonomy to complete virtual care clinics on their own, and junior residents (PGY-1 and PGY-2) having fewer opportunities to participate in virtual care clinics, as faculty members often tried to prioritize their learning from in-person ambulatory patients. In addition, during the interviews with resident physicians and faculty in the Department of Surgery, it was mentioned that ambulatory clinics were often lower in priority than other learning opportunities, such as participation in the operating room or more procedure-based clinics, such as endoscopy. As such, resident physicians in the Department of Surgery may have less exposure to virtual care than resident physicians in the Department of Medicine.
Four major themes around the transition to virtual ambulatory care emerged from our analysis: teaching and learning, assessment, logistical considerations, and suggestions. Within each theme, several subthemes were identified (Tables 2, 3, 4 and 5).
Teaching and learning
Adoption of virtual care in ambulatory settings changed many facets of teaching and learning. Four subthemes were identified: evolution, strategies, facilitators and barriers (Table 2).
The teaching strategies employed by faculty using virtual care evolved as the pandemic progressed and in-patient capacities fluctuated. The data represented a wide variety of virtual clinic structures and strategies, some occurring concurrently with in-person encounters, and some functioning solely virtually. Faculty who included resident physicians in their virtual clinics frequently asked the resident physicians to call patients independently before reviewing with them. The resident physicians would then call the patients back to discuss the plan, with or without faculty’s presence.
Many resident physicians identified opportunities during virtual care clinics to research answers to clinical questions and review with the faculty to facilitate their learning. Although this helped with resident physicians’ learning, faculty stated that this made assessing the resident physicians’ knowledge more challenging. Additionally, resident physicians noted that their learning was facilitated by program support and that faculty would go out of their way to find learning experiences for them. Overall, there were substantial barriers to teaching and learning for both faculty and residents. Most participants felt that the lack of a physical examination substantially reduced the physical learning and teaching capacity. Additionally, the decrease in patient volumes, and time constraints imposed by the virtual modalities, resulted in fewer teaching moments for resident physicians.
Assessment
Five subthemes within assessment were identified: feedback, EPAs, facilitators, barriers and CanMEDS roles (Table 3).
Methods and timing of feedback varied among participants. Some faculty provided oral feedback directly after cases, and others documented formal feedback in the workplace-based assessments. Feedback frequency was low regardless of visit type. Moreover, resident physicians in the Department of Surgery had limited EPAs to assess performance in clinics, with most EPAs pertaining to technical skills of the specialty, and resident physicians in the Department of Medicine noted that their clinic EPAs were not adapted for virtual care. In general, the current EPAs were not applicable to virtual care. Most faculty suggested creating EPAs focused on communication and virtual competencies to help address this issue.
Neither resident physicians nor faculty saw virtual care as a barrier to assessment. They did not foresee virtual care limiting a resident physicians’ ability to progress within CBME, as most resident physicians continued to engage in some form of in-person care. However, resident physicians and faculty noted that there was very little direct observation during virtual care, which was a substantial barrier to providing constructive feedback. There was also limited in-person and narrative feedback documented on workplace-based assessments. Faculty felt they were still able to assess the various CanMEDS competencies, particularly the communicator role.
Logistical considerations
Three subthemes emerged pertaining to logistical considerations: limitations of virtual care, infrastructure of virtual care and modality of virtual care (Table 4).
Although many faculty members included resident physicians in virtual ambulatory clinics, some did not owing to lack of space, the need for extra computers and phones, and time constraints. Instead, these faculty members tended to include resident physicians for in-person clinics. Faculty and resident physicians noted that virtual care had unique logistical limitations compared with in-person clinics. For example, they felt that patients treated virtual visits differently from in-person visits. Patients often missed phone calls, participated in other activities during the visit, or were less prepared to have a discussion with a physician, which contributed to the difficulties of maintaining a clinic schedule.
Resident physicians and faculty felt that using video technology, rather than telephones, could help mitigate many of the barriers to teaching and assessment, including the lack of physical examination and direct observation, while establishing patient rapport. However, video consults created a substantial burden on administrative staff and were often challenging for some patients to access. There was limited technical support or IT infrastructure for virtual clinics (e.g., Ontario Telemedicine Network), leading to virtual care clinics being conducted almost exclusively by telephone. Lastly, there was a reported lack of physical space and resources to conduct virtual care encounters in a private and confidential manner.
Suggestions
Six subthemes emerged pertaining to suggestions: preparation, resources, technology, education and support, clinic efficiency, and next steps (Table 5). Key suggestions from participants to improve teaching and learning in virtual ambulatory clinics are shown in Table 6.
Faculty and resident physicians suggested that preparation was key to successfully engaging learners in virtual care and optimizing their educational experiences. Faculty discussed improving resources, such as administrative support, to book virtual clinics. Resident physicians and faculty felt that a dedicated physical space with additional computers and phones was needed for resident physicians to conduct virtual care and maintain patient confidentiality.
Faculty and resident physicians emphasized that improvement in video technologies is critical for increased virtual care uptake, specifically for surgical specialties. Educational sessions and institutional supports surrounding virtual care would be beneficial, such as educational guidelines for resident physicians on how to effectively conduct virtual care, and training for faculty on structuring virtual ambulatory clinics to maximize resident physician learning and to deliver high-quality care. Resident physicians suggested that it would be particularly helpful to have guidance on how to adapt certain physical examination manoeuvres to the virtual care environment. Developing competencies specific to virtual care was emphasized as crucial for resident physicians’ education.
Overall, faculty and resident physicians agreed that virtual care will persist after the COVID-19 pandemic and optimizing virtual care infrastructure will be necessary. Participants expressed the desire to continue providing virtual care for follow-up visits, for disclosing patients’ results and for out-of-town visits. Participants stressed that as virtual care continues, accessibility and equity need to be considered, and suggested that it would be beneficial to better understand patient perspectives on virtual care to ensure that it is meeting patients’ needs. Key suggestions to address these barriers included ensuring appropriate logistical infrastructure, and instructional guidelines for faculty and resident physicians on how to deliver CBME in a virtual setting (Figure 2).
Interpretation
The transition to virtual care in an ambulatory setting during the COVID-19 pandemic has altered residency education through changes in the teaching strategies employed by faculty, opportunities for direct observations in assessments, and resident physicians’ interactions with faculty and patients. Primary barriers to virtual care include the inability to perform a physical examination, difficulty using video technologies and the lack of applicable EPAs.
As the provider satisfaction framework (Figure 1) suggests, we can analyze providers’ satisfaction with virtual care from 4 key domains (demographic characteristics, care settings, experiences and motivations) to better understand some of the issues faced by our participants during the transition to virtual care.5
Teaching and learning are key aspects of residency education that were affected by the transition to virtual care, with decreased in-person encounters and difficulty teaching in a virtual landscape. Using the provider satisfaction framework, 5 this transition corresponds to the “motivations” of faculty and resident physicians to teach and learn during virtual care clinics. Motivation may help explain why faculty continued to include resident physicians in virtual care clinics despite it taking more time, and why resident physicians continued to seek out virtual care clinics outside of their inpatient or operating room experiences. By continuing to explore ways to incorporate resident physicians into virtual clinics, with opportunities for direct observation by “listening in” on visits conducted by phone or videoconferencing, educators can ensure progression of resident physicians within the stages of CBME.26 However, guidance is needed on how to balance the demands of virtual care clinics with the needs of resident physician education.18 The Virtual Care Task Force of the Royal College of Physicians and Surgeons of Canada recently made a recommendation to the CanMEDS consortium to update the virtual care competencies for all levels of medical training, and we suggest that these could be used to support the production of EPAs specific to virtual care.27
In our study, assessments of resident physicians during virtual care were affected by the inability to perform a physical examination, conduct direct observations and complete EPAs specific to virtual care. This highlights a gap in assessment during virtual care, which needs to be addressed. The inability to conduct direct observations in clinic was likely a barrier even before the pandemic; studies have reported that resident physicians were rarely observed by faculty even before March 2020.16 However, phone-based virtual care creates new challenges in direct observation of resident physicians and necessitates the development of virtual care–specific competencies.28 Once developed, virtual care–specific EPAs can be assessed in the workplace or through objective structured clinical examinations (OSCEs) to ensure that resident physicians are competent in conducting virtual care visits.29 Although blended learning models of virtual and in-person care can be effective in residency education,30 blended models may not fully replace traditional in-person learning.31 Current best practices for resident physician education using virtual care are unclear, though a study by Ha and colleagues supports the use of video technologies and educational frameworks. 32 One example is the medical virtualist framework,28 which outlines 3 key domains for core competencies needed to provide adequate virtual care: communication and “webside” manner, scope and standards of care, and virtual clinical interactions. Creation or adoption of a virtual care–specific curriculum for resident physicians, similar to that of Ha and colleagues,32 would also be helpful. Their curriculum included a didactic lecture and a quick reference handout for conducting a virtual care visit, covering topics such as logistics, technology set-up, etiquette, effective communication, virtual physical examination, documentation and staffing with an attending.32 We suggest that a similar curriculum may better enable resident physicians to participate fully in virtual care.
In our study, participants were often challenged by logistical considerations, including the lack of existing infrastructure and technology, time constraints and a lack of dedicated virtual care spaces. These factors reflect the “care settings” and “experiences” components of the provider satisfaction framework. 5 They encompass the lack of technical support faced by providers, as well as limited software, hardware and technical support and training. Participants provided several suggestions to help address these challenges and to improve the implementation and uptake of technology for virtual care (Table 6). Our findings are similar to those of prior studies that explored barriers to the uptake of virtual care and found that user experience with technologies and technological limitations were key factors in satisfaction with virtual care.33,34
Despite the challenges identified with transitioning to virtual care in medical education, participants reiterated that virtual care is likely here to stay and that it adds value to many patients — particularly those who are seen for follow-up or who may be travelling from farther away. Some resident physicians also suggested that virtual care clinics allowed them to be more prepared as they were often able to look up results or clarify information for patients more readily, as they could look things up at a computer without undermining the physician–patient interaction. Most resident physicians described that they would like to incorporate virtual care into their future practices, and faculty often reported that they will continue to conduct virtual care clinics. This highlights the importance of adapting to this changing reality to better support both resident physicians and faculty in the uptake, teaching and assessment of virtual care practices.
Strengths of our study included recruitment of participants from surgery and medicine specialties and training programs, and participants with variable virtual care experiences, ranging from limited participation in virtual care to full participation with most ambulatory clinics conducted virtually. Additionally, our research team included clinicians, medical education scholars and qualitative methodologists to ensure a rigorous methodological approach.
Limitations
The single-centre nature of this study may limit the representativeness of the results outside of Queen’s University and to disciplines outside of surgery and medicine. Participants were included via a convenience sample regardless of demographic characteristics such as age, gender, practice experience and training level, as all faculty and resident physicians who agreed to participate were included. Further, we must acknowledge the limitation of self-reported data given that participants were asked to describe their virtual care practices. Our study also focused on experiences during one point in time, and thus we were unable to capture the evolution or longitudinal changes in teaching, learning or assessment during virtual care encounters. We collected data earlier in the COVID-19 pandemic, and teaching and assessment practices may have evolved as faculty gained experience with virtual care.
Conclusion
The COVID-19 pandemic and resultant rapid transition to virtual care has transformed residency education by changing teaching and assessment in ambulatory settings. This transition imposed substantial barriers on teaching, learning and assessment, including the inability to perform a physical examination, difficulty using video technologies and the lack of applicable EPAs. Increased use of video technologies with IT support, creation of virtual care–specific EPAs, and instructional guidelines on how to conduct virtual care are key suggestions to facilitate training of resident physicians who are competent in providing safe, high-quality and effective virtual care to their patients.
Footnotes
Competing interests: Ramana Appireddy has received research funding from the PSI Foundation, Canadian Institutes of Health Research, Canada Health Infoway and Southeastern Ontario Academic Medical Organization. Boris Zevin has received an education grant from Ethicon. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Jessica Ho and Rebecca Leclair are co–first authors of this article. Jessica Ho and Rebecca Leclair contributed to the conception or design of the work, and to the acquisition, analysis and interpretation of data, and drafted the article. Heather Braund, Ekaterina Kouzmina and Samantha Bruzzese contributed to the analysis and interpretation of data. Jennifer Bunn contributed to the acquisition of data. Sara Awad, Steve Mann and Ramana Appireddy contributed to the conception of the work and to the interpretation of data. Boris Zevin contributed to the conception of the work, and to the analysis and interpretation of data. All authors revised the work for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Funding: No funding was received for this study.
Data sharing: Portions of the data may be available to others with permission from the senior author. To obtain permission for use of any part of the data, please contact Dr. Boris Zevin at bz15{at}queensu.ca.
Supplemental information: For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/10/3/E762/suppl/DC1.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/
References
- © 2022 CMA Impact Inc. or its licensors