Admission factors associated with international medical graduate certification success: a collaborative retrospective review of postgraduate medical education programs in Ontario
==================================================================================================================================================================================

* Lawrence E.M. Grierson
* Mathew Mercuri
* Carlos Brailovsky
* Gary Cole
* Caroline Abrahams
* Douglas Archibald
* Glen Bandiera
* Susan P. Phillips
* Glenna Stirrett
* J. Mark Walton
* Eric Wong
* Inge Schabort

## Abstract

**Background:** The failure rate on certification examinations of The College of Family Physicians of Canada (CFPC) and the Royal College of Physicians and Surgeons of Canada (RCPSC) is significantly higher for international medical graduates than for Canadian medical school graduates. The purpose of the current study was to generate evidence that supports or refutes the validity of hypotheses proposed to explain the lower success rates.

**Methods:** We conducted retrospective analyses of admissions and certification data to determine the factors associated with success of international medical graduate residents on the certification examinations. International medical graduates who entered an Ontario residency program between 2005 and 2012 and had written a certification examination by the time of the analysis (2015) were included in the study. Data available at the time of admission for each resident, including demographic characteristics, previous experiences and previous professional experiences, were collected from each of the 6 Ontario medical schools and matched with certification examination results provided by The CFPC and the RCPSC. We developed logistic regression models to determine the association of each factor with success on the examinations.

**Results:** Data for 900 residents were analyzed. The models revealed resident age to be strongly associated with performance across all examinations. Fluency in English, female sex and the Human Development Index value associated with the country of medical school training had differential associations across the examinations.

**Interpretation:** The findings should contribute to an improved understanding of certification success by international medical graduates, help residency programs identify at-risk residents and underpin the development of specific educational and remedial interventions. In considering the results, it should be kept in mind that some variables are not amenable to changes in selection criteria.

Postgraduate training in Canada is a critical step on the path to medical practice in Canada for international medical graduates. Success rates on the Royal College of Physician and Surgeons of Canada (RCPSC) and The College of Family Physicians of Canada (CFPC) licensure examinations that follow postgraduate training indicate that international medical graduates have more difficulty achieving certification than do graduates of Canadian medical schools. For example, in 2007, the pass rate among international medical graduates who were residency trained in Canada on their first attempt of the CFPC's certification examination (Certification in The College of Family Physicians [CCFP]) was 66.0%, compared to 90.4% for their Canadian medical graduate counterparts.1 Success rates for international medical graduates varied between 51% and 74% in the ensuing 3 years.2 More recently, pass rates among international medical graduate residents on the CCFP examination were 66.7% in 2015 and 88.8% in 2016, compared to 93.3% and 93.5%, respectively, for Canadian medical graduates (data presented in an internal report to the CFPC Board of Examiners). Similarly, between 2007 and 2016, 78.9% of international medical graduate residents passed RCPSC examinations, compared to 95.2% of Canadian medical graduates.

In October 2010, the Ontario government and the deans of Ontario medical schools commissioned an independent review of barriers and solutions to fair access for international medical graduates seeking postgraduate positions with the province's faculties of medicine. With reference to initial selection criteria for admission to residencies, the resulting report3 articulated a need to better understand the predictors of certification success of these graduates:

[T]he Faculties of Medicine, supported by the Ontario Government, should identify research priorities to increase the evidence base for selection decisions and outcomes, including … predictors for success in residency and beyond, including the best ways to weigh and measure those factors. … We recommend that support for research on predictors of success and ways to improve certification examination results, using the Ontario experience, should be a priority.

In accordance with the priorities highlighted in the report, we began province-wide research to determine the factors associated with success of international medical graduates on certification examinations. Several mechanisms have been proposed to explain their lower success rates. Some of these reflect demographic characteristics of this population, such as second-language proficiency4 and older age,5 whereas others involve features of the foreign medical education experience,6-8 the time elapsed since medical training9 and the impact of previous clinical experiences.10 In addition, some investigators have considered the effect that cultural beliefs about sex roles and physician-patient communication may have in explaining the phenomenon11,12 (see Walsh and colleagues2 for a review). The purpose of the current study was to generate evidence that supports or refutes the validity of these hypotheses.

## Methods

### Design

This was a retrospective study that used information available at the time of resident selection to develop, via regression analyses, models of association for certification examination results.

### Study population

We aimed to include data for all international medical graduates who entered a residency program at 1 of the 6 Ontario medical schools (McMaster University, Northern Ontario School of Medicine, Queen's University, University of Ottawa, University of Toronto, Western University) between 2005 and 2012 and subsequently sat a certification examination. International medical graduates who had not written either the CCFP or the RCPSC certification examination were excluded from analysis. We defined an international medical graduate as a Canadian citizen or permanent resident who had received a medical degree at an institution outside Canada. Data concerning residents were de-identified at the level of the resident and institution of postgraduate medical training before analysis. Accordingly, findings for all participating universities are aggregated.

### Data sources

We collected data from application information that was submitted to postgraduate residency programs in Ontario through the Canadian Resident Matching Service or that were held at the Ontario Physician Human Resources Data Centre between 2005 and 2012. These data include demographic characteristics and details about the resident's country of medical training, medical school experiences, professional activities and, in many cases, the Medical Council of Canada Evaluating Examination (MCCEE) as well as the Clinical Examination 1 or the National Assessment Collaboration objective structured clinical examination. To accomplish this, we developed a process for collating and analyzing resident data that was in conformity with the ethics standards and security concerns of the partner institutions. This included the drafting of a comprehensive data-sharing agreement, which described considerations for governance, provisions for ensuring confidentiality and secure operational support for data management through the Ontario Physician Human Resources Data Centre.

Collation of the data from the 6 Ontario postgraduate residency programs showed the following factors to be sufficiently represented to be included in the proposed models: age, sex, previous internship (yes/no), previous residency (yes/no), previous professional experience (yes/no), research experience (yes/no), conference attendance (yes/no), publications (yes/no), teaching dossier (yes/no), fluency in English (yes/no) and country of medical school training (Human Development Index [HDI] value). Previous professional experience refers to experience that residents may have had as practising physicians elsewhere before joining their Ontario residency program. We determined fluency in English by attestation of primary and secondary, or medical school education, conducted in English.13 The HDI is a composite statistic of life expectancy, education and per capita income indicators and is used to rank countries along a scale of human development. HDI values increase with greater lifespan, education and gross domestic product, and with lower fertility and inflation rates. By transforming the country of medical education factor to an HDI value, we were able to treat it as a continuous variable. The HDI value assigned to each country was based on the 2011 rating.14

We then merged these data with certification examination data provided by the CFPC and the RCPSC with support from the Ontario Physician Human Resources Data Centre. In particular, performance on the first attempt of the CCFP and RCPSC certification examinations - specifically, the score on the Simulated Office Orals (SOOs [dichotomous]), the score on the Short Answer Management Problems (SAMPs [dichotomous]), pass/fail on the combined SOOs and SAMPs, and pass/fail on the RCPSC certification examination - was used as the dependent variables in regression models.

### Statistical analysis

We used multivariable logistic regression models to determine which variables were associated with success on the CCFP and RCPSC examinations. The methods of analysis included the construct of univariate logistic regressions for each independent variable as it pertained to each relevant dependent variable. Any independent variable that was significant at an α value set at *p* ≤ 0.1 was included in a multivariable logistic regression for each relevant dependent variable. Through a stepwise approach, we removed nonsignificant variables (*p* ≥ 0.05) until the model yielded only significant variables at an α value set at *p* < 0.05. We performed analyses using Stata version 13 (StataCorp).

### Ethics approval

Each participating school received institutional approval from its local research ethics board, including the relevant Health Sciences research ethics board, before study commencement.

## Results

In total, 900 international medical graduate residents (431 certified by the RCPSC and 469 by the CFPC) between 2005 and 2012 were included (Figure 1). A total of 1077 international medical graduates wrote the RCPSC certification examinations and 575 international medical graduates wrote the CCFP certification examination for the first time over the same period. Thus, the cases analyzed represent 40.0% and 83.6%, respectively, of the total potential populations. Stepwise multivariable regression analyses of family medicine residents reflected a sample size of 469; this discrepancy was due to missing data on age for 12 residents (Figure 1). The included participants' pass rate on the RCPSC examination was 84.4% and on the CCFP examination, 79.3%.

![Figure 1](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/5/4/E785/F1.medium.gif)

[Figure 1](http://www.cmajopen.ca/content/5/4/E785/F1)

Figure 1 
Participant selection. Note: CFPC = College of Family Physicians of Canada, RCPSC = Royal College of Physicians and Surgeons of Canada.

Univariate analyses of success on the SOOs component of the CCFP examination yielded age (OR 0.95, *p* = 0.007) and previous professional experience (OR 0.51, *p* = 0.05) as associated with success at *p* < 0.1 (Table 1). However, multivariable regression analyses for this dependent variable showed only age to be associated with success: as age increased, the residents' odds of passing this portion of the examination decreased (Table 1).

View this table:
[Table 1:](http://www.cmajopen.ca/content/5/4/E785/T1)

Table 1:   Association of factors on admission to residency with success on Simulated Office Orals component* of the Certification in The College of Family Physicians examination

Univariate analyses of success on the SAMPs component of the CCFP examination showed age (OR 0.95, *p* = 0.001), female sex (OR 2.07, *p* = 0.004), fluency in English (OR 1.68, *p* = 0.04), country of medical school training (OR 8.48, *p* = 0.03), previous internship (OR 0.54, *p* = 0.03) and previous publications (OR 2.28, *p* = 0.03) to be associated with success at *p* < 0.1 (Table 2). Age, female sex and fluency in English were retained as significant in the multivariable model. Specifically, the odds of passing this portion of the examination decreased as age increased and increased with fluency in English (Table 2).

View this table:
[Table 2:](http://www.cmajopen.ca/content/5/4/E785/T2)

Table 2: T**able 2:** Association of factors on admission to residency with success on Short Answer Management Problems component* of the Certification in The College of Family Physicians examination

Univariate analyses of success on the composite CCFP examination (SOOs and SAMPs) yielded age (OR 0.95, *p* < 0.001), female sex (OR 1.58, *p* = 0.04), fluency in English (OR 1.56, *p* = 0.05), country of medical school training (OR 7.22, *p* = 0.02), previous professional experience (OR 0.63, *p* = 0.06) and previous publications (OR 2.05, *p* = 0.03) as associated with success at *p* < 0.1 (Table 3). Age and fluency in English were retained as significant in the multivariable model. Specifically, the odds of passing the examination decreased as age increased and increased with fluency in English (Table 3).

View this table:
[Table 3:](http://www.cmajopen.ca/content/5/4/E785/T3)

Table 3:   Association of factors on admission to residency with success on composite Certification in The College of Family Physicians examination*

Univariate analyses of success on the RCPSC examination yielded age (OR 0.94, *p* = 0.001), fluency in English (OR 1.74, *p* = 0.05), country of medical school training (OR 102.70, *p* < 0.001), previous residency (OR 0.44, *p* = 0.01) and previous professional experience (OR 0.38, *p* = 0.002) as associated with success at *p* < 0.1 (Table 4). Age, fluency in English and country of medical school training were retained as significant in the multivariable model. Specifically, the odds of passing the examination decreased as age increased, increased with fluency in English and increased as the HDI rating of the country of training increased (Table 4). Of note, a correlation matrix revealed a low to moderate negative relation between age and country of medical school training (*r* = -0.36, *p* < 0.001). To understand the impact of potential multicollinearity on the age and fluency in English associations, we conducted an exploratory analysis in which the country of training variable was removed from the model. This analysis yielded no appreciable changes to the associations of age and of fluency in English with success, and, as such, we retained the country of training factor in the multivariable analysis. It should be noted that the seemingly high odds ratio associated with this factor is a function of the HDI scoring (scale between 0 and 1).

View this table:
[Table 4:](http://www.cmajopen.ca/content/5/4/E785/T4)

Table 4:   Association of factors on admission to residency with success on Royal College of Physicians and Surgeons of Canada certification examination*

## Interpretation

Although many of the factors known by the residency programs at the time of resident selection were not associated with international medical graduates' success on the CCFP and RCPSC examinations, we found some significant associations. Most prominently, resident age was strongly associated with performance across all examinations: the odds of success were lower for older residents. There are several ways in which this finding may be interpreted. Age may be a surrogate measure for time elapsed since medical education and/or a resident's most recent professional experiences. However, age may also reflect professional approaches that are more fully developed in the older international medical graduate residents, who may thus have more difficulty adapting work habits to meet the unique demands of the Canadian medical system.5 Fluency in English was significantly and positively associated with success on the SAMPs component of the CCFP examination and overall on the CCFP and RCPSC examinations. The written nature of these tests offers a likely explanation for the significant effect of this factor. The HDI score of the country of medical school training was also associated positively with success on the RCPSC examination. It is important to keep in mind that the HDI score reflects development at the level of the country and does not account for the quality of specific medical training institutions within each country. We included it in our models to inform on the potential influence that heterogeneity in global experiences may have on international medical learners in Ontario. Ideally, we would have included the individual schools as independent variables in the model. However, this would have involved too many schools and, thus, would have affected model stability. Interestingly, HDI score was not associated with CCFP examination performance.

The only other factor that emerged as significant was female sex, which was positively related to performance on the SAMPs portion of the CCFP examination. Given that this component is a written examination and that sex was not significantly associated with the oral component (i.e., the SOOs), it is difficult to interpret this finding as a function of inherent sex-based rater bias. However, this result raises possibilities that warrant further investigation. For instance, the SAMPs from this period may have covered content or have been worded in a fashion that favoured female performance. It is also possible that the imbalance of global opportunities for women in health care means that foreign-trained women seeking admission into Canadian postgraduate training programs are from an inherently more qualified applicant pool.

Importantly, this research is hypothesis generating, such that all conclusions drawn must be considered ad hoc interpretations. Furthermore, in weighing each of the factors identified as significant, one must keep in mind that the Canadian Human Rights Act15 precludes selection decisions that are made on the basis of race, colour, national or ethnic origin, religion, age, family or marital status, sex, pardoned conviction, disability or sexual orientation. In this regard, we feel that the current analysis is best conceptualized as providing evidence that could drive the improvement of in-program supports that are made available to international medical graduate residents. For instance, postgraduate medical education training programs may further investigate the challenges that older international medical graduate residents face in their learning and use the information gained to adjust the organization of curricula, the nature of assessment and performance review, and/or remediation approaches in order to offer educational activities that more appropriately address the needs of these learners.

The current endeavour is noteworthy in that it involved a data-sharing collaboration between Ontario postgraduate medical education programs, The CFPC and the RCPSC. Our ability to engage collaborators in this study suggests great promise for the development of a functional approach to conducting large-scale integration and analyses of educationally relevant trainee data that is acceptable to a number of stakeholders. As a consequence, the wider medical education research community is now well situated to scale up this process of interinstitutional data sharing to include other studies of student performance at a provincial or even a pan-Canadian level. This is not to say that the data collection and management process was not without its challenges. Indeed, our experiences with the current research project provided important lessons that will be informative to future similar medical education research. Of particular note, our original research plan aimed to include resident performance on the MCCEE as an independent variable in the regression models. However, the standard that defines an acceptable level of performance is reestablished about every 5 years. Given this, as well as the fact that the residents in our sample had graduated from medical school as early as the early 1990s and that the year in which the MCCEE was written was not indicated in the data sets, we concluded that these scores could not be reliably included in the analytical models. However, MCCEE scores are likely powerful predictors of resident success. Thus, we encourage Canadian medical education institutions to adjust their data-recording practices so that information about the year of testing and the relevant standard for passing the test are recorded. Access to the overall variability in MCCEE scoring for any one particular year would also be valuable to the use of these data. Our research group was also interested in including the Clinical Examination 1 or the newer National Assessment Collaboration objective structured clinical examination in our analyses. However, these scores, in addition to being subject to the same issues of standardization as the MCCEE, were not present in all of the applicant files because they have not always been a standard requirement in the residency selection process for international medical graduates. Accordingly, we also advocate postgraduate medical education programs to consistently update resident information as it becomes available.

### Limitations

A limitation of this study is that there is a considerable proportion of data missing for international medical graduates who wrote the examinations, particularly the RCPSC examination. Although we feel strongly that the represented data are sufficient to support the inferences that we describe, we acknowledge that analyses of fuller sets of data may yield patterns of results that are different from those presented. Moreover, this study did not include analyses of resident performance on subsequent attempts of the examinations. In this regard, it is worth noting that success is defined with specific respect to performance on the first attempt of the relevant certification examination and not the ultimate achievement of certification. It is possible that a differential pattern of associated factors would have emerged had analyses included subsequent attempts at the examinations. However, acquisition of the data needed to conduct such analyses was outside the scope of this initial study. Last, given the number of models constructed, this study is susceptible to type I error. As such, we recognize that significant associations yielded by these analyses should be viewed as suggesting support for potential hypotheses of resident success rather than confirming a predictive relation between any factor and outcome.

**Conclusion**This work represents an important initiative in developing an evidence-based platform for determining the factors that have the most substance in explaining the performance of international medical graduates on certification examinations. The research project has resulted in a relatively large data set that is reflective of the international medical graduate population province-wide and that sets the scene for rigorous hypothesis testing of the mechanisms proposed to explain examination performance. Ultimately, this work should contribute to an improved understanding of certification success by international medical graduates, help residency programs identify at-risk residents and underpin the development of specific educational and remedial interventions. Furthermore, this work serves as an important first step in developing an infrastructure by which the determinants of certification examination performance among all medical graduates may be examined. Finally, this collaborative project has addressed a major gap: the lack of data integration across institutions and with the colleges.

### Supplemental information

For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/5/4/E785/suppl/DC1.](http://www.cmajopen.ca/content/5/4/E785/suppl/DC1.)

## Acknowledgements

The authors thank the Ontario Physician Human Resources Data Centre for its invaluable contributions to the success of this project.

## Footnotes

*   **Competing interests:** None declared.

*   **Contributors:** Lawrence Grierson, Mathew Mercuri and Inge Schabort conceived the study. Lawrence Grierson designed the study, led the writing of the manuscript and supervised all aspects of the project. Mathew Mercuri managed the collation of the various data sets, designed the statistical models, conducted the statistical analysis and facilitated the interpretation of the data. Carlos Brailovsky, Gary Cole, Caroline Abrahams, Douglas Archibald, Glen Bandiera, Susan Phillips, Glenna Stirrett, Mark Walton, Eric Wong and Inge Schabort managed the collation, submission and integration of data from their respective collaborating institutions, facilitated the data transfer processes with the Ontario Physician Human Resources Data Centre and contributed to the interpretation of the data. All of the authors contributed to the critical revision of the manuscript, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.

*   **Funding:** This work was funded by Royal College of Physicians and Surgeons of Canada Medical Education Research Grant award14/MERG-05.

## References

1.  MacLellan AM, Brailovsky C, Rainsberry P, et al. (2010) Examination outcomes for international medical graduates pursuing or completing family medicine residency training in Quebec. Can Fam Physician 56:912–8.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiY2ZwIjtzOjU6InJlc2lkIjtzOjg6IjU2LzkvOTEyIjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby81LzQvRTc4NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

2.  Walsh A, Banner S, Schabort I, et al. International medical graduates - current issues. Paper 5. Members of the FMEC PG consortium; 2011. Available:[https://afmc.ca/pdf/fmec/05\_Walsh\_IMG%20Current%20Issues.pdf](https://afmc.ca/pdf/fmec/05_Walsh_IMG%20Current%20Issues.pdf). accessed 2017 Oct. 18.
    
    

3.  Thomson G, Cohl K. Selection IMG: independent review of access to postgraduate programs by international medical graduates in Ontario - volume 1: findings and recommendations. [Submitted to the Ontario Ministry of Health and Long-Term Care and the Council of Ontario Universities]; 2011. Available[www.health.gov.on.ca/en/common/ministry/publications/reports/thomson/v1\_thomson.pdf](http://www.health.gov.on.ca/en/common/ministry/publications/reports/thomson/v1_thomson.pdf). accessed 2017 Nov. 8.
    
    

4.  van Zanten M, Boulet J (2008) Medical education in the Caribbean: variability in medical school programs and performance of students. Acad Med 83(Suppl 10):S33–6.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/ACM.0b013e318183e649&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18820496&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE785.atom) 

5.  Kanna B, Gu Y, Akhuetie J, et al. (2009) Predicting performance using background characteristics of international medical graduates in an inner-city university-affiliated internal medicine residency training program. BMC Med Educ 9:42.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19594918&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE785.atom) 

6.  Gayed NM (1991) Residency directors' assessments of which selection criteria best predict the performances of foreign-born foreign medical graduates during internal medicine residencies. Acad Med 66:699–701.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/00001888-199111000-00015&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=1747183&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE785.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1991GQ42400017&link_type=ISI) 

7.  Perez JA, Greer ES (2009) Correlation of United States medical licensing examination and internal medicine in-training examination performance. Adv Health Sci Educ Theory Pract 14:753–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1007/s10459-009-9158-2&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19283500&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE785.atom) 

8.  Shiroma PR, Alarcon RD (2010) Selection factors among international medical graduates and psychiatric residency performance. Acad Psychiatry 34:128–31.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1176/appi.ap.34.2.128&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20224023&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE785.atom) 

9.  Bessant R, Bessant D, Chesser A, et al. (2006) Analysis of predictors of success in the MCRP (UK) PACES examination in candidates attending a revision course. Postgrad Med J 82:145–9.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6InBvc3RncmFkbWVkaiI7czo1OiJyZXNpZCI7czoxMDoiODIvOTY0LzE0NSI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNS80L0U3ODUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

10. Part HM, Markert R (1993) Predicting the first year performances of international medical graduates in an internal medicine residency. Acad Med 68:856–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/00001888-199311000-00015&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=8216659&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE785.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1993MG54100018&link_type=ISI) 

11. Andrew R, Bates J (2000) Program for licensure for international medical graduates in British Columbia: 7 years' experience. CMAJ 162:801–3.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNjIvNi84MDEiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzUvNC9FNzg1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

12. Bates J, Andrew R (2001) Untangling the roots of some IMGs' poor academic performance. Acad Med 76:43–6.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/00001888-200101000-00012&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=11154193&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE785.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000166373200011&link_type=ISI) 

13. Canadian Resident Matching Service (CaRMS) provincial eligibility criteria: Ontario. Ottawa: CaRMS; [updated 2017]. Available[www.carms.ca/en/residency/r-1/eligibility-criteria/provincial-criteria/ontario/](http://www.carms.ca/en/residency/r-1/eligibility-criteria/provincial-criteria/ontario/). accessed 2014 Sept. 12.
    
    

14. Human Development Index (HDI). New York: United Nations Development Programme; 2011. Available[http://hdr.undp.org/en/content/human-development-index-hdi](http://hdr.undp.org/en/content/human-development-index-hdi). accessed 2014 Sept. 12.
    
    

15. Canadian Human Rights Act, RSC 1985, c H-6. Available[http://canlii.ca/t/52c3f](http://canlii.ca/t/52c3f). accessed 2017 Sept. 2.
    
    

*   Copyright 2017, Joule Inc. or its licensors