Research
Open Access

Derivation and validation of a clinical risk score to predict death among patients awaiting cardiac surgery in Ontario, Canada: a population-based study

Louise Y. Sun, Harindra C. Wijeysundera, Douglas S. Lee, Sean van Diepen, Marc Ruel, Anan Bader Eddeen and Thierry G. Mesana
March 08, 2022 10 (1) E173-E182; DOI: https://doi.org/10.9778/cmajo.20210031

Abstract

Background: Surgical delay may result in unintended harm to patients needing cardiac surgery, who are at risk for death if their condition is left untreated. Our objective was to derive and internally validate a clinical risk score to predict death among patients awaiting major cardiac surgery.

Methods: We used the CorHealth Ontario Registry and linked ICES health administrative databases with information on all Ontario residents to identify patients aged 18 years or more who were referred for isolated coronary artery bypass grafting (CABG), valvular procedures, combined CABG–valvular procedures or thoracic aorta procedures between Oct. 1, 2008, and Sept. 30, 2019. We used a hybrid modelling approach with the random forest method for initial variable selection, followed by backward stepwise logistic regression modelling for clinical interpretability and parsimony. We internally validated the logistic regression model, termed the CardiOttawa Waitlist Mortality Score, using 200 bootstraps.

Results: Of the 112 266 patients referred for cardiac surgery, 269 (0.2%) died while awaiting surgery (118/72 366 [0.2%] isolated CABG, 81/24 461 [0.3%] valvular procedures, 63/12 046 [0.5%] combined CABG–valvular procedures and 7/3393 [0.2%] thoracic aorta procedures). Age, sex, surgery type, left main stenosis, Canadian Cardiovascular Society classification, left ventricular ejection fraction, heart failure, atrial fibrillation, dialysis, psychosis and operative priority were predictors of waitlist mortality. The model discriminated (C-statistic 0.76 [optimism-corrected 0.73]). It calibrated well in the overall cohort (Hosmer–Lemeshow p = 0.2) and across surgery types.

Interpretation: The CardiOttawa Waitlist Mortality Score is a simple clinical risk model that predicts the likelihood of death while awaiting cardiac surgery. It has the potential to provide data-driven decision support for managing access to cardiac care and preserve system capacity during the COVID-19 pandemic, the recovery period and beyond.

Waitlist management for patients at high risk such as those needing cardiac surgery is an ongoing challenge for clinicians and administrators in Canada and other countries with publicly funded health care systems, where access to these procedures is limited by surgical capacity.1 The cardiac surgery waitlist has grown during the COVID-19 pandemic through lengthening wait times2 and delayed disease presentation owing to missed cardiac specialist visits.3,4 During the first wave of the pandemic, nonemergent cardiac procedures were deferred to ensure that sufficient resources were available to treat patients with COVID-19,2 which created surgical backlogs around the globe.5 As the pandemic evolves, evidence-based criteria are needed to facilitate timely and efficient resource allocation to address this surgical backlog.

The growing backlog of patients with advanced cardiac disease needing surgery creates a dilemma for clinicians and administrators, as these patients require monitoring in the intensive care unit after surgery and may potentially compete with patients with severe COVID-19 for resources. Our group recently developed and externally validated the CardiOttawa LOS Score as an evidence-based decision-support tool to identify high and low users of intensive care unit resources after cardiac surgery.6 However, safe triage decision-making goes beyond knowing patients’ postoperative health care resource needs: it also requires an accurate estimation of the risks they will face in waiting for surgery.

Published waitlist risk models are limited to specific populations, such as patients undergoing coronary artery bypass grafting (CABG).7 The only population-based study of death among patients awaiting cardiac surgery that we were able to identify included a small number of events and was focused on the identification of risk factors rather than the prediction of risk.1 Consequently, current recommendations for waitlist management8 were developed based on expert opinion rather than clinical evidence. Therefore, the objective of this population-based study was to derive and internally validate a clinical model to predict death among patients awaiting cardiac surgery in Ontario, Canada.

Methods

Design, setting and population

We conducted a population-based, retrospective cohort study of adult patients (aged ≥ 18 yr) who were awaiting CABG, valvular surgery, combined CABG–valvular surgery, or surgery on the thoracic aorta in Ontario between Oct. 1, 2008, and Sept. 30, 2019. Ontario is the most populous province in Canada, with roughly 14.6 million residents, and it is one of the most ethnically diverse jurisdictions in the world.9

We excluded patients who were awaiting transcatheter procedures, as well as those requiring rescue procedures, defined as emergent cardiac surgeries to treat acute myocardial ischemia or failed transcatheter interventions.10,11 For patients with multiple cardiac procedures during the study period, only the index procedure was included in the analysis.

Data sources

Using unique confidential identifiers, we deterministically linked the clinical registry of CorHealth Ontario and population-level administrative health care databases available at ICES. ICES is an independent, nonprofit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyze health care and demographic data, without consent, for health system evaluation and improvement. CorHealth Ontario maintains a detailed prospective registry of all patients who undergo invasive cardiac diagnostic and therapeutic procedures in Ontario, including demographic, comorbidity and procedure-related information. CorHealth Ontario data are collected prospectively from the time of surgical referral, and undergo selected chart audits and core laboratory validation.6,12–20

We linked the CorHealth Ontario cardiac registry (detailed surgical referral and waitlist data, date and type of cardiac procedure, physiologic and comorbidity data) with the Canadian Institute for Health Information Discharge Abstract Database (comorbidities, hospital admissions and in-hospital procedures), the Ontario Health Insurance Plan database (physician service claims) and the Registered Persons Database (vital statistics) using unique confidential identifiers.

We chose potential covariates considered in the analyses on the basis of review of the literature on death and complications among patients on the cardiac surgery waitlist,1,7,21,22 as well as the consensus of clinicians on the project team. These covariates included demographic, physiologic (e.g., hemoglobin concentration, serum creatinine level), anatomic (e.g., location of coronary stenosis, valvular function) and comorbidity data, as well as information regarding the proposed procedure (operative priority status, recommended surgical wait time, preoperative cardiogenic shock, redo sternotomy and type of surgery).

As in our previous studies,6,12–20,23–26 we obtained height, weight, operative priority and information pertaining to left ventricular ejection fraction, valvular disease and coronary anatomy from the CorHealth Ontario registry. In addition, we identified comorbidities from the CorHealth Ontario registry; when these data were missing, we supplemented them with data within 5 years before the index procedure from the Discharge Abstract Database and Ontario Health Insurance Plan using International Statistical Classification of Diseases and Related Health Problems, 10th Revision, enhanced Canadian version codes27 according to validated algorithms (Appendix 1, Supplemental Table S3, available at www.cmajopen.ca/content/10/1/E173/suppl/DC1).28–31

Outcome

The primary outcome was all-cause mortality that occurred between the date of acceptance onto the waitlist and the date of removal from the waitlist.

Statistical analysis

All analyses were performed at ICES. We compared baseline characteristics between survivors and nonsurvivors. We tested differences using the 2-sample t test for normally distributed variables, the Wilcoxon rank-sum test for other continuous variables, and the χ2 test for categoric variables. All tests were 2-sided without adjustment for multiplicity.

We imputed missing values using multiple imputations with fully conditional (Markov chain Monte Carlo) methods.32 Specifically, we used logistic regression modelling to generate 5 imputed data sets using the MI procedure of SAS/STAT software (SAS Institute), where missing values were predicted drawing on all candidate covariates. Each imputation provided a complete data set, reflecting the distributions and correlations between variables. The current recommendation for multiple imputations is to generate 2–10 data sets.33 We routinely generate 5 data sets for our studies to support a good number of imputations while accommodating the computing requirement of our large dataset.

We used a hybrid approach of the random forest method for initial variable selection, followed by stepwise logistic regression for clinical interpretability and parsimony.34,35 Details of the random forest method have been described elsewhere.36–38 In short, we used a bootstrap sample of the data to build each of the classification trees. A random subset of variables was selected at each split, thereby constructing a large collection of decision trees with controlled variation. The trees are left unpruned to minimize bias. Every tree in the forest casts a “vote” for the best classification (survival v. death) for a given observation, and the class receiving the most votes results in the prediction for that specific observation. We first sampled the data set to create an in-bag partition (two-thirds of the derivation sample) to construct the decision tree, and a smaller out-of-bag partition (one-third of the derivation sample) to test the constructed tree to evaluate its performance. The random forest method calculates estimates of variable importance for classification using the permutation variable importance measure,36 which is based on the decrease of classification accuracy when values of a variable in a node of a tree are permuted randomly. Our model was based on 500 classification trees and 6 variables available for splitting at each tree node.

We identified a subset of the top 30 predictor variables out of the 40 candidate variables on the basis of variable importance from the random forest model and incorporated them into a logistic model. We entered the predictor variables into a multivariable backward stepwise logistic regression model based on both clinical and statistical significance, with p < 0.1 for entry and p < 0.05 for retention. The final prediction model was termed the CardiOttawa Waitlist Mortality Score.

We evaluated model discrimination using the C-statistic. We obtained an optimism-corrected C-statistic from 200 bootstrap samples drawn with replacement from the study sample. We reported sensitivity, specificity, positive predictive value and negative predictive value using the cut-point that maximized the sum of sensitivity and specificity (Youden index).39 We assessed calibration using the Hosmer–Lemeshow χ2 statistic and the Brier score40 (a calibration plot comparing observed versus expected mortality rates within deciles of expected risk in the overall cohort), as well as a comparison of observed versus expected mortality rates across different subgroups according to type of surgery.

We used the random Forest package for R version 3.6.3 (R Foundation for Statistical Computing), as well as SAS version 9.4. Statistical significance was defined by a 2-sided p value of < 0.05.

Sensitivity analysis

We conducted 2 post hoc sensitivity analyses to test the robustness of our findings. First, we investigated the procedures for which operative priority was unknown. We found that all of these cases were booked as thoracic aorta surgery. We then imputed the operative priority status for these procedures and followed the modelling approach described above. We then modelled death on the waitlist as a function of time. Specifically, candidate variables were entered into a multivariable backward stepwise Cox proportional hazards model with an entry threshold of p < 0.1 and were retained in the model if they had a significance threshold of < 0.05. We treated adherence to the wait time recommended by the Canadian Cardiovascular Society8 as a time-varying covariate in this model. We assessed model performance at 30 and 90 days, using the C-statistic for discrimination and the Brier score calibration plots of observed versus expected mortality rates within deciles of expected rate in the overall cohort and across each type of surgery for calibration.

Ethics approval

The data set from this study is held securely in coded form at ICES. The use of data was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a research ethics board.16

Results

Of the 112 266 patients referred for cardiac surgery, 269 (0.2%) died while awaiting surgery: 118/72 366 (0.2%) while awaiting isolated CABG, 81/24 461 (0.3%) while awaiting valvular procedures, 63/12 046 (0.5%) while awaiting combined CABG–valvular procedures, and 7/3393 (0.2%) while awaiting thoracic aorta procedures. The median wait time was 13 (interquartile range [IQR] 4–38) days overall and was 7 (IQR 3–26) days for CABG, 32 (IQR 12–62) days for valvular surgery, 21 (IQR 7–46) days for combined CABG–valvular surgery and 35 (IQR 9–64) days for thoracic aorta procedures.

Compared to patients who survived the waitlist period, those who died were older and more likely to have had a high-risk acute coronary syndrome; to have reduced left ventricular ejection fraction, heart failure, aortic or mitral regurgitation warranting operative intervention, severe aortic stenosis, and comorbidities such as diabetes, cerebrovascular disease, peripheral arterial disease, renal and liver dysfunction, anemia and psychosis; to be scheduled for urgent, reoperative valvular or combined CABG–valvular surgery with shorter recommended wait times; and to present with unexpected cardiogenic shock before the scheduled procedure (Table 1).

View this table:
Table 1:

Baseline and operative characteristics of patients awaiting isolated CABG, valvular surgery, combined CABG and valvular surgery, or surgery on the thoracic aorta in Ontario, Oct. 1, 2008, to Sept. 30, 2019, by mortality status

Predictors of waitlist mortality

Left ventricular ejection fraction was missing for 3197 patients (2.8%), preoperative serum creatinine value for 5021 (4.5%), height for 5795 (5.2%) and weight for 5464 (4.9%). No other data were missing.

The error rate of the random forest model, defined as the proportion of times the result is not accurate in the overall sample, was 24%. The resulting top 30 predictor variables are summarized in Appendix 1, Supplemental Figure S1. After we applied stepwise logistic regression to achieve parsimony, the final model consisted of 11 variables (Table 2). Sex, type of surgery, left main coronary artery equivalent disease anatomy and Canadian Cardiovascular Society classification were forced into the model on the basis of clinical importance. Other multivariable predictors of waitlist mortality were age, left ventricular ejection fraction, history of heart failure, atrial fibrillation, dialysis, psychosis and operative priority.

View this table:
Table 2:

Multivariable predictors of death while awaiting surgery

Model performance

The C-statistic of the multivariable model was 0.76. After optimism correction, the C-statistic was 0.73, the Hosmer–Lemeshow χ2 statistic was 10.76 with 8 degrees of freedom (p = 0.2), and the Brier score was 0.0024. Figure 1A shows the observed versus expected waitlist mortality rates according to decile of expected rate. The lowest risk decile had a waitlist mortality rate of 0.018% (95% confidence interval 0.0%–0.045%), and the highest risk decile had a rate of 0.99% (95% confidence interval 0.82%–1.15%). The observed and predicted numbers of waitlist deaths were similar across all probability deciles. The model was highly calibrated across each type of surgery (Figure 1B).

Figure 1:
Figure 1:

(A) Calibration plot of observed versus expected rate of waitlist mortality according to decile of expected rate. (B) Observed versus expected rate of waitlist mortality by type of cardiac surgery. Error bars indicate 95% confidence intervals, obtained through 200 bootstraps. Note: CABG = coronary artery bypass grafting.

The receiver-operating characteristic curve for the CardiOttawa Waitlist Mortality Score is presented in Figure 2. The cut-point that maximized the Youden index on the curve was at a predicted probability of 0.24%, with sensitivity of 69.1%, specificity of 72.1%, positive predictive value of 0.59% and negative predictive value of 99.9%.

Figure 2:
Figure 2:

Receiver-operating characteristic curve of the CardiOttawa Waitlist Mortality Score.

CardiOttawa Waitlist Mortality Score

The β-coefficients for the final model are presented in Table 2.

Sensitivity analysis

The model with imputed unknown operative priority (Appendix 1, Supplemental Table S1) was similar to the original model, with a C-statistic of 0.75 (0.72 after optimism correction), Hosmer–Lemeshow χ2 statistic of 8.00 with 8 degrees of freedom (p = 0.4) and Brier score of 0.0024. The time-to-event model was also similar to the original model (Appendix 1, Supplemental Table S2). The model C-statistic was 0.82 (optimism-corrected, 0.80) at 30 days and 0.78 (optimism-corrected, 0.76) at 90 days. Brier scores at 30 and 90 days were 0.0029 and 0.0061, respectively. The model tends to overestimate in the highest risk decile in the overall cohort (Appendix 1, Supplemental Figure S2) and overpredicts across all types of surgery, especially in those awaiting CABG and combined CABG–valvular procedures (Appendix 1, Supplemental Figure S3).

Interpretation

We developed the CardiOttawa Waitlist Mortality Score to provide triage decision support for patients awaiting cardiac surgery using variables that are readily available at the time of surgical referral. The model was derived and validated in a large, representative population. It discriminates moderately, has excellent calibration across all types of surgery and applies to a broad range of cardiac surgical procedures.

In comparison, Senaratne and colleagues1 found that wait times recommended by the Canadian Cardiovascular Society poorly discriminated waitlist mortality across a similar variety of proposed cardiac procedures (C-statistic = 0.577). A smaller, single-centre Scandinavian study that produced a mortality risk score from 42 deaths among patients awaiting CABG did not report performance metrics for the model.7

The few existing waitlist risk algorithms were single-centre, included small numbers of events, or were tailored to specific populations such as patients awaiting CABG or heart transplantation.7,21,43–45 The only contemporary study of waitlist mortality broadly encompassing major types of cardiac procedures that we were able to identify was limited to risk factor identification.1 The current Canadian benchmarks for cardiac surgery wait times were developed based on expert opinion in 2005 and have limited ability to prevent death among patients on the waitlist.8 In their study investigating 101 deaths among patients awaiting cardiac surgery in Alberta, Canada, Senaratne and colleagues1 found that many patients died within the waitlist time frames recommended by the Canadian Cardiovascular Society. A recent statement from the Canadian Society of Cardiac Surgeons highlighted that “it is critically important that cardiac surgeons ensure the presence of a robust wait-times database at their institutions that captures rates of adverse events in these patients while on the wait list so that decisions around the reallocation of resources may be made in a timely fashion.”46 Despite the need for a data-driven waitlist assessment tool to improve patient care, the triage classification system proposed in this statement had not been tested with real-world data.47

Current recommended wait times are based primarily on anatomic factors such as coronary and valvular heart disease.8 The CardiOttawa predictor variables are consistent with those identified in the literature1,7,21,22 and capture important information on baseline patient factors (demographic characteristics, medical conditions, hemodynamic stability) as well as proposed surgical information, in addition to anatomic factors. Whereas the wait times for cardiac procedures recommended by the Canadian Cardiovascular Society8 are likely too long to ensure patient safety, the CardiOttawa Waitlist Mortality Score has the potential to reduce patient mortality through better risk stratification at the time of referral. The waitlist mortality score could be combined with the CardiOttawa LOS Score6 to identify patients at high risk and enable evidence-based surgical scheduling to optimize postoperative use of intensive care unit resources. The combined risk calculator, termed the CardiOttawa 2.1: COVID Triage Tool, can be accessed at https://cardiottawa.ottawaheart.ca. The caveat that applies to all decision-support tools is pertinent because the CardiOttawa Waitlist Mortality Score is intended to assist the clinician, who should ultimately synthesize the predictive score with clinical judgment in making decisions.

Limitations

Major strengths of our waitlist mortality tool are its derivation from a large, ethnically diverse population, its high degree of calibration across a broad spectrum of cardiac procedures and its suitability for use at the time of surgical referral. As the model is intended to guide waitlist triage decisions, it is important that it be validated in a patient sample that is representative of the population for which the tool will be used.

Because universal drug coverage is available in Ontario only to residents aged 65 years or more (as well as certain other eligible groups), we were not able to include information on prescription medications for all patients in the modelling process. However, medications have not routinely been incorporated in models of cardiac surgical risk to date. In addition, model simplicity is an important element of decision-support tools, and thus it is better to carefully select potential factors rather than incorporate an exhaustive list.

The low event rates within each type of surgery precluded procedure-specific modelling. Nonetheless, a multicentre, omnibus risk model is efficient and practical, as operating time is a shared resource. Certain detailed physiologic measures such as brain natriuretic peptide level were lacking in the databases used. However, brain levels of this hormone are not routinely measured in the perioperative setting. There were advances in transcatheter techniques over the course of the study period. Further research is needed to identify how the advent of these minimally invasive procedures may have influenced the referral process and outcomes.

We performed optimism correction on the logistic regression model alone, and not on the random forest model. As death while awaiting cardiac surgery is rare, we were unable to account for nonlinearity of the continuous variables or to split our cohort into derivation and validation samples. Our model should ideally be validated in the contemporary pandemic era to evaluate its performance in the setting of prolonged wait times.

Conclusion

The CardiOttawa Waitlist Mortality Score is a simple clinical risk model that predicts the likelihood of death among patients awaiting major cardiac surgery. It included a population-based sample and had excellent calibration across all procedure types. It could be validated during the present era of prolonged waitlist times and be combined with the CardiOttawa LOS Score to provide rapid, data-driven decision support for clinicians, hospital administrators and policymakers as they manage access to cardiac care and preserve system capacity during the COVID-19 pandemic, the recovery period and beyond.

Acknowledgment

The authors acknowledge the use of data compiled and provided by the Canadian Institute for Health Information.

Footnotes

  • Competing interests: None declared.

  • This article has been peer reviewed.

  • Contributors: Louise Sun conceived and designed the study and drafted the manuscript. Louise Sun and Anan Bader Eddeen acquired, analyzed and interpreted the data. All of the authors revised the manuscript critically for important intellectual content, approved the final version to be published and agreed to be accountable for all aspects of the work.

  • Funding: This study was supported by an Innovation grant from the Ontario Ministry of Health (MOH). Louise Sun was named National New Investigator by the Heart and Stroke Foundation of Canada and is supported by a Tier 2 Clinical Research Chair in Big Data and Cardiovascular Outcomes at the University of Ottawa. Harindra Wijeysundera is supported by a Phase 2 Clinician-Scientist award from the Heart and Stroke Foundation of Canada, Ontario Office. Douglas Lee is supported by a Mid-Career Investigator Award from the Heart and Stroke Foundation. Marc Ruel and Thierry Mesana are supported by endowed research chairs at the University of Ottawa Heart Institute. This study was also supported by ICES, which is funded by an annual grant from the Ontario MOH and the Ontario Ministry of Long-Term Care.

  • Data sharing: The data set from this study is held securely in coded form at ICES. Although legal data-sharing agreements between ICES and data providers (e.g., health care organizations and government) prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at https://www.ices.on.ca/DAS (email: das{at}ices.on.ca). The full data set creation plan and underlying analytic code are available from the authors on request, with the understanding that the computer programs may rely on coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.

  • Disclaimer: The opinions, results and conclusions reported in this article are those of the authors and are independent from the funding and sources. No endorsement by ICES or the Ontario Ministry of Health (MOH) is intended or should be inferred. Parts of this material are based on data and information compiled and provided by the Canadian Institute for Health Information. The authors acknowledge that the clinical registry data used in this analysis are from participating hospitals through CorHealth Ontario, which serves as an advisory body to the Ontario MOH and is funded by the Ontario MOH.

  • Supplemental information: For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/10/1/E173/suppl/DC1.

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/

References

  1. ↵
    1. Senaratne JM,
    2. Norris CM,
    3. Youngson E,
    4. et al.
    (2019) Variables associated with cardiac surgical waitlist mortality from a population-based cohort. Can J Cardiol 35:61–7.
    OpenUrl
  2. ↵
    1. Ad N,
    2. Luc JGY,
    3. Nguyen TC
    (2021) Cardiac surgery in North America and coronavirus disease 2019 (COVID-19): regional variability in burden and impact. J Thorac Cardiovasc Surg 162:893–903e4.
    OpenUrl
  3. ↵
    1. Lange SJ,
    2. Ritchey MD,
    3. Goodman AB,
    4. et al.
    (2020) Potential indirect effects of the COVID-19 pandemic on use of emergency departments for acute life-threatening conditions — United States, January–May 2020. MMWR Morb Mortal Wkly Rep 69:795–800.
    OpenUrlCrossRefPubMed
  4. ↵
    1. Frankfurter C,
    2. Buchan TA,
    3. Kobulnik J,
    4. et al.
    (2020) Reduced rate of hospital presentations for heart failure during the COVID-19 pandemic in Toronto, Canada. Can J Cardiol 36:1680–4.
    OpenUrlPubMed
  5. ↵
    1. Wang J,
    2. Vahid S,
    3. Eberg M,
    4. et al.
    (2020) Clearing the surgical backlog caused by COVID-19 in Ontario: a time series modelling study. CMAJ 192:E1347–56.
    OpenUrlAbstract/FREE Full Text
  6. ↵
    1. Sun LY,
    2. Bader Eddeen A,
    3. Ruel M,
    4. et al.
    (2020) Derivation and validation of a clinical model to predict intensive care unit length of stay after cardiac surgery. J Am Heart Assoc 9:e017847.
    OpenUrl
  7. ↵
    1. Rexius H,
    2. Brandrup-Wognsen G,
    3. Odén A,
    4. et al.
    (2004) Mortality on the waiting list for coronary artery bypass grafting: incidence and risk factors. Ann Thorac Surg 77:769–74, discussion 74–5.
    OpenUrlCrossRefPubMed
  8. ↵
    1. CCS Access to Care Working Group
    (2005) Canadian Wait Time Alliance, Wait-time benchmarks for cardiovascular services and procedures. Available: https://www.waittimealliance.ca/wp-content/uploads/2014/05/Cardiovasuclar_Services_and_Procedures.pdf. accessed 2020 Nov. 10.
  9. ↵
    1. Sun LY,
    2. Eddeen AB,
    3. Mesana TG
    (2021) Disability-free survival after major cardiac surgery: a population-based retrospective cohort study. CMAJ Open 9:E384–93.
    OpenUrlAbstract/FREE Full Text
  10. ↵
    1. Shahriari A,
    2. Eng M,
    3. Tranquilli M,
    4. et al.
    (2009) Rescue coronary artery bypass grafting (CABG) after aortic composite graft replacement. J Card Surg 24:392–6.
    OpenUrlCrossRefPubMed
  11. ↵
    1. Menaissy Y,
    2. Alkady H,
    3. El-Saiedi S
    (2019) Rescue cardiac surgeries after pediatric catheter-based interventions: a ten-year retrospective study. World J Pediatr Congenit Heart Surg 10:539–42.
    OpenUrl
  12. ↵
    1. Hayatsu Y,
    2. Ruel M,
    3. Bader Eddeen A,
    4. et al.
    (June 24, 2020) Single versus multiple arterial revascularization in patients with reduced renal function: long-term outcome comparisons in 23 406 CABG patients from Ontario, Canada. Ann Surg doi:10.1097/SLA.0000000000003908, [Epub ahead of print].
    OpenUrlCrossRef
    1. Johnston A,
    2. Mesana TG,
    3. Lee DS,
    4. et al.
    (2019) Sex differences in long-term survival after major cardiac surgery: a population-based cohort study. J Am Heart Assoc 8:e013260.
    OpenUrlPubMed
    1. Rubens FD,
    2. Wells GA,
    3. Coutinho T,
    4. et al.
    (2022) Sex differences after coronary artery bypass grafting with a second arterial conduit. J Thorac Cardiovasc Surg 163:686–95.e10.
    OpenUrl
    1. Sun LY,
    2. Gaudino M,
    3. Chen RJ,
    4. et al.
    (2020) Long-term outcomes in patients with severely reduced left ventricular ejection fraction undergoing percutaneous coronary intervention v. coronary artery bypass grafting. JAMA Cardiol 5:631–41.
    OpenUrl
  16. ↵
    1. Sun LY,
    2. Spence SD,
    3. Benton S,
    4. et al.
    (June 11, 2020) Age, not sex, modifies the effect of frailty on long-term outcomes after cardiac surgery. Ann Surg doi:10.1097/SLA.0000000000004060, [Epub ahead of print].
    OpenUrlCrossRef
    1. Sun LY,
    2. Tu JV,
    3. Bader Eddeen A,
    4. et al.
    (2018) Prevalence and long-term survival after coronary artery bypass grafting in women and men with heart failure and preserved versus reduced ejection fraction. J Am Heart Assoc 7:e008902.
    OpenUrlAbstract/FREE Full Text
    1. Sun LY,
    2. Tu JV,
    3. Lee DS,
    4. et al.
    (2018) Disability-free survival after coronary artery bypass grafting in women and men with heart failure. Open Heart 5:e000911.
    OpenUrlAbstract/FREE Full Text
    1. Tam DY,
    2. Dharma C,
    3. Rocha R,
    4. et al.
    (2020) Long-term survival after surgical or percutaneous revascularization in patients with diabetes and multivessel coronary disease. J Am Coll Cardiol 76:1153–64.
    OpenUrlFREE Full Text
  20. ↵
    1. Tran DTT,
    2. Tu JV,
    3. Dupuis JY,
    4. et al.
    (2018) Association of frailty and long-term survival in patients undergoing coronary artery bypass grafting. J Am Heart Assoc 7:e009882.
    OpenUrl
  21. ↵
    1. Morgan CD,
    2. Sykora K,
    3. Naylor CD
    (1998) Analysis of deaths while waiting for cardiac surgery among 29,293 consecutive patients in Ontario, Canada. The Steering Committee of the Cardiac Care Network of Ontario. Heart 79:345–9.
    OpenUrlAbstract/FREE Full Text
  22. ↵
    1. Jasseron C,
    2. Legeai C,
    3. Jacquelinet C,
    4. et al.
    (2017) Prediction of waitlist mortality in adult heart transplant candidates: the candidate risk score. Transplantation 101:2175–82.
    OpenUrl
  23. ↵
    1. Sun LY,
    2. Gershon AS,
    3. Ko DT,
    4. et al.
    (2015) Trends in pulmonary function testing before noncardiothoracic surgery. JAMA Intern Med 175:1410–2.
    OpenUrl
    1. Sun LY,
    2. Kimmoun A,
    3. Takagi K,
    4. et al.
    (2019) Ethnic differences in acute heart failure outcomes in Ontario. Int J Cardiol 291:177–82.
    OpenUrlPubMed
    1. Sun LY,
    2. Tu JV,
    3. Coutinho T,
    4. et al.
    (2018) Sex differences in outcomes of heart failure in an ambulatory, population-based cohort from 2009 to 2013. CMAJ 190:E848–54.
    OpenUrlAbstract/FREE Full Text
  26. ↵
    1. Sun LY,
    2. Tu JV,
    3. Sherrard H,
    4. et al.
    (2018) Sex-specific trends in incidence and mortality for urban and rural ambulatory patients with heart failure in Eastern Ontario from 1994 to 2013. J Card Fail 24:568–74.
    OpenUrlPubMed
  27. ↵
    1. Quan H,
    2. Sundararajan V,
    3. Halfon P,
    4. et al.
    (2005) Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 43:1130–9.
    OpenUrlCrossRefPubMed
  28. ↵
    1. Tu K,
    2. Campbell N,
    3. Chen Z,
    4. et al.
    (2007) Accuracy of administrative databases in identifying patients with hypertension. Open Med 1:e18–26.
    OpenUrlPubMed
    1. Hux JE,
    2. Ivis F,
    3. Flintoft V,
    4. et al.
    (2002) Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care 25:512–6.
    OpenUrlAbstract/FREE Full Text
    1. Gershon AS,
    2. Wang C,
    3. Guan J,
    4. et al.
    (2009) Identifying individuals with physician diagnosed COPD in health administrative databases. COPD 6:388–94.
    OpenUrlCrossRefPubMed
  31. ↵
    1. Schultz SE,
    2. Rothwell D,
    3. Chen Z,
    4. et al.
    (2013) Identifying cases of congestive heart failure from administrative data: a validation study using primary care patient records. Chronic Dis Inj Can 33:160–6.
    OpenUrlCrossRefPubMed
  32. ↵
    1. Sterne JA,
    2. White IR,
    3. Carlin JB,
    4. et al.
    (2009) Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ 338:b2393.
    OpenUrlFREE Full Text
  33. ↵
    1. van Buuren S
    (2018) Flexible imputation of missing data, 2nd ed Chapter 2: Multiple imputation Interdisciplinary statistics series (CRC Press, Boca Raton (FL)).
  34. ↵
    1. Lertampaiporn S,
    2. Thammarongtham C,
    3. Nukoolkit C,
    4. et al.
    (2014) Identification of non-coding RNAs with a new composite feature in the Hybrid Random Forest Ensemble algorithm. Nucleic Acids Res 42:e93.
    OpenUrlCrossRefPubMed
  35. ↵
    1. Petrosyan Y,
    2. Thavorn K,
    3. Smith G,
    4. et al.
    (2021) Predicting postoperative surgical site infection with administrative data: a random forests algorithm. BMC Med Res Methodol 21:179.
    OpenUrl
  36. ↵
    1. Breiman L
    (2001) Random forests. Mach Learn 45:5–32.
    OpenUrlCrossRefPubMed
    1. Liam A,
    2. Wiener M
    (2002) Classification and regression by random forest. R News 2:315–26.
    OpenUrl
  38. ↵
    1. Touw WG,
    2. Bayjanov JR,
    3. Overmars L,
    4. et al.
    (2013) Data mining in the life sciences with random forest: A walk in the park or lost in the jungle? Brief Bioinform 14:315–26.
    OpenUrlCrossRefPubMed
  39. ↵
    1. Ruopp MD,
    2. Perkins NJ,
    3. Whitcomb BW,
    4. et al.
    (2008) Youden Index and optimal cut-point estimated from observations affected by a lower limit of detection. Biom J 50:419–30.
    OpenUrlCrossRefPubMed
  40. ↵
    1. Harrell FE Jr.,
    2. Lee KL,
    3. Mark DB
    (1996) Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15:361–87.
    OpenUrlCrossRefPubMed
  41. (Sept 23, 1976) (2015) Canadian Cardiovascular Society grading of angina pectoris (Canadian Cardiovascular Society), Archived from original (PDF), Available: https://web.archive.org/web/20150923201210/http://www.ccs.ca/images/Guidelines/Guidelines_POS_Library/Ang_Gui_1976.pdf. accessed 2016 Nov 12.
    1. Raphael C,
    2. Briscoe C,
    3. Davies J,
    4. et al.
    (2007) Limitations of the New York Heart Association functional classification system and self-reported walking distances in chronic heart failure. Heart 93:476–82.
    OpenUrlAbstract/FREE Full Text
  43. ↵
    1. Koomen EM,
    2. Hutten BA,
    3. Kelder JC,
    4. et al.
    (2001) Morbidity and mortality in patients waiting for coronary artery bypass surgery. Eur J Cardiothorac Surg 19:260–5.
    OpenUrlCrossRefPubMed
    1. da Fonseca VBP,
    2. De Lorenzo A,
    3. Tura BR,
    4. et al.
    (2018) Mortality and morbidity of patients on the waiting list for coronary artery bypass graft surgery. Interact Cardiovasc Thorac Surg 26:34–40.
    OpenUrlPubMed
  45. ↵
    1. Cesena FHY,
    2. Favarato D,
    3. Cesar LAM,
    4. et al.
    (2004) Cardiac complications during waiting for elective coronary artery bypass graft surgery: incidence, temporal distribution and predictive factors. Eur J Cardiothorac Surg 25:196–202.
    OpenUrlCrossRefPubMed
  46. ↵
    1. Hassan A,
    2. Arora RC,
    3. Adams C,
    4. et al.
    (2020) Cardiac surgery in Canada during the COVID-19 pandemic: a guidance statement from the Canadian Society of Cardiac Surgeons. Can J Cardiol 36:952–5.
    OpenUrl
  47. ↵
    1. Patel V,
    2. Jimenez E,
    3. Cornwell L,
    4. et al.
    (2020) Cardiac surgery during the coronavirus disease 2019 pandemic: perioperative considerations and triage recommendations. J Am Heart Assoc 9:e017042.
    OpenUrl
Back to top

In this issue

CMAJ Open: 10 (1)
Vol. 10, Issue 1
1 Jan 2022

Article tools

Respond to this article
Print
Download PDF
Article Alerts
Email Article
Citation Tools
Derivation and validation of a clinical risk score to predict death among patients awaiting cardiac surgery in Ontario, Canada: a population-based study
Louise Y. Sun, Harindra C. Wijeysundera, Douglas S. Lee, Sean van Diepen, Marc Ruel, Anan Bader Eddeen, Thierry G. Mesana
Jan 2022, 10 (1) E173-E182; DOI: 10.9778/cmajo.20210031
Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo

Related Articles

Cited By...

  • No citing articles found.

Similar Articles

Collections