Bleeding impacting mortality after noncardiac surgery: a protocol to establish diagnostic criteria, estimate prognostic importance, and develop and validate a prediction guide in an international prospective cohort study
============================================================================================================================================================================================================================

* Pavel S. Roshanov
* John W. Eikelboom
* Mark Crowther
* Vikas Tandon
* Flavia K. Borges
* Clive Kearon
* Andre Lamy
* Richard Whitlock
* Bruce M. Biccard
* Wojciech Szczeklik
* Gordon H. Guyatt
* Mohamed Panju
* Jessica Spence
* Amit X. Garg
* Michael McGillion
* Tomas VanHelder
* Peter A. Kavsak
* Justin de Beer
* Mitchell Winemaker
* Daniel I. Sessler
* Yannick Le Manach
* Tej Sheth
* Jehonathan H. Pinthus
* Lehana Thabane
* Marko R.I. Simunovic
* Ryszard Mizera
* Sebastian Ribas
* P.J. Devereaux
* the Vascular Events In Noncardiac Surgery Patients Cohort Evaluation (VISION)
* Investigators

## Abstract

**Introduction:** Various definitions of bleeding have been used in perioperative studies without systematic assessment of the diagnostic criteria for their independent association with outcomes important to patients. Our proposed definition of bleeding impacting mortality after noncardiac surgery (BIMS) is bleeding that is independently associated with death during or within 30 days after noncardiac surgery. We describe our analysis plan to sequentially 1) establish the diagnostic criteria for BIMS, 2) estimate the independent contribution of BIMS to 30-day mortality and 3) develop and internally validate a clinical prediction guide to estimate patient-specific risk of BIMS.

**Methods:** In the Vascular Events In Noncardiac Surgery Patients Cohort Evaluation (VISION) study, we prospectively collected bleeding data for 16 079 patients aged 45 years or more who had noncardiac inpatient surgery between 2007 and 2011 at 12 centres in 8 countries across 5 continents. We will include bleeding features independently associated with 30-day mortality in the diagnostic criteria for BIMS. Candidate features will include the need for reoperation due to bleeding, the number of units of erythrocytes transfused, the lowest postoperative hemoglobin concentration, and the absolute and relative decrements in hemoglobin concentration from the preoperative value. We will then estimate the incidence of BIMS and its independent association with 30-day mortality. Last, we will construct and internally validate a clinical prediction guide for BIMS.

**Interpretation:** This study will address an important gap in our knowledge about perioperative bleeding, with implications for the 200 million patients who undergo noncardiac surgery globally every year. **Trial registration:** ClinicalTrials.gov, no [NCT00512109](http://www.cmajopen.ca/lookup/external-ref?link_type=CLINTRIALGOV&access_num=NCT00512109&atom=%2Fcmajo%2F5%2F3%2FE594.atom).

More than 200 million people undergo major surgery worldwide each year.1 Prior studies have associated perioperative bleeding with higher risk of postoperative death and complications, longer hospital stay and higher health care costs.2-4 Various definitions of bleeding have been used.5,6 Consensus definitions were developed without systematic assessment of the diagnostic criteria for their independent association with poor patient outcomes.7

There is value in establishing diagnostic criteria for bleeding impacting mortality after noncardiac surgery (BIMS). Our proposed definition of BIMS is bleeding that independently increases patients' 30-day probability of death and occurs during or within 30 days after noncardiac surgery. In this methods paper, we report our plan for analysis of data from the Vascular Events in Noncardiac Surgery Patients Cohort Evaluation (VISION) study to, in the following order, 1) determine the diagnostic criteria for BIMS, 2) estimate its incidence, prognostic impact and population attributable risk fraction with respect to 30-day mortality and 3) develop and validate a clinical prediction guide to estimate patient-specific risk of BIMS.

## Methods

Figure 1 summarizes the flow of participants through the study. Figure 2 summarizes the methods described in this protocol. We will use Stata/MP version 13.1 (StataCorp) and R version 3.3 (R Development Core Team) with the rms package8 for all analyses.

![Figure 1](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/5/3/E594/F1.medium.gif)

[Figure 1](http://www.cmajopen.ca/content/5/3/E594/F1)

Figure 1 
Participant flow. Note: BIMS = bleeding impacting mortality after noncardiac surgery, MINS = myocardial injury after noncardiac surgery, VISION = Vascular Events In Patients Undergoing Noncardiac Surgery Cohort Evaluation.

![Figure 2](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/5/3/E594/F2.medium.gif)

[Figure 2](http://www.cmajopen.ca/content/5/3/E594/F2)

Figure 2 
Summary of analysis plan to address study objectives sequentially. Note: BIMS = bleeding impacting mortality in noncardiac surgery, MINS = myocardial injury after noncardiac surgery, VISION = Vascular Events In Patients Undergoing Noncardiac Surgery Cohort Evaluation.

### Study design

We will analyze data from the VISION study, a prospective international cohort study that included 16 079 patients from 12 centres in 8 countries (throughout North and South America, Australia, Asia and Europe) recruited from August 2007 to January 2011 ([ClinicalTrials.gov](http://ClinicalTrials.gov) [NCT00512109](http://www.cmajopen.ca/lookup/external-ref?link_type=CLINTRIALGOV&access_num=NCT00512109&atom=%2Fcmajo%2F5%2F3%2FE594.atom)). VISION enrolment and data collection have been described in previous reports.9-11 Briefly, eligible participants aged 45 years or more undergoing noncardiac inpatient surgery (i.e., with planned overnight stay) answered a series of questions regarding their past medical, surgical and social history. Study personnel reviewed the participants' medical charts for additional history. Throughout each participant's hospital stay, research personnel performed clinical evaluations, reviewed medical records and noted outcome events. A follow-up telephone interview was conducted with the participant or his or her caregiver 30 days after surgery. If the interview indicated the occurrence of an outcome, the primary care physician was contacted to obtain further documentation.

Data monitoring involved central data consistency checks, statistical monitoring and on-site monitoring for all centres. For on-site monitoring, the central coordinator randomly selected participants with and without a perioperative complication, and an on-site monitor then audited the participant's medical records and all supporting documents.

### Sample size and completeness of study data

We completed 30-day follow-up for 16 026 (99.7%) of 16 079 patients; the remaining 53 patients did not die within 30 days of surgery, and their data were censored at the time of hospital discharge. The protocol is divided into 3 objectives to be addressed sequentially in the analysis. The sample size and missing data are given separately for each objective in Figure 1. Where specified, we will impute missing data using single stochastic conditional imputation with predictive mean matching12 for continuous variables and augmented logistic regression13 for binary variables, both with fully conditional specification.14 Box 1 lists the variables to be included in the imputation model. Single stochastic imputation is much more practical for our analysis than multiple imputation, and with few missing data, its drawback - slightly more narrow confidence intervals - will be negligible.15 Pertinent cohort characteristics are shown in Appendix 1, Supplementary Table 1, available at [www.cmajopen.ca/content/5/3/E594/suppl/DC1.](http://www.cmajopen.ca/content/5/3/E594/suppl/DC1.)

Box 1: 
### Variables included in imputation model

Age, sex, age-by-sex interaction, preoperative weight, height, preoperative serum creatinine level, preoperative hemoglobin concentration, active cancer, preoperative dialysis, required assistance with activities of daily living preoperatively, endoscopic surgical technique, open surgical technique, duration of surgery, history of chronic obstructive pulmonary disease, history of coronary artery disease (not recent high-risk coronary artery disease), history of recent high-risk coronary artery disease, history of diabetes not requiring insulin preoperatively, history of diabetes requiring insulin preoperatively, history of congestive heart failure, history of transient ischemic attack, history of stroke, history of hypertension, peripheral vascular disease, any use of acetylsalicylic acid within 7 d before surgery, any cyclooxygenase-2 inhibitor use within 7 d before surgery, any use of other nonsteroidal anti-inflammatory medication within 7 d before surgery, any use of angiotensin II receptor blocker or angiotensin-converting-enzyme inhibitor within 7 d before surgery, any β-blocker use within 7 d before surgery, any oral anticoagulant treatment within 7 d before surgery, any prophylactic subcutaneous treatment with an antithrombotic agent within 7 d before surgery, any subcutaneous or intravenous treatment with a therapeutic antithrombotic agent within 7 d before surgery, any clopidogrel or ticlopidine use within 7 d before surgery, any statin use within 7 d before surgery, any α2 agonist use within 7 d before surgery, any use of a rate-controlling calcium-channel blocker within 7 d before surgery, any use of a dihydropyridine calcium-channel blocker within 7 d before surgery, any use of a nonstatin cholesterol-lowering medication within 7 d before surgery, any long-acting nitrate use within 7 d before surgery, thoracic aorta reconstructive vascular surgery, aortoiliac reconstructive vascular surgery, peripheral vascular reconstruction without aortic cross-clamping, extracranial cerebrovascular surgery, endovascular abdominal aortic aneurysm repair, complex visceral resection general surgery, partial or total colectomy or stomach surgery, other intra-abdominal surgery, major head and neck resection for nonthyroid tumour, pneumonectomy, lobectomy, other thoracic surgeries, visceral urologic or gynecologic resection, cytoreductive surgery, hysterectomy, radical hysterectomy, radical prostatectomy, transurethral prostatectomy, major hip or pelvic surgery, internal fixation of femur, knee arthroplasty, above-knee amputation, lower leg amputation (amputation below knee but above foot), craniotomy, major spine surgery involving multiple levels of the spine, myocardial injury after noncardiac surgery, stroke within 30 d of surgery, pulmonary embolus within 30 d of surgery, sepsis within 30 d of surgery, death within 30 d of surgery, number of units of erythrocytes transfused, reoperation for reasons of bleeding, postoperative hemoglobin nadir, study centre, calendar year of surgery.

**Definitions**

**Preoperative variables**

**• Age:** Patient's age in years, calculated as the difference between birthdate and date of surgery and rounded down to nearest year.

**• Preoperative hemoglobin level:** Latest available routinely measured preoperative hemoglobin value.

**• Preoperative estimated glomerular filtration rate:** Calculated with the Chronic Kidney Disease Epidemiology Collaboration equation; latest available routinely measured preoperative serum creatinine value.

**• Requires assistance with activities of daily living**: Patient requires assistance from another person with any of the following activities: dressing, eating, ambulating, toileting or hygiene. If a patient has incurred an acute injury leading to the need for surgery (e.g., hip fracture), the assessment for requirement of help for activities of daily living is based on his or her condition before the acute injury.

**• Congestive heart failure:** Physician diagnosis of a current or prior episode of congestive heart failure or prior radiographic evidence of vascular redistribution, interstitial pulmonary edema or frank alveolar pulmonary edema.

**• Recent high-risk coronary artery disease:** Diagnosis ≤ 6 mo before noncardiac surgery of myocardial infarction, acute coronary syndrome, Canadian Cardiovascular Society grade III angina (angina occurring with level walking of 1-2 blocks or climbing 1 flight of stairs at a normal pace) or grade IV angina (inability to perform any physical activity without the development of angina).

**• Cerebral vascular event:** Physician diagnosis of stroke, computed tomography (CT) or magnetic resonance imaging evidence of a prior stroke, or physician diagnosis of a prior transient ischemic attack.

**• Peripheral vascular disease:** Current or prior history of physician-diagnosed intermittent claudication, vascular surgery for atherosclerotic disease, an ankle-arm systolic blood pressure ratio ≤ 0.90 in either leg at rest, or angiographic or Doppler study showing ≥ 70% stenosis in a noncardiac artery.

**• Chronic obstructive pulmonary disease (COPD):** If the chart or a physician had ever indicated that a patient has chronic bronchitis, we accepted this as a patient's having COPD. If there was no mention of this but the patient reported that he or she had daily production of sputum for at least 3 mo in 2 consecutive yr, the patient was classified as having COPD. Likewise, if a physician had ever indicated that a patient had emphysema or if a patient's pulmonary function tests stated fixed or irreversible airflow limitation and/or emphysema, the patient was classified as having COPD.

**• Active cancer:** Patient had a current diagnosis of cancer or was undergoing surgery for cancer.

**Surgical variables**

If the patient underwent more than 1 operation, all operations performed were included.

**• Major vascular surgery**

1. Thoracic aorta reconstructive vascular procedures (thoracic aortic aneurysm repair, repair of supra-aortic trunks not requiring total cardiopulmonary bypass, thoracoabdominal aortic aneurism repair with or without aortofemoral bypass)

2. Aortoiliac reconstructive vascular surgery (open abdominal aortic aneurysm repair, aortofemoral bypass, iliac-femoral bypass, renal artery revascularization, celiac artery revascularization, superior mesenteric artery revascularization)

3. Peripheral vascular reconstruction without aortic cross-clamping (axillofemoral bypass, femorofemoral bypass, femoroinfragenicular bypass, profundoplasty, other angioplasty of the infrainguinal arteries)

4. Extracranial cerebrovascular surgery (carotid endarterectomy, carotid-subclavian bypass)

5. Endovascular abdominal aortic aneurysm repair.

**• Major general surgery**

1. Complex visceral resection (surgery involving the liver, esophagus, pancreas or multiple organs)

2. Partial or total colectomy or stomach surgery

3. Other intra-abdominal surgery (gallbladder, appendix, adrenal gland, spleen, regional lymph node dissection)

4. Major head and neck resection for nonthyroid tumour.

• **Thoracic surgery**

1. Pneumonectomy

2. Lobectomy

3. Other thoracic surgical procedures (wedge resection of lung, resection of mediastinal tumour, major chest wall resection).

**• Major urologic or gynecologic surgery**

1. Visceral resection (nephrectomy, ureterectomy, bladder resection, retroperitoneal tumour resection, exenteration [i.e., radical procedure for cancer to remove pelvic organs])

2. Cytoreductive surgery "debulking" done when cancer has spread in the pelvic/abdominal area, to remove as much of the tumour as possible

3. Radical hysterectomy: Surgery to remove the uterus, cervix and part of the vagina

4. Hysterectomy: Surgery to remove the uterus and usually the cervix

5. Radical prostatectomy: Surgery to remove the entire prostate gland and surrounding tissue

6. Transurethral prostatectomy: Surgery to remove overgrowth of prostate tissue.

**• Major orthopedic surgery**

1. Major hip or pelvic surgery (hemi or total hip arthroplasty, internal fixation of hip, pelvic arthroplasty)

2. Internal fixation of femur

3. Knee arthroplasty

4. Above-knee amputation

5. Lower leg amputation (amputation below knee but above foot).

• **Major neurosurgery**

1. Craniotomy

2. Major spine surgery: Surgery involving multiple levels of the spine.

**• ****Low-risk surgery**: Any parathyroid, thyroid, breast, hernia or local anorectal procedure, oopherectomy, salpingectomy, endometrial ablation, peripheral nerve surgery, ophthalmologic surgery, ear/nose/throat surgery, vertebral disc surgery, hand surgery, cosmetic surgery, arteriovenous access surgery for dialysis, other procedures.

**• ****Urgent or emergency surgical procedures:** Procedures performed within 72 hours of acute event that led to the need for surgery.

**• ****Duration of surgery:** Time elapsed, in minutes, between when the surgeon began the procedure and when he or she closed the wound.

• **Surgical techniques**

1. Endoscopic or open surgery: Can be categorized as both endoscopic and open if surgery started endoscopically and finished open

2. Endoscopic techniques: Include all endoscopic, laparoscopic, thoracoscopic, endovascular and arthroscopic techniques.

**Postoperative complications**

**• Bleeding:** Defined as bleeding that results in a decrease in hemoglobin level of 30 g/L (3 g/dL) or more, leads to transfusion or reoperation, or is thought to be the cause of death.

**• Myocardial injury after noncardiac surgery:** Any peak cardiac troponin T level ≥ 0.03 ng/mL resulting from myocardial ischemia (i.e., without evidence of a nonischemic cause) that occurred with the first 30 d after surgery.10 We measured non**-**high-sensitivity cardiac troponin T levels using a fourth-generation Elecsys assay (Roche Diagnostics) 6-12 h postoperatively and during the first 3 d after surgery to look for myocardial injury.

**• Stroke:** New focal neurologic deficit thought to be vascular in origin with signs and symptoms lasting more than 24 h.

**• Pulmonary embolus:** The diagnosis of pulmonary embolus required any 1 of the following:

1. A high-probability ventilation/perfusion lung scan

2. An intraluminal filling defect of the segmental or larger artery on a helical CT scan

3. An intraluminal filling defect on pulmonary angiography

4. A positive diagnostic test result for deep vein thrombosis (e.g., compression ultrasonography) and 1 of the following:

A. Nondiagnostic (i.e., low- or intermediate-probability) ventilation/perfusion lung scan

B. Nondiagnostic (i.e., subsegmental defects or technically inadequate study) helical CT scan.

**• Sepsis:** A clinical syndrome defined by the presence of both infection and a systemic inflammatory response. Defined as a pathological process caused by the invasion of normally sterile tissue or fluid or body cavity by pathogenic or potentially pathogenic organisms. Systemic inflammatory response requires 2 or more of the following factors: core temperature > 38°C or < 36°C, heart rate > 90/min, respiratory rate > 20/min, leukocyte count > 12 × 109/L or < 4 × 109/L.

### Objective 1: establish diagnostic criteria for BIMS

We will restrict the analysis for objective 1 to 5476 patients who experienced a bleeding event to better protect against residual confounding and time-dependent biases.16 Of the 5476, 167 died within 30 days of surgery. In the VISION study, bleeding was defined broadly to avoid missing prognostically important bleeding events. The definition included all bleeding that resulted in a decrease in hemoglobin concentration of at least 30 g/L, led to a transfusion of blood products or reoperation, or was thought to be the immediate cause of death. If a patient experienced more than 1 bleeding episode during the first 30 days after surgery, we will evaluate only the first episode in all analyses.

The diagnostic criteria for BIMS should identify, among people who experience a bleeding event, as many patients as possible who will die as a consequence of the bleeding within 30 days of surgery and should exclude as many patients as possible who will not die within this period.

We will consider 5 candidate features for inclusion in the diagnostic criteria, in the following order: 1) reoperation for reasons of bleeding, 2) number of units of erythrocytes transfused, 3) the lowest (nadir) postoperative hemoglobin concentration associated with the bleeding, 4) the absolute decrease in hemoglobin concentration from the preoperative value and 5) the relative decrease in hemoglobin concentration from the preoperative value. The features that are the least subjective and easiest to ascertain will be tested first, to ensure that they have a greater chance of becoming part of the BIMS diagnostic criteria compared to less practical correlated candidate features that are similarly associated with mortality. In the VISION study, bleeding was suspected by the clinical team to be the direct cause of death in some patients; this feature will be included in the diagnostic criteria without statistical testing.

We will model the association between 30-day mortality and candidate BIMS criteria using shared (by study centre) frailty multivariable Cox proportional hazards regression models adjusted for preoperative patient characteristics, surgical factors (type and timing of surgery) and other postoperative complications (Table 1). We selected these adjustment variables on the basis of previous VISION work that identified variables independently associated with death among all patients, with the assumption that the same factors are associated with death in patients who have experienced a bleeding event.10 We will adjust for patients' requirement of assistance with activities of daily living because functional status has been associated with death in prior studies.17-19

View this table:
[Table 1:](http://www.cmajopen.ca/content/5/3/E594/T1)

Table 1:   Hierarchy for entry of candidate features of bleeding impacting mortality in noncardiac surgery into regression model

We will also adjust for postoperative complications including sepsis, pulmonary embolism, stroke and myocardial injury after noncardiac surgery (MINS)10 that occurred on a day before the day of a bleeding event, but not those that occurred on the same day or in the days after the bleeding event because BIMS may cause these complications directly (e.g., MINS due to supply-demand mismatch from a low hemoglobin concentration) or indirectly (e.g., pulmonary embolism due to BIMS that resulted in the withdrawal of anticoagulant treatment; sepsis through prolongation of hospital stay and exposure to additional interventions). Adjusting for complications that BIMS may have caused would underestimate the association between a candidate feature and death.20

Figure 3 summarizes the algorithm for selecting diagnostic criteria for BIMS. This is an iterative process that begins with a baseline model in which the explanatory variables include only the adjustment variables. Candidate features are added to the baseline model, 1 at a time, in the order described in Table 1. We will test the statistical significance of the first feature (adjusted for the other variables in the model) using a likelihood ratio test. If this test produces a *p* value ≥ 0.05, we will consider the candidate feature not to be an independent predictor of death, and it will no longer be considered for inclusion in the criteria. The next candidate feature will replace it and will be tested in the same way. If the test produces a *p* value < 0.05, the candidate feature will be considered a proven independent predictor of death and will be retained in the model. When subsequent candidate features are tested, they will be compared with the model that contains already-proven features and the adjustment variables.

![Figure 3](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/5/3/E594/F3.medium.gif)

[Figure 3](http://www.cmajopen.ca/content/5/3/E594/F3)

Figure 3 
Summary of algorithm for selecting diagnostic criteria for bleeding impacting mortality in noncardiac surgery. Candidate features will be tested for association with 30-day mortality in shared frailty Cox proportional hazards model, adjusted for potential pre-, intra- and postoperative confounders. The threshold for retaining a candidate feature in the model is *p* < 0.05 from a likelihood ratio test comparing the model with the feature to a model without it.

To simplify integration of continuous variables (e.g., number of units of erythrocytes transfused) into the diagnostic criteria, these variables will be dichotomized at thresholds according to Table 1, and a dichotomous version representing each threshold will be tested in the model. We will select the threshold that returns the highest χ2 statistic from the likelihood ratio test for inclusion in the diagnostic criteria, as long as *p* < 0.05 for that threshold. If *p* ≥ 0.05, we will reject the entire variable, as it was not related to death at any dichotomization threshold. The process will continue until all candidate features have been tested.

We will then join the retained features with a series of "or" statements along with "bleeding thought to be the cause of death" (which will not be subjected to the selection process). This series will form the BIMS diagnostic criteria.

#### Sensitivity analysis

We will repeat the analysis for objective 1 by additionally adjusting for major vascular surgery, thoracic surgery, orthopedic surgery, and major urologic or gynecologic surgery.

### Objective 2: estimate prognostic value of BIMS

We will perform this analysis in all 15 109 patients with available data for MINS. Of the 15 109, 268 died within 30 days of surgery. We will categorize patients as having experienced BIMS, non-BIMS bleeding or no bleeding. We will estimate the association between BIMS and death in a shared frailty Cox proportional hazards model. BIMS will enter the model as a time-varying covariate. We will adjust the model for the same adjustment variables used in the process to select candidate features, except that in this model we will also adjust (as time-varying covariates) for MINS, sepsis, pulmonary embolism and stroke. To aid in the interpretation of the results, we will estimate the proportion of deaths potentially attributable to BIMS and all other statistically significant variables (i.e., the population attributable risk fraction) with corresponding 95% confidence intervals. We will repeat this analysis without adjustment for MINS, sepsis, pulmonary embolism or stroke because for many patients these complications may be the direct result of BIMS or its management. Comparing population attributable risk fractions adjusted and unadjusted for these complications will provide a minimum and maximum estimate of the potential independent contribution of BIMS to death.

#### Subgroup analyses

We will estimate the incidence and prognostic value of BIMS with respect to death in subgroups defined by age less than 75 versus 75 years or more, preoperative hemoglobin level less than 120 g/L versus 120 g/L or higher, men versus women and known cardiovascular disease versus no cardiovascular disease. We will interpret a subgroup effect as significant if the effect BIMS is associated with death in 1 of the subgroups but not in the other and if a statistical test of interaction shows a *p* value < 0.01. We use this stringent *p* value for interaction to protect against spurious findings in subgroups with few events. We additionally require that BIMS be associated with death in 1 of the subgroups but not in the other because a weaker association of BIMS with death would still satisfy the definitional requirement that BIMS be positively associated with death.

### Objective 3: develop and internally validate clinical prediction guide to predict BIMS

This analysis will be performed in all 16 079 patients. We will first construct a single candidate logistic regression model that includes all preoperative and surgical variables listed in Figure 1. We will substitute a preoperative estimated glomerular filtration rate value of 5 mL/min/1.73 m2, calculated with the Chronic Kidney Disease Epidemiology Collaboration equation,21 for any patients who were receiving dialysis preoperatively. We will model continuous variables using restricted cubic spline functions. Next, we will simplify the model through backward elimination with a *p *value criterion for removal of *p* > 0.10. In large samples with many events per variable tested, backward elimination produces models that can outperform competing methods.22 We expect there will be many BIMS events given that one-third of patients experienced bleeding and 167 of them died. If there are not enough BIMS events to maintain at least 10 events per variable tested, we will combine types of surgery into larger categories (e.g., major orthopedic, major general).

We will repeat the modelling procedure in each of 1000 bootstrap samples and test each resultant version of the model on the original data, reporting model discrimination using the C-statistic and calibration employing a plot of observed versus predicted probabilities. We will report the full model as a risk-estimating equation that can be integrated into software for use on hand-held devices.

We will attempt to further simplify this model into a risk index consisting of no more than 5 equally weighted risk factors, the sum of which can stratify patients into just a few risk categories. We will report the proportion of patients who experience BIMS across the categories of this risk index, along with its C-statistic to evaluate discrimination.

### Ethics approval

The research ethics board at each site approved the protocol before patient recruitment.

## Interpretation

Although perioperative bleeding is common, the nature and characteristics of bleeding that increase the risk of perioperative death are unclear. We have described our methods for establishing the diagnostic criteria for BIMS and for estimating its incidence and prognostic importance. We have also described the methods for developing and testing a statistical model to predict BIMS.

We are not aware of studies that have systematically evaluated potential diagnostic criteria for BIMS. Transfusion of just 1 unit of packed erythrocytes was associated with perioperative cardiovascular events in a retrospective cohort study of hospital administrative data for 1.6 million adults.3 This association may represent harm from transfusion or the impact of the bleeding that led to transfusion. We considered the range of hemoglobin nadir values that one might expect to contain the most discriminating threshold. In patients at high risk for cardiovascular disease who underwent surgery for hip fracture, a liberal strategy for blood transfusion (hemoglobin concentration 100 g/L) did not affect rates of death or functional outcome compared to a restrictive strategy (hemoglobin concentration < 80 g/L),23 consistent with a recent meta-analysis of 23 trials across surgical and nonsurgical settings.24 These data suggest that a perioperative bleeding event should not increase the risk of death unless the hemoglobin level goes below 80 g/L or perhaps 70 g/L. Finally, numerous investigators have attempted to predict bleeding requiring perioperative blood transfusion in noncardiac surgery,25-31 but those studies were small compared to our study, with samples ranging from 94 to 1875 patients, and were often limited to a single centre and type of surgery.

### Limitations

The number of deaths among people who experienced a bleeding event limits the number of thresholds that we can assess for units of blood products transfused, hemoglobin nadir and hemoglobin decrement. As we assess more thresholds, we risk establishing diagnostic criteria for BIMS that are the product of statistical overfitting. Simulation studies show that, for causal inference, the risk of spurious findings is only marginally higher when we test 1 variable for every 5 events than when we test 1 variable for every 10 or more events but becomes more concerning with 4 events or fewer per variable.32 Our sample size is also insufficient to reliably identify diagnostic criteria for BIMS in specific types of noncardiac surgery. We did not collect data at the level of individual surgeons and will not be able to adjust for the potential effects of surgeon experience on perioperative bleeding and death; our account for centre effects will serve as an imperfect surrogate. We also did not collect data regarding prior bleeding history, and such information may enhance the prediction of BIMS in future studies.

### Conclusion

This study will have implications for over 200 million patients who undergo noncardiac surgery globally every year. Recognition of BIMS can direct closer monitoring and supportive care, and an estimate of the prognostic importance of BIMS can serve as an estimate of the maximum potential benefit of interventions that prevent bleeding still to be developed and tested. Prediction of BIMS can be used to enrich clinical trials and to inform the timing and appropriateness of surgery, and can guide surgical technique and perioperative care with emphasis on hemostasis.

### Supplemental information

For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/5/3/E594/suppl/DC1](http://www.cmajopen.ca/content/5/3/E594/suppl/DC1)

## Acknowledgements

This study was coordinated by the Clinical Advances Through Research and Information Translation (CLARITY) project office in the Department of Clinical Epidemiology and Biostatistics (now the Department of Health Research Methods, Evidence, and Impact) and the Population Health Research Institute, Hamilton Health Sciences, McMaster University, Hamilton, Ont.

## Footnotes

*   **Competing interests:** See the end of the article.

*   **Contributors:** Pavel Roshanov, John Eikelboom, Mark Crowther, Vikas Tandon, Flavia Borges, Clive Kearon, Andre Lamy, Richard Whitlock, Bruce Biccard, Wojciech Szczeklik, Gordon Guyatt, Mohamed Panju, Jehonathan Pinthus, Amit Garg, Michael McGillion, Tomas VanHelder, Peter Kavsak, Justin de Beer, Mitchell Winemaker, Daniel Sessler, Yannick Le Manach, Tej Sheth, Jessica Spence, Lehana Thabane, Marko Simunovic, Ryszard Mizera, Sebastian Ribas and P.J. Devereaux were responsible for the study conception and design. Pavel Roshanov, John Eikelboom, Mark Crowther, Vikas Tandon, Flavia Borges, Clive Kearon, Andre Lamy, Richard Whitlock, Bruce Biccard, Wojciech Szczeklik, Gordon Guyatt, Mohamed Panju, Jehonathan Pinthus, Amit Garg, Michael McGillion, Tomas VanHelder, Peter Kavsak, Justin de Beer, Mitchell Winemaker, Daniel Sessler, Yannick Le Manach, Tej Sheth, Jessica Spence, Lehana Thabane, Marko Simunovic, Ryszard Mizera, Sebastian Ribas and P.J. Devereaux acquired, analyzed or interpreted the data. Pavel Roshanov conducted the statistical analysis. Pavel Roshanov, John Eikelboom, Mark Crowther, Vikas Tandon, Flavia Borges, Clive Kearon, Andre Lamy, Richard Whitlock, Bruce Biccard, Wojciech Szczeklik, Gordon Guyatt, Mohamed Panju, Jehonathan Pinthus, Amit Garg, Michael McGillion, Tomas VanHelder, Peter Kavsak, Justin de Beer, Mitchell Winemaker, Daniel Sessler, Yannick Le Manach, Tej Sheth, Jessica Spence, Lehana Thabane, Marko Simunovic, Ryszard Mizera, Sebastian Ribas and P.J. Devereaux drafted the manuscript and critically revised it for important intellectual content. All of the authors approved the final version to be published and agreed to act as guarantors of the work.

*   **Funding:** Funding was provided by the following Canadian institutions: Canadian Institutes of Health Research, Ottawa, Ont. (6 grants); Heart and Stroke Foundation of Ontario, Toronto, Ont. (2 grants); Academic Health Science Centres Alternative Funding Plan Innovation Fund Grant, Toronto, Ont.; Population Health Research Institute, Hamilton, Ont.; Clinical Advances Through Research and Information Translation (CLARITY) Research Group, Hamilton, Ont.; Surgical Associates Research Grant, Department of Surgery, McMaster University, Hamilton, Ont.; Hamilton Health Science New Investigator Fund Grant, Hamilton, Ont.; Hamilton Health Sciences Grant, Hamilton, Ont.; Ontario Ministry of Resource and Innovation Grant, Toronto, Ont.; Stryker Canada, Waterdown, Ont.; Department of Anesthesiology, McMaster University, Hamilton, Ont. (2 grants); Department of Medicine, Saint Joseph's Healthcare, Hamilton, Ont. (2 grants); Father Sean O'Sullivan Research Centre, Hamilton, Ont. (2 grants); Department of Medicine, McMaster University, Hamilton, Ont. (2 grants); Hamilton Health Sciences Summer Studentships, Hamilton, Ont. (6 grants); Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ont.; Division of Cardiology, McMaster University, Hamilton, Ont.; Canadian Network and Centre for Trials Internationally, Hamilton, Ont.; Winnipeg Health Sciences Foundation Operating Grant, Winnipeg, Man.; Department of Surgery Research Grant, University of Manitoba, Winnipeg, Man. (2 grants); Diagnostic Services of Manitoba Research Grant, Winnipeg, Man. (2 grants); Manitoba Medical Services Foundation, Winnipeg, Man.; Manitoba Health Research Council, Winnipeg, Man.; Faculty of Dentistry Operational Fund, University of Manitoba, Winnipeg, Man.; Department of Anesthesia, University of Manitoba, Winnipeg, Man.; University Medical Group Start-up Fund, Department of Surgery, University of Manitoba, Winnipeg, Man. Funding from Australia: National Health and Medical Research Council Program Grant, Canberra; Australian and New Zealand College of Anaesthetists Grant, Sydney. Funding from Brazil: Projeto Hospitais de Excelência a Serviço do SUS (PROADI-SUS) grant from the Brazilian Ministry of Health in Partnership with HCor (Cardiac Hospital São Paulo), São Paulo, and Support from the National Council for Scientific and Technological Development (CNPq-Brazil). Funding from China: Public Policy Research Fund, Research Grant Council, Hong Kong SAR; General Research Fund, Research Grant Council, Hong Kong SAR. Funding from Colombia: School of Nursing, Universidad Industrial de Santander, Bucaramanga; Grupo de Cardiología Preventiva, Universidad Autónoma de Bucaramanga, Bucaramanga; Fundación Cardioinfantil - Instituto de Cardiología, Bogotá; Alianza Diagnóstica S.A., Bucaramanga. Funding from India: Division of Clinical Research and Training Grant, St. John's Medical College and Research Institute, Bangalore. Funding from Malaysia: University of Malaya Research Grant, Kuala Lumpur; University of Malaya Penyelidikan Jangka Pendek Grant, Kuala Lumpur. Funding from Spain: Instituto de Salud Carlos III, Madrid; Fundació La Marató de TV3, Esplugues de Llobregat. Funding from Switzerland: Roche Diagnostics Global Office, Basel (3 grants). Funding from United Kingdom: National Institute for Health Research, London. Funding from United States: American Heart Association Grant, Dallas, Tex. Mark Crowther holds a Career Investigator Award from the Heart and Stroke Foundation. Amit Garg is supported by the Dr. Adam Linton Chair in Kidney Health Analytics, Western University. Clive Kearon is supported by an Investigator Award from the Heart and Stroke Foundation of Canada and the Jack Hirsh Professorship in Thromboembolism, Department of Medicine, McMaster University.

*   **Disclaimer:** The Vascular Events In Noncardiac Surgery Patients Cohort Evaluation (VISION) funding sources had no role in the design and conduct of the study; collection, management, analysis or interpretation of the data; or preparation, review or approval of the manuscript. This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results and conclusions reported in this article are those of the authors and are independent from the funding sources. No endorsement by the ICES or the Ontario MOHLTC is intended or should be inferred.

## References

1.  Weiser TG, Regenbogen SE, Thompson KD, et al. (2008) An estimation of the global volume of surgery: a modelling strategy based on available data. Lancet 372:139–44.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0140-6736(08)60878-8&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18582931&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000257552400028&link_type=ISI) 

2.  Wu WC, Smith TS, Henderson WG, et al. (2010) Operative blood loss, blood transfusion, and 30-day mortality in older patients after major noncardiac surgery. Ann Surg 252:11–7.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/SLA.0b013e3181e3e43f&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20505504&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000278921200004&link_type=ISI) 

3.  Whitlock EL, Kim H, Auerbach AD (2015) Harms associated with single unit perioperative transfusion: retrospective population based analysis. BMJ 350:h3037.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE4OiIzNTAvanVuMTJfMjEvaDMwMzciO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzUvMy9FNTk0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  Weber EWG, Slappendel R, Prins MH, et al. (2005) Perioperative blood transfusions and delayed wound healing after hip replacement surgery: effects on duration of hospitalization. Anesth Analg 100:1416–21.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1213/01.ANE.0000150610.44631.9D&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15845698&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000228755400041&link_type=ISI) 

5.  Dahl OE, Quinlan DJ, Bergqvist D, et al. (2010) A critical appraisal of bleeding events reported in venous thromboembolism prevention trials of patients undergoing hip and knee arthroplasty. J Thromb Haemost 8:1966–75.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1538-7836.2010.03965.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20586919&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 

6.  Acedillo RR, Shah M, Devereaux P, et al. (2013) The risk of perioperative bleeding in patients with chronic kidney disease: a systematic review and meta-analysis. Ann Surg 258:901–13.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/SLA.0000000000000244&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24169162&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 

7.  Schulman S, Anger SU, Bergqvist D, et al. (2010) Definition of major bleeding in clinical investigations of antihemostatic medicinal products in surgical patients. J Thromb Haemost 8:202–4.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1538-7836.2009.03678.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19878532&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000272864200031&link_type=ISI) 

8.  Harrell FE. rms: regression modeling strategies. R package. 2016. Available[https://cran.r-project.org/web/packages/rms/rms.pdf](https://cran.r-project.org/web/packages/rms/rms.pdf). accessed 2016 Aug. 4.
    
    

9.  Devereaux PJ, Chan MTV, Alonso-Coello P, et al. (2012) Association between postoperative troponin levels and 30-day mortality among patients undergoing noncardiac surgery. JAMA 307:2295–304.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2012.5502&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22706835&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000304801700024&link_type=ISI) 

10. Botto F, Alonso-Coello P, Chan MTV, et al. (2014) Myocardial injury after noncardiac surgery: a large, international, prospective cohort study establishing diagnostic criteria, characteristics, predictors, and 30-day outcomes. Anesthesiology 120:564–78.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/ALN.0000000000000113&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24534856&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000332840300011&link_type=ISI) 

11. Berwanger O, Le Manach Y, Suzumura EA, et al. (2016) Association between pre-operative statin use and major cardiovascular complications among patients undergoing non-cardiac surgery: the VISION study. Eur Heart J 37:177–85.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/eurheartj/ehv456&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26330424&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 

12. Landerman LR, Land KC, Pieper CF (1997) An empirical evaluation of the predictive mean matching method for imputing missing values. Sociol Methods Res 26:3–33.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1177/0049124197026001001&link_type=DOI) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1997XQ13600001&link_type=ISI) 

13. White IR, Daniel R, Royston P (2010) Avoiding bias due to perfect prediction in multiple imputation of incomplete categorical variables. Comput Stat Data Anal 54:2267–75.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.csda.2010.04.005&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24748700&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000279096500006&link_type=ISI) 

14. van Buuren S, Groothuis-Oudshoorn K (2011;45:1-67) mice: multivariate imputation by chained equations in R. J Stat Softw.
    
    

15. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer-Verlag New York; 2009
    
    

16. Wolkewitz M, Allignol A, Harbarth S, et al. (2012) Time-dependent study entries and exposures in cohort studies can easily be sources of different and avoidable types of bias. J Clin Epidemiol 65:1171–80.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2012.04.008&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23017635&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 

17. Khan MA, Grinberg R, Johnson S, et al. (2013) Perioperative risk factors for 30-day mortality after bariatric surgery: Is functional status important? Surg Endosc 27:1772–7.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1007/s00464-012-2678-5&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23299129&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 

18. Endicott KM, Emerson D, Amdur R, et al. (2017) Functional status as a predictor of outcomes in open and endovascular abdominal aortic aneurysm repair. J Vasc Surg 65:40–5.
    
    

19. Crawford RS, Cambria RP, Abularrage CJ, et al. (2010) Preoperative functional status predicts perioperative outcomes after infrainguinal bypass surgery. J Vasc Surg 51:351–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.jvs.2009.08.065&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20141958&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000274602800011&link_type=ISI) 

20. Schisterman EF, Cole SR, Platt RW (2009) Overadjustment bias and unnecessary adjustment in epidemiologic studies. Epidemiology 20:488–95.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/EDE.0b013e3181a819a1&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19525685&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000267065500003&link_type=ISI) 

21. Levey AS, Stevens LA, Schmid CH, et al. (2009) A new equation to estimate glomerular filtration rate. Ann Intern Med 150:604–12.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-150-9-200905050-00006&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19414839&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000265903800004&link_type=ISI) 

22. Ambler G, Brady AR, Royston P (2002) Simplifying a prognostic model: a simulation study based on clinical data. Stat Med 21:3803–22.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/sim.1422&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=12483768&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000180039600005&link_type=ISI) 

23. Carson JL, Terrin ML, Noveck H, et al. (2011) Liberal or restrictive transfusion in high-risk patients after hip surgery. N Engl J Med 365:2453–62.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJMoa1012452&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22168590&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000298544600006&link_type=ISI) 

24. Holst LB, Petersen MW, Haase N, et al. (2015) Restrictive versus liberal transfusion strategy for red blood cell transfusion: systematic review of randomised trials with meta-analysis and trial sequential analysis. BMJ 350:h1354.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNTAvbWFyMjRfOS9oMTM1NCI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNS8zL0U1OTQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

25. van Klei WA, Moons KGM, Rheineck Leyssius AT, et al. (2001) A reduction in type and screen: preoperative prediction of RBC transfusions in surgery procedures with intermediate transfusion risks. Br J Anaesth 87:250–7.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/bja/87.2.250&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=11493498&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000170363200015&link_type=ISI) 

26. Shah MD, Goldstein DP, McCluskey SA, et al. (2010) Blood transfusion prediction in patients undergoing major head and neck surgery with free-flap reconstruction. Arch Otolaryngol Head Neck Surg 136:1199–204.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/archoto.2010.202&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21173368&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 

27. Rashiq S, Shah M, Chow AK, et al. (2004) Predicting allogeneic blood transfusion use in total joint arthroplasty. Anesth Analg 99:1239–44.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1213/01.ANE.0000132928.45858.92&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15385383&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 

28. Larocque BJ, Gilbert K, Brien W (1998) Prospective validation of a point core system for predicting blood transfusion following hip or knee replacement. Transfusion 38:932–7.
    
    

29. Larocque BJ, Gilbert K, Brien WF (1997) A point score system for predicting the likelihood of blood transfusion after hip or knee arthroplasty. Transfusion 37:463–7.
    
    

30. Guerin S, Collins C, Kapoor H, et al. (2007) Blood transfusion requirement prediction in patients undergoing primary total hip and knee arthroplasty. Transfus Med 17:37–43.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17266702&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000243941700003&link_type=ISI) 

31. Carabini LM, Zeeni C, Moreland NC, et al. (2014) Development and validation of a generalizable model for predicting major transfusion during spine fusion surgery. J Neurosurg Anesthesiol 26:205–15.
    
    

32. Vittinghoff E, McCulloch CE (2007) Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol 165:710–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/aje/kwk052&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17182981&link_type=MED&atom=%2Fcmajo%2F5%2F3%2FE594.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000244655200014&link_type=ISI) 

*   Copyright 2017, Joule Inc. or its licensors