Skip to main content
  • Research article
  • Open access
  • Published:

Developing an adapted Charlson comorbidity index for ischemic stroke outcome studies

Abstract

Background

The Charlson comorbidity index (CCI) is commonly used to adjust for patient casemix. We reevaluated the CCI in an ischemic stroke (IS) cohort to determine whether the original seventeen comorbidities and their weights are relevant.

Methods

We identified an IS cohort (N = 6988) from the Ontario Stroke Registry (OSR) who were discharged from acute hospitals (N = 100) between April 1, 2012 and March 31, 2013. We used hospital discharge ICD-10-CA data to identify Charlson comorbidities. We developed a multivariable Cox model to predict one-year mortality retaining statistically significant (P < 0.05) comorbidities with hazard ratios ≥1.2. Hazard ratios were used to generate revised weights (1–6) for the comorbid conditions. The performance of the IS adapted Charlson comorbidity index (ISCCI) mortality model was compared to the original CCI using the c-statistic and continuous Net Reclassification Index (cNRI).

Results

Ten of the 17 Charlson comorbid conditions were retained in the ISCCI model and 7 had reassigned weights when compared to the original CCI model . The ISCCI model showed a small but significant increase in the c-statistic compared to the CCI for 30-day mortality (c-statistic 0.746 vs. 0.732, p = 0.009), but no significant increase in c-statistic for in-hospital or one-year mortality. There was also no improvement in the cNRI when the ISCCI model was compared to the CCI.

Conclusions

The ISCCI model had similar performance to the original CCI model. The key advantage of the ISCCI model is it includes seven fewer comorbidities and therefore easier to implement in situations where coded data is unavailable.

Peer Review reports

Background

The Charlson comorbidity index (CCI) is the most commonly used casemix adjustment method in health outcome studies that use administrative data [1,2,3,4,5,6,7,8,9,10,11,12,13]. In short, using a population of general medical inpatients at one hospital over 30 years ago Charlson identified 17 comorbidites that were associated with one-year mortality and assigned weights to these conditions that when summed created an index that predicted mortality [11]. However, recent work by Quan et al. suggests reweighting of the original Charlson score may be appropriate as new data sources become available, and as the management and outcomes of patients with chronic conditions evolve [3]. In the case of stroke, although it remains a common reason for hospitalization, there have been significant declines in stroke hospitalization and mortality rates over the past fifteen years [13,14,15,16,17,18]. Considering the median age of stroke patients is 75 years, many of these patients have a high pre-existing comorbid burden that can impact mortality making adjustment for differences in comorbidity burden critical when comparing quality of care across provider groups, regions, and countries [13, 17]. The validity of the CCI to adjust for comorbidity in stroke care, was done over 10 years ago, using ICD-9-CM coded inpatient adminstrative data from acute stroke patients across nine Vetern Affairs hospitals [4]. These circumstances support testing the accuracy of the original Charlson weights in a contemporary cohort of ischemic stroke inpatients.

We sought to determine whether the original Charlson comorbidities and corresponding weights were applicable in a cohort of ischemic stroke patients in Ontario by applying Quan et al’s approach to re-calculating Charlson comorbidity weights [3]. We also examined the impact of including atrial fibrillation into the CCI given it is a highly prevalent comorbid condition for ischemic stroke patients and is associated with higher mortality [19,20,21,22,23].

Methods

Setting

This is a retrospective population-based cohort of adults hospitalized with acute ischemic stroke in Ontario, Canada (population 13.3 million). Ontario has a publicly funded health care system that provides coverage of all medically necessary services delivered in hospitals, emergency departments and physician offices and prescription medications for adults aged 65 years and older.

Data sources

The Ontario Stroke Registry’s (OSR) provincial acute stroke audit was used to identify hospitalized ischemic stroke cases. The stroke audit database consists of audited medical records of a population-based sample of patients aged 18 years and older discharged from provincial hospitals between April 1, 2012 and March 31, 2013 with transient ischemic attack (TIA), ischemic stroke, or intracerebral hemorrhage. Details of the audit methodology have been reported previously [18]. The audit was conducted by trained research personnel with access to stroke specialists for consultation and validation by duplicate chart abstraction has demonstrated excellent agreement for stroke type and stroke severity [24]. We restricted the cohort to patients with ischemic stroke. However because several of the 17 Charlson comorbidity conditions that make up the CCI were not captured in the chart audit, we linked the OSR ischemic cohort to the Discharge Abstract database (DAD) and Same-day Surgery database (SDS) using unique encoded identifiers to identify all Charlson comorbidities as well as atrial fibrillation. We included atrial fibrillation as a comorbid condition given its association with stroke mortality [19, 20, 22].

The DAD and SDS is compiled and maintained by the Canadian Institute for Health Information (CIHI) and contains information on all inpatient and day-surgery discharges from acute care hospitals in the province. Data elements in the DAD and SDS include the most responsible diagnosis and up to 24 other diagnoses that are coded according to the International Classification of Diseases, 10th revision, Canada (ICD-10-CA) standard. For each ischemic stroke record we determined the presence or absence of individual Charlson comorbidities as defined by Quan et al ICD-10-CA codes and the atrial fibrillation ICD-10-CA code (see Additional file 1) found in the DAD and SDS databses using a two-year look-back of DAD and SDS records and for index event (i.e., hosptialization for acute stroke), all the diagnoses coded and any diagnosis type (most responsible, pre-admission or post-admission) were also included in the identification process with exception of cerebrovascular disease [25,26,27]. In-hospital death was captured in the DAD and to obtain one-year and 30-day mortality we linked the ischemic stroke cohort to the Ontario Registered Persons Database (RPDB), a database of health insurance plan registrants that includes date of death. All linkages were done using unique encoded identifiers and analyzed at ICES.

Analysis

The ischemic stroke cohort was randomly split into two cohorts; a test cohort (2/3rd) and validation cohort (1/3rd). Descriptive analysis compared characteristics of test and validation cohorts. In the test cohort we developed a multivariable Cox-proportional hazards model (Cox-PH) with one-year mortality as the dependent variable; predictor variables included age groups (< 45 years, 45–54, 55–64, 65–74, 75–84, 85+), sex, individual Charlson comorbidities as well as atrial fibrillation. Charlson comorbidities and atrial fibrillation were considered if they had a frequency of at least 10 patients and a bivariate association with one-year mortality of p < = 0.15. After adding all eligible comorbidites to the model we retained conditions with hazard ratios (HR) greater or equal to 1.2 and p-value < 0.05. The revised comorbidity weights were assigned to the individual comorbidities according to the following algorithm as developed by Quan: a weight of 1 for risk-adjusted hazard ratio of > = 1.2 but < 1.5, a weight of 2 for a hazard ratio of > = 1.5 but < 2.5, a weight of 3 for a hazard ratio of > = 2.5 but < 3.5, a weight of 4 for a hazard ratio of > = 3.5 but < 4.5, a weight of 5 for a hazard ratio of > = 4.5 but < 6 and a weight of 6 for a hazard ratio > = 6 [3]. These steps were repeated with the addition of atrial fibrillation. In summary, we 1) re-weighted the CCI from an ischemic stroke cohort (ISCCI); and 2) re-weighted the CCI adding atrial fibrillation as a comorbid condition (ISCCI-AF). In the validation cohort, we 1) assigned the original Charlson weights to the 17 comorbidities in the ischemic stroke cohort; 2) assigned the ISCCI recalibrated weights; and 3) assigned the ISCCI-AF weights. We then used logistic regression to model three outcomes; in-patient, 30 day and 1 year mortality following ischemic stroke adjusting for age, sex and each of the three comorbidity indices: the original CCI, the ischemic stroke Charlson index (ISCCI) and the ISCCI-AF; which created a total of nine separate models. We used the Hosmer-Lemeshow goodness-of-fit test to assess model calibration and, the c-statistic to assess model discrimination. A c-statistic between 0.7–0.8 indicates reasonable model and > 0.8 is considered a strong model [28]. We also computed the continuous net reclassification index (cNRI) to determine whether the ISCCI model had an improved ability to predict risk of death compared to the original Charlson index (ISCCI vs CCI). The cNRI measures the improvement in correctly classifying patients as high (or low) risk for death by calculating the sum of the differences in the estimated probability of net upward reclassification of death and the estimated probability of net downward reclassification for no death [29]. We used SAS Enterprise Guide, version 6.1 for all analyses (SAS Institute, Cary, NC).

Results

Characteristics of ischemic stroke patients in the test and validation cohorts were similar (Table 1). The majority of ischemic stroke patients had mild stroke (59.7% in the test cohort and 58.6% in the validation). In addition to the most responsible diagnosis of ischemic stroke, a median of four diagnosis codes were found in the index hospitalization record in both cohorts. All-cause death within one year of admission was similar for the test (24.6%) and validation (23.4%) cohorts. The median survival time among those who died within 1 year was 30 days and 32 days, in the 2 cohorts, respectively. The frequency of individual comorbid conditions was similar with the exception of mild liver disease where the validation cohort had a slightly higher proportion (Table 1). The most frequently reported comorbidities were diabetes with complication (28%), atrial fibrillation (27%), and hemi or paraplegia (17%).

Table 1 Characteristics of Ischemic Stroke test and validation cohorts, Ontario Stroke Audit, 2012/13

Comorbidity hazard ratios and weights for ISCCI derived from the test cohort are shown in Table 2. For comparison purposes, the original Charlson weights are also shown [11]. There was no difference in weights when atrial fibrillation (ISCCI-AF) was included in the model compared to the ISCCI model weights except cerebrovascular disease became non-significant and atrial fibrillation became significant with a weight of 1 (data not shown). Ten of the 17 Charlson comorbid conditions in the ISCCI were statistically significant (i.e, a HR ≥ 1.2). Compared to Charlson weights, the ISCCI weights were one point higher for four conditions (myocardial infarction (MI), congestive heart failure (CHF), dementia, rheumatologic disease), remained the same for three conditions (cerebrovascular disease (CVD), chronic obstructive pulmonary disease (COPD), any malignancy,) and, one point lower for three conditions (diabetes with chronic complications, hemi or paraplegia, metastatic solid tumor). Figure 1 compares the frequency of CCI scores with ISCCI, and ISCCI-AF scores in the validation cohort. The distribution of scores shows a similar proportion of patients with 0 (~ 38%) and 5 and higher scores (~ 6%), but the two new models, had more than double the proportion of patients with a score of 1 (26.9, 30.3%, vs 12.3%), and a smaller proportion of patients with scores of 2 and 4 compared to the original CCI. Figure 1 also illustrates the observed 30-day and one-year mortality. As the comorbidity score increases, the proportion of 1 year mortality correspondingly increases. For example, among patients with an ISCCI of 5 or greater 33.3% died within 30 days and 61.9% within one year.

Table 2 Comorbidity weights derived from the ischemic stroke test cohort (April 1, 2012 to March 31, 2013) compared with weights of the original Charlson comorbidity index [11]
Fig. 1
figure 1

Distribution of comorbidity score, by index, and mortality associated with ISCCI among the validation cohort (N = 2331), April 1, 2012 to March 31, 2013

Table 3 shows the calibration and predictive accuracy of modelling the probability of death in-hospital, within 30 days or 1 year of admission for the three models. For all mortality outcomes, thec-statistics for the ISCCI models were higher compared to the CCI model. For example, for 30-day mortality the c-statistic for the original CCI was 0.732 compared to 0.746, for the ISCCI p = 0.009. However the difference between the CCI and the ISCCI model were negligible and non-significiant for in-hospital death and one-year (0.722 vs 0.729 p = 0.343 and 0.760 vs 0.764, p = 0.398, respectively). Including atrial fibrillation (ISCCI-AF) did not improve the ISCCI model discrimination. The cNRI analysis did not include the ISCCI-AF model given the lower c-statistic compared to the ISCCI. The cNRI showed the ISCCI model compared to the CCI model did not improve patient net mortality risk reclassification (Table 3).

Table 3 Comparison of Model performance among ischemic stroke patients in the validation cohort (N = 2331, discharged between April 1, 2012 and March 31, 2013)

Discussion

Using a large population-based ischemic stroke cohort discharged from acute hospitals in Ontario, Canada between April 1, 2012 and March 31, 2013, we examined the association between the Charlson comorbidities and mortality to determine which Charlson comorbidities are relevant among ischemic stroke patients. Similar to other studies, we found several Charlson comorbidities were not signifinicant predictors of one-year mortality in ischemic stroke patients [3, 6, 31,32,33]. In fact, we found just ten of the 17 Charlson comorbidities to be associated with one-year mortality following an acute ischemic stroke. The very low prevalence of moderate or severe liver disease and AIDS/HIV (n < 10) meant they were not considered for the ISCCI. Of the remaining Charlson comorbidities, peripheral vascular disease, peptic ulcer disease, mild liver disease, diabetes without complications, renal disease, were not associated with one-year mortality in our ischemic stroke cohort (p > 0.05).

Although the ISCCI model demonstrated marginal improved model performance (< 2% increase in c-statistic) compared with the original CCI, the extent of model improvement is similar to other studies comparing performance of revised CCI models to the originial CCI [3, 7, 31, 32]. Not surprisingly, there was little difference in model performance between the ISCCI and ISCCI-AF models across all outcomes given no difference in the total number of comorbidities and the corresponding weights. Additionally, the ISCCI didn’t result in significant gain or loss in net patient mortality risk reclassifications.

A criticism of the CCI derived from administrative databases is the concern over misclassifying complications as comorbidities [5, 26, 34]. Although the CIHI DAD and SDS databases have diagnosis type fields to allow differentiation between a pre-admission condition and conditions that developed during the hospitalization, reabstraction studies have found modest validity for the diagnosis type field and therefore distinguishing conditions that arise as a consequence of natural disease progression from complications of care is limited [27, 34]. However, most Charlson comorbidities, with the possible exception of hemi- or paraplegia, would not be considered complications in ischemic stroke patients. The prevalence of existing hemi- or paraplegia among acute stroke patients in the OSR is ~ 2% (data not shown). We examined the 1138 IS patients identified as having hemi- or paraplegia and determined all were coded in the index hospitalization record. The majority (87.7%) classified hemi- or paraplegia as a pre-existing condition, 2.8% a post admission condition or complication and 9.5% a secondary diagnosis. A secondary diagnosis does not meet the requirement for determining comorbidity, ie., capturing a symptom of the stroke [27]. CIHI DAD data quality studies have revealed overall coder agreement of diagnosis type was 76%, ranging from 65% for post-admission diagnoses, 67% for pre-existing diagnoses to 86% for most responsible diagnosis [27]. Given coders are limited to physician documentation only when assigning diagnosis types and, the way physicians document is to record conditions relevant to patient treatment/management, it is not surprising coder assignment of diagnosis type is challenging [35, 36].

We suspect the coding of hemi- or paraplegia may be caputuring either; 1) acute stroke associated symptoms or, 2) prior stroke given the low prevalence of prior stroke in our cohort (cerebrovascular disease ~ 6%). If, hemi- paraplegia coding in the index acute stroke event is capturing the symptoms of the acute stroke this may reflect stroke severity and given stroke severity is not available in administrative databases, it may be reasonable to include hemi- and paraplegia in an administrative database derived risk-adjustment model.

Stroke severity is strongly associated with mortality and recommended to be included in risk adjustment models [37]. Given, our ischemic stroke cohort was from the OSR and the OSR captures stroke severity through the Canadian Neurological Scale, we added stroke severity to the ISCCI model as an additional covariate and model performance improved substantially (c = 0.855 vs 0.746, for 30-day mortality, data not shown).

If we consider the 87.7% of the 1138 IS patients with hemi- or paraplegia coded as a pre-existing condition to be a result of a previous stroke; and combine it with the cerebrovascular disease prevalence, (n = 397) the prevalence of previous stroke would be ~ 20% and is within the range reported in the literature [17, 18, 38,39,40]. Further investigation of including hemi- or paraplegia diagnostic codes into algorithms to identify prior stroke in ICD coded databases is warranted.

Our study is not without limitations, firstly, we only examined mortality and our findings may not apply to other outcomes including length of stay, cost, readmission and patient functional outcomes. We also limited comorbidity identification to acute inpatient and same-day surgery hospital-based claims with a two-year lookback. However, we found little gain in model performance when we used a three-year lookback (data not shown) and a longer lookback was not examined . Secondly, the accuracy of ICD codes and number of diagnostic code fields available and or completed to capture comorbidity conditions is jurisdictional dependent [3, 41]. In Ontario, the low prevalence of comorbidities in administrative databases compared with clinical data obtained in reabstraction studies has been reported, although when a lookback period was applied, prevalence improved for several comorbidities [42,43,44,45,46]. Other stroke-related comorbidities associated with mortality and higher population attributable risk of stroke, such as obesity, smoking and hypertension were not examined given the low prevalence and unreliable coding of smoking and obesity in hospital-based administrative data and hypertension has been shown to be negatively associated with mortality [23, 47,48,49]. Additionally, we did not examine other administrative data sources like physician billing, emergency department, drug and laboratory databases for the comorbidity history of our cohort. However, little gain in model performance has been observed when physician billing data were included with hospitalization data [5, 50]. We did not use the OSR to identify comorbidities due to the inability to map one to one the 17 Charlson comorbidities (e.g., the OSR does not distinguish between diabetes with and without complications, mild vs moderate/severe liver disease and does not capture history of rheumatic disease) and our intended focus was on administrative data derived CCI not comparing a claims-based CCI to a medical record-based CCI. The ability to identify comorbidities in various databases such as-physician claims, laboratory, diagnostic imaging and drug databases and electronic medical records, is worthy of future research especially with the growing access to these data sources and increasing computational capacity and advanced analytical techniques like machine learning algorirthms that allow for the integration of time-dependent data. Finally, our findings are based on Ontario administrative data within the context of a universal health care system with mandatory hospitalization data submission and processes for error checking; therefore our results may not be generalizable to other settings or populations. Despite this, our findings are from a large, province-wide sample of ischemic stroke in-patients with complete follow-up for deaths of varying time frames.

Conclusion

We have shown the ISCCI model had similar performance to the original CCI model and therefore in the context of an ischemic stroke cohort the CCI remains a valid measure of comorbidity when using administrative data. The key advantage of the ISCCI model is it includes seven fewer comorbidities (10 vs 17) and therefore easier to implement in situations where coded data is unavailable (e.g. chart reviews, clinical trials, surveys and clinical registries).

Availability of data and materials

The full data set creation plan is available from the corresponding author upon request. The datasets from this study are held securely in coded form at ICES. While data sharing agreements prohibit ICES from making the datasets publicly available, access may be granted to those who meet pre-specified criteria for confidential access, available at www.ices.on.ca/DAS.

Abbreviations

CCI:

Charlson comorbidity index

CHF:

Congestive heart failure

CIHI:

Canadian Institute for Health Information

cNRI:

continuous Net Reclassification Index

COPD:

Chronic obstructive pulmonary disease

CVD :

Cerebrovascular disease

DAD:

Discharge abstract database

ICD :

International classification of diseases

ICD-10-CA:

International classification of diseases, 10th revision, Canada

ISCCI:

Ischemic stroke comorbidity index

ISCCI-AF:

Ischemic stroke comorbidity index-atrial fibrillation

MI :

Myocardial infarction

OSR:

Ontario stroke registry

RPDB :

Registered Persons Database (RPDB)

SDS:

Same day surgery database

TIA:

Transient ischemic attack

References

  1. DeGroot V, Beckerman H, Lankhorst GJ, Bouter LM. How to measure comorbidity: a critical review of available methods. J Clin Epi. 2003;56(3):221–9.

    Article  Google Scholar 

  2. Sundararajan V, Henderson T, Perry C, Muggivan A, Quan H, Ghali WA. New ICD-10 version of the Charlson comorbidity index predicted in-hospital mortality. J Clin Epidemiol. 2004 Dec;57(12):1288–94.

    Article  PubMed  Google Scholar 

  3. Quan H, et al. Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries. Am J Epidemiol. 2011;173:676–82.

    Article  PubMed  Google Scholar 

  4. Goldstein LB, Samsa GP, Matchar DB, Horner RD. Charlson index comorbidity adjustment for ischemic stroke outcome studies. Stroke. 2004;35:1941–5.

    Article  PubMed  Google Scholar 

  5. Radley DC, Gottlieb DJ, Fisher ES, Tosteson AN. Comorbidity risk-adjustment strategies are comparable among persons with hip fracture. J Clin Epidemiol. 2008;61(6):580–7.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Lui J, Huang Z, Gibertson DT, Foley RN, Collins AJ. An improved comorbidity index for outcome analyses among dialysis patients. Kidney Int. 2010;77:141–51.

    Article  Google Scholar 

  7. Quail JM, Lix LM, Osman BA, Teare GF. Comparing comorbidity measures for predicting mortality and hospitalization in three population-based cohorts. BMC Health Serv Res. 2011;11:146.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Sharabiani MTA, Aylin P, Bottle A. Systematic review of comorbidity indices for administrative data. Med Care. 2012;50(12):1109–18.

    Article  PubMed  Google Scholar 

  9. Charlson M, Wells MT, Ullman R, King F, Shmukler C. The Charlson comorbidity index can be used prospectively to identify patients who will incur high future costs. PLoS One. 2014;9(12):e112479. https://doi.org/10.1371/journal.pone.0112479.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Austin SR, Wong YN, Uzzo RG, Beck R, Egleston BL. Why summary comorbidity measures such as the Charlson comorbidity index and the Elixhauser score work? Med Care. 2015;53(9):e65–e72.9.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chron Dis. 1987;40(5):373–83.

    Article  CAS  PubMed  Google Scholar 

  12. Lu M, Sajobi T, Lucyk K, Lorenzetti D, Quan H. Systematic review of risk adjustment models of hospital length of stay (LOS). Med Care. 2015;53(4):355–65.

    Article  PubMed  Google Scholar 

  13. Schmidt M, Jacobsen JB, Johnsen SP, Botker HE, Sorenson HT. Eighteen-year trends in stroke mortality and the prognostic influence of comorbidity. Neurology. 2014;82:340–50.

    Article  PubMed  Google Scholar 

  14. Kapral MK, Fang J, Silver FL, et al. Effect of a provincial system of stroke care delivery on stroke care and outcomes. CMAJ. 2013;185:E483–91.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Kamal N, Lindsay MP, Côté R, Fang J, Kapral MK, Hill MD. Ten-year trends in stroke admissions and outcomes in Canada. Can J Neurol Sci. 2015;42(3):168–75.

    Article  PubMed  Google Scholar 

  16. Hall RE, Khan F, Levi J, Ma H, Fang J, Martin C, Morrison K, Fearn J, Brown G, Kelloway L, Kapral MK, O’Callaghan C, Bayley M, Linkewich B. Ontario and LHIN 2015/16 stroke report cards and Progress reports: setting the Bar higher. Institute for Clinical Evaluative Sciences: Toronto; 2017.

    Google Scholar 

  17. Fonarow G, Reeves M, Smith EE, Saver JL, Zhao X, Olson DW, et al. Characteristics, performance measures and in-hospital outcomes of the first one million stroke transient ischemic attack admissions in get with the guidelines-stroke. Circ Cardiovasc Qual Outcomes. 2010;3:291–302.

    Article  PubMed  Google Scholar 

  18. Hall R, Khan F, O’Callaghan C, Kapral MK, Levi J, Cullen A, Wu J, Fang J, Bayley MB. Ontario stroke evaluation report 2014: on target for stroke prevention and care. Institute for Clinical Evaluative Sciences: Toronto; 2014.

    Google Scholar 

  19. Otite FO, Khandelwal P, Chaturvedi S, Romano JG, Sacco RL, Malik AM. Increasing atrial fibrillation prevalence in acute ischemic stroke and TIA. Neurology. 2016;87:1–8.

    Article  Google Scholar 

  20. Benjamin EJ, Wolf PA, D’Agostino RB, Silbershatz H, Kannel WB, Levy D. Impact of atrial fibrillation on the risk of death. Circulation. 1998;98:946–52.

    Article  CAS  PubMed  Google Scholar 

  21. Schwamm LH, Ali SF, Reeves MJ, Smith EE, Saver JL, Messe S, et al. Temporal trends in patient characteristics and treatment with intravenous thrombolysis among ischemic stroke patients at get with guidelines – stroke hospitals. Circ Cardiovascular Quality Outcomes. 2013;6:543–9.

    Article  PubMed  Google Scholar 

  22. Katzan IL, Spertus J, Pruv Bettger J, Bravata DM, Reeves MJ, Smith EE, et al. Risk adjustment of ischemic stroke outcomes for comparing hospital performance: a statement for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2014;45. https://doi.org/10.1161/01.str.0000441948.35804.77.

    Article  PubMed  Google Scholar 

  23. Benrheim S, Wang C, Wang Y, Bhat K, Savage S, Lichtman J, Phipps MS, Drye EE, Krumholz HM. Hospital 30-day mortality following acute ischemic stroke hospitalization measure. Methodology Report. September 2010. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/Measure-Methodology.html. Accessed 13 Aug 2019.

  24. Kapral MK, Fang J, Alibhai SMH, et al. Risk of fractures after stroke. Results from the Ontario stroke registry. Neurology. 2017;88:1–8.

    Article  Google Scholar 

  25. Quan H, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care. 2005 Nov;43(11):1130–9.

    Article  PubMed  Google Scholar 

  26. Ghali WA, Quan H, Brant R. Risk Adjustment Using Administrative Data. J Gen Intern Med. 2001 Aug;16(8):519–24.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Canadian Institute for Health Information. CIHI data quality study of the 2008–2009 discharge abstract database. Ottawa: CIHI; 2010.

    Google Scholar 

  28. Steyerberg E. Clinical prediction models: a practical approach to development, validation and updating. Springer Science & Business Media; 208:508.

  29. Pencina MJ, D'Agostino RB Sr, D’Agostino RB Jr. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27:157–72.

    Article  PubMed  Google Scholar 

  30. Cote R, Battista RN, Wolfson C, Boucher J, Adam J, Hachinski V. The Canadian neurological scale: validation and reliability assessment. Neurology. 1989;39:638–43.

    Article  CAS  PubMed  Google Scholar 

  31. Park JY, Kim MH, Han SS, Cho H, Kim H, Ryu DR, et al. Recalibration and Validation of the Charlson Comorbidity Index in Korean Incident Hemodialysis patients. PLOS One. 2015. https://doi.org/10.1371/journal.pone.0127240.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  32. Hemmelgarn BR, Manns BJ, Quan H, Ghali WA. Adapting the charlson comorbidity index for use in patients with ESRD. Am J Kidney Dis. 2003;42(1):125–32.

    Article  PubMed  Google Scholar 

  33. Ghali WA, Hall RE, Rosen AK, Ash AS, Moskowitz MA. Choosing a measure of comorbidity for comparing outcomes of care for patients undergoing coronary artery bypass surgery. J Clin Epidem. 1995;49(3):273–8.

    Article  Google Scholar 

  34. Quan H, Parsons GA, Ghali WA. Assessing accuracy of diagnosis-type indicators for flagging complications in administrative data. J of Clin Epi. 2004;57:366–72.

    Article  Google Scholar 

  35. Canadian Institute for Health Information. Canadian Coding Standards for Version 2015 ICD-10-CA and CCI. Canadian Institute for Health Information. Ottawa: ON; 2015.

  36. Lucyk K, Tang K, Quan H. Barriers to data quality resulting from the process of coding health information to administrative data: a qualitative study. BMC Health Serv Res. 2017;17:766. https://doi.org/10.1186/s12913-017-2697-y.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Fonarow GC, Wenqin P, Saver JL, Smith EE, Reeves MJ, Boderick JP, Kleindorfer DO, Sacco RL, Olson DWM, Hernandez AF, Peterson ED, Schwamm LH. Comparison of 30-day mortality models for profiling hospital performance in acute ischemic stroke with vs without adjustment for stroke severity. JAMA. 2012;308(3):257–64. https://doi.org/10.1001/jama.2012.7870.

    Article  CAS  PubMed  Google Scholar 

  38. Luengo-Fernandez R, ST PNLMGAMP, Bull LM, SJV W, Cuthertson FC, Rothwell PM, on behalf of the Oxford Vascular Study. Population-based study of disability and institutionalization after transient ischemic attack and stroke: 10-year results of the Oxford vascular study. Stroke. 2013;44:2854–61.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Castro HHG, Alencar AP, Benseñor IM, Lotufo PA, Goulart AC. Multimorbidities are associated to lower survival in Ischaemic stroke: results from a Brazilian stroke cohort (EMMA study). Cerebrovasc Dis. 2017;44:232–9.

    Article  PubMed  Google Scholar 

  40. Willeit J, Geley T, Schoch J, et al. Thrombolysis and clinical outcome in patients with stroke after implementation of the Tyrol stroke pathway: a retrospective observational study. Lancet Neurol. 2015;14:48–56.

    Article  PubMed  Google Scholar 

  41. Thygesen SK, Christiansen CF, Christensen S, Lash TL, Sorensen HT. The predictive value of ICD-10 diagnostic coding used to assess Charlson comorbidity index conditions in the population-based Danish National Registry of patients. BMC Med Res Methodol. 2011;11:83.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Juurlink DN, Preyra C, Croxford R, et al. Canadian Institute for Health Information Discharge Abstract Database: AValidation study. Toronto: Institute for Clinical Evaluative Sciences; 2006. Available at: http://www.ices.on.ca/Publications/Atlases-and-Reports/2006/Canadian-Institutefor-Health-Information. Accessed 26 Apr 2016

    Google Scholar 

  43. Lee DS, Donovan L, Austin PC, Gong Y, Liu P, Rouleau JL, Tu JV. Comparison of coding of heart failure and comorbidities in administrative and clinical data for use in outcomes research. Med Care. 2005;43(2):182–8.

    Article  PubMed  Google Scholar 

  44. Hux J, Ivis V, Flintoft V, Bica A. Diabetes in Ontario: determination of prevalence and incidence using a validated administrative data algorithm. Diabetes Care. 2002;25(3):512–6.

    Article  PubMed  Google Scholar 

  45. Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T. Identifying individuals with physician-diagnosed COPD in health administrative databases. J Chron Obstruct Pulmon Dis. 2009;6(5):388–94.

    Article  CAS  Google Scholar 

  46. Hall R, Mondor L, Porter J, Fang J, Kapral MK. Accuracy of administrative data for the coding of acute stroke and TIAs. Can J Neurol Sci. 2016;22(7):765–73.

    Article  Google Scholar 

  47. Huo J, Yang M, Shih YT. Sensitivity of claims-based algorithms to ascertain smoking status more than doubled with meaningful use. Value Health. 2018;21(3):334–40.

    Article  PubMed  Google Scholar 

  48. Harvard A, Jorm LR, Lujic S. Risk adjustment for smoking identified through tobacco use diagnoses in hospital data: a validation study. PLoS One. 2014;9(4):e95029. https://doi.org/10.1371/journal.pone.0095029.

    Article  Google Scholar 

  49. Martin BJ, Chen G, Graham M, Quan H. Coding obesity in administrative hospital discharge abstract data: accuracy and impact for future research. BMC Health Serv Res. 2014;14:70.

    Article  PubMed  PubMed Central  Google Scholar 

  50. Crooks CJ, West J, Card TR. A comparison of the recording of comorbidity in primary and secondary care by using the Charlson index to predict short-term and long-term survival in a routine linked data cohort. BMJ Open. 2015;5:e007974.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors thank Tara O’Neill for assisting with the preparation of this manuscript.

Funding

This study was supported by ICES, a Foundation Grant (FDN 143303) from the Canadian Institutes of Health Research (CIHR) and, the Ontario Stroke Network (OSN). ICES is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. The funders had no role in the design of the study, the collection, analysis and interpretation of data or writing the manuscript. No endorsement by ICES, OSN, CIHR or the Ontario MOHLTC is intended or should be inferred.

Parts of this material are based on data and information compiled and provided by CIHI. However, the analyses, conclusions, opinions and statements expressed herein are those of the author, and not necessarily those of CIHI.

Author information

Authors and Affiliations

Authors

Contributions

REH conceived the design of the work, acquisition, and interpretation of the data and drafted the work and revising it critically for important intellectual content, approved the final version to be published and is accountable for all aspects of the work. JP performed the analyses, interpreted the data, drafted and revised the work for critically important intellectual content, approved the final version to be published and is accountable for all aspects of the work. HQ contributed substantially to the interpretation of the data, provided critical revisions to the work for important intellectual content, approved the final version to be published and is accountable for all aspects of the work. MJR contributed substantially to the interpretation of the data, provided critical revisions to the work for important intellectual content, approved the final version to be published and is accountable for all aspects of the work.

Corresponding author

Correspondence to Ruth E. Hall.

Ethics declarations

Ethics approval and consent to participate

ICES is a prescribed entity under section 45 of Ontario’s Personal Health Information Protection Act. Section 45 authorizes ICES to collect personal health information, without consent, for the purpose of analysis or compiling statistical information with respect to the management of, evaluation or monitoring of, the allocation of resources to or planning for all or part of the health system. Projects conducted under section 45, by definition, do not require review by a Research Ethics Board. This project was conducted under section 45, and approved by ICES’ Privacy and Legal Office.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

ICD-10-CA codes for comorbidity conditions.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hall, R.E., Porter, J., Quan, H. et al. Developing an adapted Charlson comorbidity index for ischemic stroke outcome studies. BMC Health Serv Res 19, 930 (2019). https://doi.org/10.1186/s12913-019-4720-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12913-019-4720-y

Keywords