Identifying heart failure in patients with chronic obstructive lung disease through the Canadian Primary Care Sentinel Surveillance Network in British Columbia: a case derivation study
========================================================================================================================================================================================

* Rohit Vijh
* Sabrina T. Wong
* Matthew Grandy
* Sandra Peterson
* Allison M. Ezzat
* Andrew G. Gibb
* Nathaniel M. Hawkins

## Abstract

**Background:** Heart failure (HF) poses a substantial global health burden, particularly in patients with chronic obstructive pulmonary disease (COPD). The objective of this study was to validate an electronic medical record–based definition of HF in patients with COPD in primary care practices in the province of British Columbia, Canada.

**Methods:** We conducted a cross-sectional retrospective chart review from Sept. 1, 2018, to Dec. 31, 2018, for a cohort of patients from primary care practices in BC whose physicians were recruited through the BC node of the Canadian Primary Care Sentinel Surveillance Network. Heart failure case definitions were developed by combining diagnostic codes, medication information and laboratory values available in primary care electronic medical records. These were compared with HF diagnoses identified through detailed chart review as the gold standard. Sensitivity, specificity, negative (NPV) and positive predictive values (PPV) were calculated for each definition.

**Results:** Charts of 311 patients with COPD were reviewed, of whom 72 (23.2%) had HF. Five categories of definitions were constructed, all of which had appropriate sensitivity, specificity and NPV. The optimal case definition consisted of 1 HF billing code or a specific combination of medications for HF. This definition had an excellent specificity (93.3%, 95% confidence interval [CI] 89.4%–96.1%), sensitivity (90.3%, 95% CI 81.0%–96.0%), PPV (80.2%, 95% CI 69.9%–88.3%) and NPV (97.0%, 95% CI 93.8%–98.8%).

**Interpretation:** This comprehensive case definition improves upon previous primary care HF definitions to include medication codes and laboratory data, along with previously used billing codes. A case definition for HF was derived and validated and can be used with data from electronic medical records to identify HF in patients with COPD in primary care accurately.

Heart failure (HF) is a global public health problem affecting 40 million people worldwide,1 with health care costs exceeding $100 billion.2 Chronic obstructive pulmonary disease (COPD) is a frequent comorbidity, occurring in 20%–30% of patients with HF.3 The combination presents diagnostic and therapeutic challenges, with recognized undertreatment of patients with HF and concurrent COPD.4 This combination of conditions is also associated with greater morbidity and mortality than either condition alone.3

To assess the quality of the care this high-risk population receives, an accurate case definition is necessary. This would allow epidemiologic studies to define the overall population burden of disease, characterize contemporary medical management in primary care, create opportunities for quality improvement and earlier intervention in the disease pathway and facilitate future studies examining this cardiopulmonary intersection.

Even though the care of many patients with HF is largely managed in primary care without specialist contact,5 studies examining the accuracy of HF diagnosis and coding have been predominantly hospital based.6–8 A systematic review of validated case definitions for HF found that hospital discharge data were used in 25 out of 35 studies, with all studies using exclusively *International Classification of Diseases, Ninth Revision and 10th Revision* (ICD-9 and ICD-10) codes.9 Additionally, physician billing services provide only limited data, resulting in variable accuracy parameters (sensitivity, specificity, positive predictive value [PPV] and negative predictive value [NPV]).9,10 In Canada, HF is often identified through discharge abstract databases or physician billing data.11,12

Identifying HF in patients with COPD from primary care would be beneficial for several reasons. First, electronic medical records (EMRs) systematically collect clinically verified data regarding the management of individual patients. Second, the availability of additional clinical variables such as medication history and laboratory data has the potential to improve case validity. Third, EMRs can characterize community-dwelling patients with COPD and HF who would not otherwise be captured from hospital sources.

Only a handful of studies have evaluated an HF case definition using primary care databases and compared it against a gold standard (i.e., chart review) in an unselected generalizable population.12–14 Of these studies, 2 originated in the United States; the remaining study compared administrative and billing data with primary care records in Ontario.12 Finally, to our knowledge, no studies have examined a case definition of HF in a population with COPD.9,15 The objective of this study was to derive and validate an EMR-based definition of HF among patients with COPD in primary care in British Columbia, Canada.

## Methods

### Study design

The study was a cross-sectional retrospective chart review of a cohort of patients from primary care practices in BC recruited through the BC node of the Canadian Primary Care Sentinel Surveillance Network (CPCSSN).

### CPCSSN data

Case definitions for multiple chronic diseases, including COPD, have been validated using the CPCSSN database.16 The COPD case definition has been found to have a sensitivity of 82.1 (95% confidence interval [CI] 76.0–88.2), a specificity of 97.3 (95% CI 96.5–98.0), a PPV of 72.1 (95% CI 65.4–78.8) and an NPV of 98.4 (95% CI 97.9–99.0).16

CPCSSN is a pan-Canadian network of networks that includes over 1500 primary care providers, covering all provinces except Saskatchewan, using 17 different EMR systems.17 Point-of-care deidentified data are extracted semiannually and transformed to a standard CPCSSN schema. The architecture and approach have been described previously, including data flow, quality, mapping, cleaning and deidentification (Appendix 1, available at [www.cmajopen.ca/content/9/2/E376/suppl/DC1](http://www.cmajopen.ca/content/9/2/E376/suppl/DC1)).17 As of December 2019, CPCSSN includes over 200 million records from almost 2 million patients.

This project used BC-CPCSSN data from the CPCSSN extraction for the fourth quarter of 2018 (2018-Q4; Sept. 1–Dec. 31, 2018). The data include information on patients’ sociodemographic characteristics, providers, encounters, health conditions, risk factors, biometrics, laboratory results, procedures, medications and referrals.16

### Study population

We invited 18 primary care practices that had higher numbers of patients with COPD than other practices in the BC-CPCSSN to participate in this study. Three practices accepted our invitation. Physicians in these practices were recruited through 1 of the study authors (S.W.). Study participation was voluntary and no remuneration was provided. Assuming a prevalence for HF of 20% among patients with COPD, a sample size of 311 patients with COPD was estimated a priori to achieve a precision of 10% for sensitivity and 5% for specificity.

Nine of the 14 physicians consented to provide their patient panel for chart review. We used the EMR data extracted for CPCSSN for each consenting physician. For each participating physician, a member of our team (A.G.) identified all patients in the practice with COPD. Patients were excluded if they did not meet the CPCSSN COPD case definition (which excludes people < 35 yr).

### Sampling

Author A.G. generated the initial sample of all BC patients in CPCSSN (*n* = 102 867 patients) from the 2018-Q4 period. From this cohort, we selected patients aged 35 years and older with COPD (based on the validated CPCSSN definition) from the participating 3 clinics and 9 physicians (*n* = 625). From this cohort, a medical resident (R.V.) randomly selected 311 of these patient charts (Figure 1).18

![Figure 1:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/9/2/E376/F1.medium.gif)

[Figure 1:](http://www.cmajopen.ca/content/9/2/E376/F1)

Figure 1: 
Flow diagram showing process of selection of patient charts from electronic medical records in BC-CPCSSN database. Note: BC-CPCSSN = British Columbia node of the Canadian Primary Care Surveillance Sentinel Network, COPD = chronic obstructive pulmonary disease. Anonymized data from the research sample of patient charts were reviewed. Case definitions for heart failure were then applied to this cohort of patients with COPD for derivation and validation analysis.

### Chart review and gold standard validation

For the 311 patients included in the study, gold standard validation15 was conducted by R.V., who verified the presence or absence of HF for each patient by manually reviewing their entire EMR, between September and October 2019. R.V. was masked to the diagnosis of HF, as per coding in CPCSSN, in these charts.

A standardized data abstraction tool was developed in Qualtrics with input from a cardiologist (N.H.) and family physicians (R.V., M.G), focused on variables required to establish a diagnosis of HF (Appendix 2, available at [www.cmajopen.ca/content/9/2/E376/suppl/DC1](http://www.cmajopen.ca/content/9/2/E376/suppl/DC1)). The chart review included a review of unstructured data from clinic site visits (e.g., echocardiogram reports, free text notes) to obtain further evidence to confirm HF. The abstracted data were reviewed initially by the abstractor (R.V.) and subsequently by a cardiologist (N.H.), who ultimately determined the classification of HF status.

The presence of HF was defined by national19 and international guidelines.20,21 Determination of HF requires the presence of symptoms or signs (or both) of reduced cardiac output or pulmonary or systemic congestion (or both), supported by objective evidence of structural or functional cardiac abnormalities (or both), including left ventricular systolic dysfunction (defined by reduced left ventricular ejection fraction using any imaging modality), diastolic dysfunction (typically determined by echocardiography), elevated natriuretic peptides or structural disease (such as severe valve disease).

### Case definition

An initial case definition was developed by the Maritime Family Practice Research Network (MaRNet-FP), on the basis of ICD-9 and ICD-10 codes or prescribed combinations of medical therapies for HF (Table 1 and Appendix 3, available at [www.cmajopen.ca/content/9/2/E376/suppl/DC1](http://www.cmajopen.ca/content/9/2/E376/suppl/DC1)). BC-CPCSSN examined this original definition, then proposed and tested amendments (Table 1). These modifications were developed for several reasons: to take advantage of the location of additional data elements within different EMR systems (e.g., billing v. encounter diagnoses), expanded ICD-9 codes (based on review of HF coding literature), addition of HF-specific medications (e.g., sacubitril-valsartan, ivabradine) and laboratory data (brain natriuretic peptide [BNP] and N-terminal pro brain natriuretic peptide [NT-proBNP]) (Table 1). Two thresholds for natriuretic peptides were applied: the recommended low “rule out” threshold (BNP > 50 or NT-proBNP > 125) and the higher “probable diagnosis” threshold (BNP > 400 or NT-proBNP > 450 for patients aged < 50 yr, > 900 for patients aged 50–75 yr and > 1800 for patients aged > 75 yr).18

View this table:
[Table 1:](http://www.cmajopen.ca/content/9/2/E376/T1)

Table 1: 
Case definitions of heart failure used in the British Columbia node of the Canadian Primary Care Sentinel Surveillance Network

We compared each of these case definitions in BC-CPCSSN data against the HF cases identified using the manual chart review validation process. Case definitions were plotted by their sensitivity and PPV.

### Statistical analysis

We conducted a descriptive statistical analysis to summarize our study population’s demographic characteristics (age, sex), clinic characteristics (urban v. rural) and presence and number of other chronic conditions (for which there are existing validated definitions in CPCSSN). We calculated sensitivity, specificity, PPV and NPV. We organized the data into 2 × 2 tables comparing each case definition (case or no case) with the chart review diagnosis (case or no case). We constructed 95% confidence intervals (CIs) for each validity parameter using the Clopper–Pearson approach for proportions. Measures above 80% were considered acceptable for epidemiologic research.15 All data were analyzed in SAS version 9.4 (SAS Institute Inc.).

#### Sensitivity analysis

We performed a sensitivity test because of an unanticipated consequence of the timing of the chart reviews (September–November 2019) and the dates of data available for developing and testing the HF definition (up until Dec. 31, 2018). We hypothesized that some of the cases categorized as false negatives may have been incident in 2019, and so the algorithms would be unable to detect an HF diagnosis using data to the end of 2018. As a sensitivity test, we were able to examine BC-CPCSSN data to June 30, 2019 (2019-Q2) for a subset of the false-negative cases, to determine if they would meet the HF definition criteria with the addition of more up-to-date information.

### Ethics approval

This project fell within the scope of CPCSSN’s ongoing quality improvement efforts to improve the operationalization of the primary care network. Each clinician gave consent to access and use EMR data for their patients. BC-CPCSSN received ethics approval from the University of British Columbia Clinical Research Ethics Board, and this project is part of its ongoing quality improvement initiatives.

## Results

Among 649 patients with COPD from 3 primary care practices (with 14 participating CPCSSN physicians) in BC, 311 (113 [36.3%] men and 198 [63.7%] women) were randomly selected for full chart abstraction. Seventy-two (23.2%) were identified in chart review as having HF. The mean age of cases was 83.6 (standard deviation [SD] 10.9) years, and the mean age of noncases was 70.0 (SD 12.8) years. Thirty (41.7%) of the cases were men and 42 (58.3%) were women (Table 2).

View this table:
[Table 2:](http://www.cmajopen.ca/content/9/2/E376/T2)

Table 2: 
Patient characteristics

Table 3 and Appendix 4 (available at [www.cmajopen.ca/content/9/2/E376/suppl/DC1](http://www.cmajopen.ca/content/9/2/E376/suppl/DC1)) detail the accuracy parameters and counts for all tested case definitions in BC-CPCSSN, respectively. Overall, there was high sensitivity (range 75%–94.4%; this excludes definition 2.4 [Table 1]), specificity (range 87%–99.6%) and NPV (range 80.1%–98.1%). PPV had a greater variability, ranging from 68.4% to 92.9%. The addition of billing data (definitions 1.1 and 1.2) increased the sensitivity and NPV but decreased the specificity and PPV. Modification of the diagnosis codes and drugs included from Appendix 3, Appendix 4 and Appendix 5 (available at [www.cmajopen.ca/content/9/2/E376/suppl/DC1](http://www.cmajopen.ca/content/9/2/E376/suppl/DC1)) resulted in a mild decrease in performance (definition 1.1 v. definition 2.5), with a small reduction in specificity, sensitivity, PPV and NPV. Further adjustment in definition 3.2 built upon definition 2.5 with the inclusion of furosemide and billing data, which resulted in performance comparable to that of definition 1.1, with a much poorer PPV.

View this table:
[Table 3:](http://www.cmajopen.ca/content/9/2/E376/T3)

Table 3: 
Accuracy parameters of case definitions in the British Columbia node of the Canadian Primary Care Sentinel Surveillance Network

Our preferred case definition with optimal accuracy parameters was definition 1.1. This definition had an excellent specificity (93.3%, 95% CI 89.4%–96.1%), sensitivity (90.3%, 95% CI 81.0%–96.0%), PPV (80.2%, 95% CI 69.9%–88.3%) and NPV (97.0%, 95% CI 93.8%–98.8%).

Definitions 4 and 5 included BNP and NT-proBNP laboratory data. Of the 311 patients, 65 (20.9%) patients had BNP or NT-proBNP, of which 33 (50.8%) were cases and 32 (49.2%) were noncases from chart review. The inclusion of either low or high thresholds for BNP did not substantially alter or improve the accuracy of the case definition.

Definition 2.4 had excellent PPV but extremely low sensitivity. Most case definitions had high sensitivity with varying PPV as well as a high true-positivity rate with a low false-positivity rate (Figure 2). Definitions 1.1 and 5.1 had an excellent combination of sensitivity (90.3%, 91.7%) and PPV (80.2%, 79.5%).

![Figure 2:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/9/2/E376/F2.medium.gif)

[Figure 2:](http://www.cmajopen.ca/content/9/2/E376/F2)

Figure 2: 
True positivity rate versus false positivity rate (A) and sensitivity versus positive predictive value (B) for case definitions of heart failure in a population of patients with chronic obstructive pulmonary disease. The figure depicts all case definitions except definition 2.4. The arrows indicate that the 2 definitions have the same values on both axes. Note: BC-1 = definitions with expanding codes and testing iterations and specific combinations, BC-2 = definitions with the addition of furosemide, high NP = definitions including high-threshold natriuretic peptide, low NP = definitions including low-threshold natriuretic peptide, NS = original Nova Scotia Maritime Family Practice Research Network definitions. See Table 1 for details on the definitions.

For the sensitivity test, data from Jan. 1, 2019, to June 30, 2019, were examined for 10 patients who were false negatives for 1 or more of the case definitions. These 10 patients accounted for 94 (85.5%) of the 110 false negatives across all case definitions excluding definition 2.4. With the additional data, 2 of these patients met the criteria for case definitions that they did not otherwise meet using data to Dec. 31, 2018. This marginally increased the sensitivity, PPV and NPV for all of the case definitions except definitions 2.3 and 2.4 (Appendix 6, available at [www.cmajopen.ca/content/9/2/E376/suppl/DC1](http://www.cmajopen.ca/content/9/2/E376/suppl/DC1)).

## Interpretation

Overall, most of the case definitions of HF performed well on data in the BC-CPCSSN database and met our criteria for acceptable validity parameters. Identification of the best case definition for a particular investigation depends on the context and intent of the research. High sensitivity improves case findings as it more completely captures a population, increases the incidence and prevalence and enhances generalizability. This is relevant when researchers are attempting to estimate the burden of HF and to reduce bias when examining health inequities. High PPV improves identification of true cases of HF. Case definitions with low sensitivity or NPV will bias indicators toward the null, whereas those with low specificity or PPV will bias indicators away from the null.

Nonetheless, there are several different options with both high PPV and sensitivity that could be used to meet the surveillance and epidemiologic objectives of various stakeholders, as they allow for more complete case ascertainment. In this context, definition 1.1 would be the recommended case definition for HF among patients with COPD in the BC-CPCSSN database. Definition 5.1 also performed well, but the abstraction of and reliance on BNP data may vary among provincial networks. In addition, data on BNPs may not be collected from all patients with HF in a jurisdiction, creating a biased sample. In situations in which diagnostic certainty is needed, definition 2.4 is the preferred option as it has the highest specificity and PPV.

Our findings are comparable to those of other studies looking at HF case definitions in EMR databases and networks, although future validation in practice settings outside of CPCSSN would bolster the external validity of our results. In a primary care EMR database in the US, reported sensitivities for an HF case definition ranged from 41.3% to 78.7% and PPVs ranged from 68.5% to 86.5%.13 Our case definitions performed slightly better. In Ontario, the Electronic Medical Record Administrative data Linked Database (EMRALD) was used to validate HF case definitions. The optimal case definition included 1 hospital visit or physician billing visit and a second physician billing visit within 1 year. This yielded a sensitivity of 84.8%, a specificity of 97% and a PPV 55.7%.12 Our preferred case definition has better sensitivity and PPV and slightly lower, but still acceptable, specificity. This highlights the utility of the medication codes in CPCSSN compared with those in EMRALD.12 Lastly, the accuracy parameters of our HF definition are similar to those of other validated case definitions in CPCSSN.16

Our findings have several implications for future research. Our case definition provides a relatively accurate sample of patients with COPD and HF in the BC-CPCSSN. Subsequent analyses will provide insights into the contemporary management of these patients in primary care and explore HF classification in terms of reduced versus preserved ejection fraction. This case definition will permit epidemiologic estimates to be made of the prevalence of HF in COPD across Canada. Our findings will support screening for cardiovascular risk and treatment of cardiovascular disease in patients with COPD in primary care. Lastly, this work has instigated the development of a Primary Care Clinician Advisory Group to guide further projects in BC-CPCSSN. This will enable a collaborative integrated knowledge translation strategy to be undertaken and will inform the design of future epidemiologic and intervention studies, as well as quality improvement initiatives.

There are several strengths to our study. We used an excellent gold standard reference in the form of chart review by 2 medical professionals who used a standardized chart abstraction form that minimized misclassification and instrument bias. A variety of case definitions were tested with minimal alterations to the accuracy parameters, providing evidence that a simple definition is most robust in accurately identifying HF in patients with COPD. Finally, our case definitions used a variety of parameters beyond billing codes, including medication codes and laboratory data.

### Limitations

This work is not without limitations. The chart validation occurred in 1 province, where clinicians may record data more similarly than those in another province. Data abstraction was performed by a single person (a public health and family medicine resident), and this work was reviewed by a cardiologist. The case definition may perform differently with other clinicians and in other settings.

Data were abstracted from various EMR systems; there are differences in how EMRs record data (e.g., ICD-9 codes may be recorded in the health condition and encounter tables or the billing table). The data in this study are for patients with COPD from 3 clinics and may not be representative of all patients with COPD and HF in BC. In addition, the 9 clinicians who participated in the study may have different practices than other clinicians.

Chart reviews were performed in late 2019; data available for developing and testing the HF definition were to Dec. 31, 2018. Thus, patients with newly diagnosed HF in 2019 might not have been detectable in the data available for algorithm development (i.e., the algorithms would not have been able to see the relevant data from 2019). Our sensitivity test found marginal improvement in validity parameters with the addition of a subset of 2019 data; however, in future studies there should be better correspondence between the dates of the data used to develop the case definitions and the dates of the chart review. Lastly, use of our case definitions outside of this selected patient population with COPD is not warranted. Further validation of our preferred case definition in the unselected HF population in CPCSSN is underway.

### Conclusion

Our study provides valid case definitions for HF in patients with COPD in the pan-Canadian CPCSSN database. Several different case definitions were constructed and tested with excellent performance in the BC-CPCSSN database. With further external validation of this algorithm, the findings of this study will support ongoing research activities, chronic disease surveillance, and quality improvement initiatives in primary care for HF among people with COPD across Canada.

## Acknowledgements

The BC Canadian Primary Care Sentinel Surveillance Network would like to thank primary care clinicians Drs. Cameron Ross, Nardia Strydom and Ilona Hale for facilitating access to their practices, including access to their physician colleagues. The authors would also like to acknowledge the Maritime Family Practice Research Network team for developing the initial heart failure case definition that was used in our study. The authors thank Dr. John Queenan, Canadian Primary Care Sentinel Surveillance Network epidemiologist, for his contributions to sample size calculation for the chart review.

## Footnotes

*   **Competing interests:** None declared.

*   This article has been peer reviewed.

*   **Contributors:** Sabrina Wong, Matthew Grandy and Nathaniel Hawkins conceived and designed the study. Rohit Vijh, Sandra Peterson and Andrew Gibb acquired the data, which Rohit Vijh and Sandra Peterson analyzed. All authors contributed to the data interpretation. Rohit Vijh drafted the manuscript and all authors subsequently revised it critically for important intellectual content. All authors gave final approval of version to be published and agreed to be accountable for all aspects of the work.

*   **Funding:** The project was funded through the UBC Division of Cardiology Academic Practice Plan.

*   **Data sharing:** The data used in this study are available through the BC-CPCSSN. A data access request would need to be completed, along with an application. University-based researchers are eligible to apply for access to these data.

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/9/2/E376/suppl/DC1](http://www.cmajopen.ca/content/9/2/E376/suppl/DC1).

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: [https://creativecommons.org/licenses/by-nc-nd/4.0/](https://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  Vos T, Flaxman AD, Naghavi M, et al. (2012) Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 380:2163–96.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0140-6736(12)61729-2&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23245607&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000312387000015&link_type=ISI) 

2.  Cook C, Cole G, Asaria P, et al. (2014) The annual global economic burden of heart failure. Int J Cardiol 171:368–76.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.ijcard.2013.12.028&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24398230&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 

3.  Hawkins NM, Petrie MC, Jhund PS, et al. (2009) Heart failure and chronic obstructive pulmonary disease: diagnostic pitfalls and epidemiology. Eur J Heart Fail 11:130–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/eurjhf/hfn013&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19168510&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 

4.  Hawkins NM, Petrie MC, MacDonald MR, et al. (2011) Heart failure and chronic obstructive pulmonary disease: the quandary of beta-blockers and beta-agonists. J Am Coll Cardiol 57:2127–38.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czo0OiJhY2NqIjtzOjU6InJlc2lkIjtzOjEwOiI1Ny8yMS8yMTI3IjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby85LzIvRTM3Ni5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

5.  Koudstaal S, Pujades-Rodriguez M, Denaxas S, et al. (2017) Prognostic burden of heart failure recorded in primary care, acute hospital admissions, or both: a population-based linked electronic health record cohort study in 2.1 million people. Eur J Heart Fail 19:1119–27.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 

6.  Khand AU, Shaw M, Gemmel I, et al. (2005) Do discharge codes underestimate hospitalisation due to heart failure? Validation study of hospital discharge coding for heart failure. Eur J Heart Fail 7:792–7.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.ejheart.2005.04.001&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16054867&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000236408500014&link_type=ISI) 

7.  Schellenbaum GD, Heckbert SR, Smith NL, et al. (2006) Congestive heart failure incidence and prognosis: case identification using central adjudication versus hospital discharge diagnoses. Ann Epidemiol 16:115–22.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.annepidem.2005.02.012&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15964203&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000235553400007&link_type=ISI) 

8.  Ingelsson E, Ärnlöv J, Sundström J, et al. (2005) The validity of a diagnosis of heart failure in a hospital discharge register. Eur J Heart Fail 7:787–91.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.ejheart.2004.12.007&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15916919&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000236408500013&link_type=ISI) 

9.  Saczynski JS, Andrade SE, Harrold LR, et al. (2012) A systematic review of validated methods for identifying heart failure using administrative data. Pharmacoepidemiol Drug Saf 21:129–40.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/pds.2313&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22262599&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 

10. Quach S, Blais C, Quan H (2010) Administrative data have high variation in validity for recording heart failure. Can J Cardiol 26:306–12.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0828-282X(10)70438-4&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20931099&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 

11. Lee DS, Johansen H, Gong Y, et al., Canadian Cardiovascular Outcomes Research Team (2004) Regional outcomes of heart failure in Canada. Can J Cardiol 20:599–607.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15152289&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000221425200001&link_type=ISI) 

12. Schultz SE, Rothwell DM, Chen Z, et al. (2013) Identifying cases of congestive heart failure from administrative data: a validation study using primary care patient records. Chronic Dis Inj Can 33:160–6.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.24095/hpcdp.33.3.06&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23735455&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 

13. Patel YR, Robbins JM, Kurgansky KE, et al. (2018) Development and validation of a heart failure with preserved ejection fraction cohort using electronic medical records. BMC Cardiovasc Disord 18:128.
    
    

14. Tison GH, Chamberlain AM, Pletcher MJ, et al. (2018) Identifying heart failure using EMR-based algorithms. Int J Med Inform 120:1–7.
    
    

15. McCormick N, Lacaille D, Bhole V, et al. (2014) Validity of heart failure diagnoses in administrative databases: a systematic review and meta-analysis. PLoS One 9:e104519.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1371/journal.pone.0104519&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25126761&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 

16. Williamson T, Green ME, Birtwhistle R, et al. (2014) Validating the 8 CPCSSN case definitions for chronic disease surveillance in a primary care database of electronic health records. Ann Fam Med 12:367–72.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTIvNC8zNjciO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzkvMi9FMzc2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

17. Birtwhistle R, Keshavjee K, Lambert-Lanning A, et al. (2009) Building a pan-Canadian primary care sentinel surveillance network: initial development and moving forward. J Am Board Fam Med 22:412–22.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjIvNC80MTIiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzkvMi9FMzc2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

18. Williamson T, Miyagishima RC, Derochie JD, et al. (2017) Manual review of electronic medical records as a reference standard for case definition development: a validation study. CMAJ Open 5:E830–3.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiY21ham8iO3M6NToicmVzaWQiO3M6ODoiNS80L0U4MzAiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzkvMi9FMzc2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

19. Ezekowitz JA, O’Meara E, McDonald MA, et al. (2017) 2017 comprehensive update of the Canadian Cardiovascular Society guidelines for the management of heart failure. Can J Cardiol 33:1342–433.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 

20. Ponikowski P, Voors AA, Anker SD, et al. (2016) 2016 ESC guidelines for the diagnosis and treatment of acute and chronic heart failure: The Task Force for the Diagnosis and Treatment of Acute and Chronic Heart Failure of the European Society of Cardiology (ESC). Developed with the special contribution of the Heart Failure Association (HFA) of the ESC. Eur J Heart Fail 18:891–975.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/ejhf.592&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=27207191&link_type=MED&atom=%2Fcmajo%2F9%2F2%2FE376.atom) 

21. Yancy CW, Jessup M, Bozkurt B, et al. (2013) 2013 ACCF/AHA guideline for the management of heart failure: executive summary: a report of the American College of Cardiology Foundation/American Heart Association Task Force on practice guidelines. Circulation 128:1810–52.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6ImNpcmN1bGF0aW9uYWhhIjtzOjU6InJlc2lkIjtzOjExOiIxMjgvMTYvMTgxMCI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vOS8yL0UzNzYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

*   © 2021 Joule Inc. or its licensors