Abstract
Background: Accurate identification of maternal deaths is paramount for audit and policy purposes. Our aim was to determine the accuracy and completeness of data on maternal deaths in hospital and those recorded on a death certificate, and the level of agreement between the 2 data sources.
Methods: We conducted a retrospective population-based study using data for Ontario, Canada, from Apr. 1, 2002, to Dec. 31, 2015. We used Canadian Institute for Health Information (CIHI) databases to identify deaths during inpatient, emergency department and same-day surgery encounters. We captured Vital Statistics deaths in the Office of the Registrar General, Deaths (ORGD) data set. Deaths were considered within 42 days and within 365 days after a pregnancy outcome (live birth, miscarriage, ectopic pregnancy or induced abortion) for all multiple and singleton pregnancies. We calculated agreement statistics and 95% confidence intervals (CIs).
Results: Among 1 679 455 live births and stillbirths, 398 pregnancy-related deaths in the ORGD data set were mapped to a birth in CIHI databases, and 77 (16.2%) were not. Among 2 039 849 recognized pregnancies, 534 pregnancy-related deaths in the ORGD data set were linked to CIHI records, and 68 (11.3%) were not. Among live births and stillbirths, after pregnancy-related deaths in the ORGD data set not matched to a maternal death in the CIHI databases were removed, concordance measures between CIHI and ORGD records for maternal death within 42 days after delivery included a κ value of 0.87 (95% CI 0.82–0.91) and positive percent agreement of 0.88 (95% CI 0.83–0.94). The corresponding measures were similar for maternal death within 42 days after the end of a recognized pregnancy. When unlinked pregnancy-related deaths in the ORGD data set were retained, agreement measures declined for death within 42 days after a live birth or stillbirth (κ = 0.68, 95% CI 0.62–0.74). For maternal death within 365 days after a live birth or stillbirth, or after the end of a recognized pregnancy, the concordance statistics were generally favourable when unlinked pregnancy-related deaths in the ORGD data set were removed but were substantially declined when they were retained.
Interpretation: Maternal mortality cannot be ascertained solely with the use of hospital data, including beyond 42 days after the end of pregnancy. To improve linkage, we propose including health insurance numbers on provincial and territorial medical death certificates.
Maternal death, although rare, is an often preventable event with devastating consequences for families and care providers. The World Health Organization estimates that more than 300 000 women die each year from pregnancy- or childbirth-related complications.1 The World Health Organization and the Public Health Agency of Canada reported a rise in maternal mortality in Canada between 1990 and 2015, from 5.1 to 11.9 per 100 000 live births.2,3
These findings prompted the Society of Obstetricians and Gynaecologists of Canada to work with federal and provincial partners to review maternal mortality surveillance.4 The investigating committee found that national maternal health surveillance faced serious barriers relating to data access, coverage, timeliness and completeness. Moreover, they concluded that there was little standardization across jurisdictions with respect to definitions, data collection and maternal mortality review processes.4,5 Although several provinces and territories have since established maternal death review committees, there is no national system to synthesize and report on maternal mortality and, thus, no accurate national picture of mortality prevalence and trends.3,4 This is especially so for maternal deaths occurring before the conventional birthing period, such as with an ectopic pregnancy, induced abortion or miscarriage, and those occurring up to 42 days after delivery.
Since Ontario contributes about 35% of all births in Canada,6,7 there arises an opportunity to examine the systems responsible for monitoring and recording maternal mortality. Accordingly, we conducted the current study to determine the accuracy and completeness of data on maternal deaths in hospital and those recorded on a death certificate, and the level of agreement between the 2 data sources by taking into consideration not only births, but also ectopic pregnancies, induced abortions and miscarriages, as well as capturing maternal deaths up to 365 days after the end of a pregnancy.
Methods
Study design and data sources
We performed this retrospective population-based study using data for the entire province of Ontario, Canada. We used existing data sets linked by means of unique encoded identifiers and analyzed at ICES. We used the Ontario Office of the Registrar General, Deaths (ORGD) data set to identify deaths based on a Medical Certificate of Death (Form 16), completed by a physician.8 The ORGD shares Vital Statistics death data with Statistics Canada for national reporting. The ORGD data set includes date and location of death and International Statistical Classification of Diseases and Related Health Problems, 10th Revision (ICD-10) codes for cause of death.
We identified deaths that occurred in a hospital setting using the Canadian Institute for Health Information (CIHI) Discharge Abstract Database (DAD), National Ambulatory Care Reporting System (NACRS) database and Same Day Surgery (SDS) database.9 The DAD contains administrative, clinical and demographic information pertaining to hospital discharges (i.e., deaths, sign-outs and transfers). Diagnostic codes are based on the Canadian version of the ICD-10 (ICD-10-CA), and procedural codes are based on the Canadian Classification of Health Interventions.10 The NACRS database contains information on hospital- and community-based emergency and ambulatory care visits, and the SDS database contains patient-level data for day surgery institutions in Ontario.10,11
Participants
We identified live births and stillbirths in the DAD and the NACRS database. We used the DAD, the NACRS database and the SDS database to capture all recognized pregnancies, namely, a live birth, a stillbirth at 20 weeks’ gestation or later, a miscarriage before 20 weeks’ gestation, an ectopic pregnancy or an induced abortion for all multiple and singleton pregnancies. We used the ORGD data set to identify the deaths of females within 365 days after the end of a recognized pregnancy. We further used an ICD-10 cause of death code starting with “O” to identify a pregnancy-related death.
We included female Ontario residents aged 10–55 years at the time of a documented encounter for a recognized pregnancy outcome in the CIHI databases or with a registered death in the ORGD data set, for the period Apr. 1, 2002, to Dec. 31, 2015. We excluded records with an invalid Ontario Health Insurance Plan number or hospitalization number, a non-Ontario postal code at the time of delivery or death, or nonunique patient or hospital record identifiers. Continued residency in Ontario between the documented encounter date for a recognized pregnancy outcome and death was not a requirement for study inclusion. The CIHI and ORGD records were then linked by means of unique encrypted personal identifiers and date of death between 0 and 365 days after the pregnancy outcome date.
Statistical analysis
Among females who had a live birth or stillbirth in the CIHI databases and a death record in the ORGD data set, we described cause of death for those who died within 42 days after delivery, as well as from 43 to 365 days after delivery.
We generated agreement statistics (and 95% confidence intervals [CIs]) from 2 × 2 tables describing the concordance between deaths identified in the CIHI databases and those in the ORGD data set, first for deaths among females with a live birth or stillbirth, and then among all recognized pregnancies, with consideration of deaths within 42 days and within 365 days of the end of the pregnancy. Agreement measures included Cohen κ, positive predictive value, negative predictive value, overall percent agreement, positive percent agreement and negative percent agreement. Overall percent agreement, positive percent agreement and negative percent agreement (calculations shown in Appendix 1, Table S1, available at www.cmajopen.ca/content/9/2/E539/suppl/DC1) are generally used in the absence of a perfect reference standard,12 as might be the case when using ORGD data, if a pregnancy is unrealized or maternal death is beyond 42 days after delivery. The agreement measures did include maternal deaths in the ORGD data set that were not linked to a recognized pregnancy in the CIHI databases.
Ethics approval
The use of data in this project was authorized under section 45 of Ontario’s Personal Health Information Protection Act, which does not require review by a research ethics board.
Results
There were 2 039 849 recognized pregnancies in the study (Table 1), including 1 679 455 live birth or stillbirth deliveries identified in the DAD or NACRS database (Figure 1). We matched these 1 679 455 records to an identified maternal death in the ORGD data set (Figure 1). There were 398 pregnancy-related deaths in the ORGD data set that could be linked to a birth within the preceding 365 days in the DAD or NACRS database; 77 pregnancy-related deaths (16.2%) had no birth record in the CIHI databases (Figure 1). Cause of death within 42 days after the end of pregnancy, classified by the 3-digit ICD-10 code, is shown in Appendix 1, Table S2, and that for deaths between 43 and 365 days after the end of pregnancy is shown in Appendix 1, Table S3.
There were 2 039 849 recognized pregnancies in the DAD, NACRS database or SDS database, resulting in 1 668 936 live births (81.8%), 10 069 stillbirths (0.5%), 217 474 miscarriages or ectopic pregnancies (10.7%), and 143 370 induced abortions (7.0%). A total of 534 pregnancy-related deaths in the ORDG data set could be linked to the CIHI databases, and 68 (11.3%) could not (Figure 2).
Measures of concordance
Among live births and stillbirths, after pregnancy-related deaths in the ORGD data set not matched to a maternal death in the CIHI databases were removed, concordance measures for maternal death within 42 days after delivery were high (Table 2). For example, the Cohen κ was 0.87 (95% CI 0.82–0.91), positive percent agreement 0.88 (95% CI 0.83–0.94) and negative percent agreement 1.00 (95% CI 1.00–1.00). For maternal deaths within 42 days after the end of a recognized pregnancy, the corresponding measures were 0.82 (95% CI 0.78–0.87), 0.84 (95% CI 0.78–0.90) and 1.00 (95% CI 1.00–1.00).
When unlinked pregnancy-related deaths in the ORGD data set were retained, however, agreement measures declined for maternal death within 42 days after a live birth or stillbirth (Cohen κ = 0.68, 95% CI 0.62–0.74; positive percent agreement 0.57, 95% CI 0.50–0.63) (Table 3). The same pattern was seen for maternal death within 42 days after the end of a recognized pregnancy (Table 3).
For maternal death within 365 days after a live birth or stillbirth, or within 365 days after the end of a recognized pregnancy, the concordance statistics were generally favourable when unlinked ORGD pregnancy-related deaths were removed (Table 4) but declined substantially when they were retained (Table 5).
Interpretation
In this population-based study, about 16% of maternal deaths following a live birth or stillbirth, and 11% of deaths after a recognized pregnancy (i.e., live birth, stillbirth, ectopic pregnancy, miscarriage or induced abortion) were not captured during a hospital encounter in CIHI databases when they were documented in the ORGD record. When linkage between the CIHI and ORGD records was possible, there was high concordance for maternal death, which declined when nonlinked cases were also considered.
A complete picture (the “universe”) of maternal deaths in Canada is lacking.13 Nearly 60% of all deaths in Ontario between 2001 and 2015 occurred in a hospital setting.14 Our findings suggest that maternal mortality cannot be ascertained solely with the use of hospital data and highlight the need for improved methodology to effectively capture maternal mortality beyond 42 days after the end of pregnancy.
It would be challenging to develop a national enquiry system for maternal mortality, given Canada’s province-based approach to health care administration. Nevertheless, Canadian provinces and territories can, and do, collaborate in the design, development and collection of national disease surveillance.4 Replication of the current study methods in other provinces and the territories would be one step in achieving a better perspective about how maternal deaths can be captured accurately and efficiently; this should include input from the chief coroner of each province and territory.
To ascertain more accurate and complete information on maternal mortality, and to enable deterministic linkage between hospital and Vital Statistics data sources, we suggest amending provincial and territorial death certificates to include a fillable field for a provincial or territorial health insurance plan number, such as the Ontario Health Insurance Plan number in Ontario. We also suggest that all pregnancies be captured — regardless of how the pregnancy ends — with the inclusion of women who die while still pregnant.
Limitations
Not all miscarriages or induced abortions are captured by the data sets used in this study.15,16 To parallel the data sets currently available to the Canadian Perinatal Surveillance System, we opted to use only the CIHI and Vital Statistics files. A broader data set might also include data such as provincial billing information, which would enhance the capture of induced abortion codes.15,16 For example, in a study from British Columbia, Canada, the sensitivity of identifying induced abortion in the DAD was 91.9%; this increased to 99.1% when physician billing codes were also used.16 Furthermore, out-of-hospital births are not documented completely in the current administrative databases.17 Pregnancy-related deaths in the ORGD data set not captured by CIHI may be attributable to deaths during home delivery or out-of-hospital maternal deaths.
A second major limitation is that there were inherent problems in creating the current data linkages. In Ontario, a Medical Certificate of Death does not include a fillable field for an Ontario Health Insurance Plan number, which serves as a unique identifier for all residents of Ontario. Matching maternal deaths in the Vital Statistics registry to hospitalization information depended on a probabilistic linkage method, which uses nonunique linkage keys, such as first and last name combinations and date of birth. Such an approach is prone to false-positive and false-negative linkages.18 For example, if we were unable to link a hospital record to an existing Vital Statistics record with certainty, the hospital record was deemed a nonmatch.
The age of our data represents a limitation to our study, as the comprehensiveness of CIHI data holdings may have improved since 2015.19 In addition, ICD-10-CA diagnostic codes include up to 6 characters in hospital data, whereas only the first 4 characters are used in Vital Statistics data, based on ICD-10 codes. Furthermore, Canadian provinces differentially switched from the Canadian version of the International Classification of Diseases, 9th Revision to ICD-10-CA over an 8-year period, starting around 2000. This staggered introduction of ICD-10-CA coding may have affected the quality of data available in provincial hospital administrative data, limiting the generalizability of our findings to some parts of Canada.
Births occurring outside of a traditional health care setting, such as a birthing centre or private residence, are not captured in the DAD or the NACRS or SDS databases, and would not be included in the current study. In addition, maternal deaths occurring outside of a health care setting may not have been documented in the ORGD data set. Last, deaths within the CIHI databases are not linkable for people without a registered provincial health insurance plan number, which might miss maternal deaths among temporary visa holders, recent refugees and undocumented residents, the majority of whom reside in larger urban centres, including Toronto.20–22 Hence, the true scope of maternal mortality in Ontario may be higher than that estimated in the current study.
Conclusion
Vital Statistics death records likely capture the scope of maternal deaths in Ontario more accurately and more completely than the use of CIHI hospital records alone. To improve the quality of data linkage, we propose amending provincial and territorial medical death certificates to include fields for health insurance numbers, and the capture of all deaths during or after pregnancy.
Footnotes
Competing interests: None declared.
This article has been peer reviewed.
Contributors: Chantal Nelson, Alison Park and Joel Ray conceived of the study. Chantal Nelson, Wei Luo, Alison Park and Joel Ray designed the study. Kayvan Aflaki, Alison Park and Joel Ray drafted the manuscript. All of the authors contributed to data analysis and interpretation, revised the manuscript critically for important intellectual content, approved the final version to be published and agreed to be accountable for all aspects of the work.
Funding: This study was funded by the Public Health Agency of Canada and was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health.
Data sharing: The data set from this study is held securely in coded form at ICES. Although data-sharing agreements prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at https://www.ices.on.ca/DAS. The full data set creation plan and underlying analytic code are available from the authors on request, with the understanding that the computer programs may rely on coding templates or macros that are unique to ICES and are therefore inaccessible or may require modification.
Disclaimer: Parts of this material are based on data and information compiled and provided by the Ontario Ministry of Health and the Canadian Institute for Health Information. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred. Parts of this report are based on Ontario Registrar General (ORG) information on deaths, the original source of which is ServiceOntario. The views expressed herein are those of the authors and do not necessarily reflect those of the ORG or the Ministry of Government Services.
Supplemental information: For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/9/2/E539/suppl/DC1.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/
References
- © 2021 CMA Joule Inc. or its licensors