Using OHIP physician billing claims to ascertain individual influenza vaccination status

doi:10.1016/j.vaccine.2006.10.004

Vaccine

Volume 25, Issue 7, 26 January 2007, Pages 1270-1274

https://doi.org/10.1016/j.vaccine.2006.10.004 Get rights and content

Abstract

The objective of this study was to validate physician billing claims against self-reported influenza vaccination to assess individual-level vaccination status. We compared responses to the Canadian Community Health Survey 1.1 (CCHS) and Ontario Health Insurance Plan (OHIP) physician billing claims and found moderate agreement. Using self-report as the gold standard, OHIP claims based on using both influenza-specific and general vaccination codes have high specificity and positive predictive value (PPV), reasonable negative predictive value (NPV), but only fair sensitivity. OHIP physician billing claims are suboptimal for ascertaining the vaccination status of individuals because many individuals receive their vaccinations outside doctor's offices, but may be used as the backbone for the creation of an immunization registry.

Introduction

In 2000, Ontario introduced the world's first large-scale Universal Influenza Immunization Program (UIIP) to provide free influenza vaccines for the entire population 6 months of age or older [1]. Efforts to evaluate the benefits of the UIIP have been ecological in nature because the absence of an immunization registry has precluded the ability to accurately ascertain the vaccination status of Ontario residents. Studies using individual-level health administrative data may permit controlling for patient-level confounders, such as co-morbidities, whereas ecological studies are susceptible to the ecological fallacy [2].

Ontario has universal, publicly insured health care services. Physicians bill the provincial government for services performed by themselves or by delegates such as nurses, therefore billing claims represent a possible data source to assess the vaccination status of individuals. Comparison of physician billing claims with self-reported vaccination is possible using the Canadian Community Health Survey (CCHS) 1.1 Linked File for Ontario, which permits individual-level linkage of survey responses to use of publicly insured health care services. Previous studies have found self-reported influenza vaccination status to be valid [3], [4], [5], [6]. However, a survey conducted after the 2000–2001 influenza season revealed that only about 65% of respondents received their “flu shot” at a doctor's office (personal communication, Anne-Luise Winter, Ministry of Health and Long Term Care), therefore Ontario Health Insurance Plan (OHIP) billing claims are likely to underestimate true vaccination rates. Other vaccine delivery settings include schools, workplaces, pharmacies, community centers and shopping malls.

The objective of this study was to validate the use of OHIP physician billing claims to assess the influenza vaccination status of individuals against self-reported influenza vaccination status as captured by a survey, and to determine the extent of misclassification.

Section snippets

Study population and setting

This study included Ontario residents who were surveyed in CCHS cycle 1.1 from January to August 2001 (Quarters 2–4) and agreed to have their survey responses linked with provincial health administrative datasets.

Canadian Community Health Survey

The CCHS is a biennial cross-sectional survey conducted through telephone and in-person interviews. The survey covers the household population aged 12 years or older and excludes members of the Canadian Forces and residents of Canadian Forces bases, Indian reserves and some remote

Descriptive statistics

The characteristics of the 22,164 respondents included in this study are presented in Table 1. Among these respondents, 36.7% reported receiving influenza vaccination in the past year. Vaccination rates were higher among females, older age groups and those with chronic conditions.

Agreement between OHIP claims and CCHS responses using kappa statistic

The kappa statistics comparing OHIP claims and CCHS responses indicated moderate agreement, with overall values of 0.52 for definition #1 (using only influenza-specific vaccination codes) and 0.58 for definition #2

Discussion

We found moderate agreement between self-reported immunization status using health survey data from CCHS 1.1 and OHIP physician billing claims. If self-reported vaccination status is used as the gold standard, OHIP claims using both influenza-specific and general vaccination codes have high specificity and positive predictive value, reasonable negative predictive value, but only fair sensitivity. Most discordant individuals reported having received the vaccine in CCHS but were not found in the

References (10)

R. Mac Donald et al.
Validation of self-report of influenza and pneumococcal vaccination status in elderly outpatients
Am J Prev Med
(1999)
L.M. Martin et al.
Validation of self-reported chronic conditions and health services in a managed care population
Am J Prev Med
(2000)
K.L. Nichol et al.
Estimation of outpatient risk characteristics and influenza vaccination status: validation of a self-administered questionnaire
Am J Prev Med
(1991)
Canada NewsWire. Ontario invests $38 million to ease emergency room pressures with universal vaccination program....
J.M. Last
A dictionary of epidemiology
(2001)

There are more references available in the full text version of this article.

Cited by (23)

The safety of seasonal influenza vaccination among adults prescribed immune checkpoint inhibitors: A self-controlled case series study using administrative data
2024, Vaccine
Immune checkpoint inhibitor (ICI) therapy for patients undergoing cancer treatment carries a risk of severe immune-related adverse events (IRAEs). Questions remain about whether seasonal influenza vaccination might increase the risk of developing IRAEs among these patients given that vaccines are immunomodulatory. Previous vaccine safety studies on patients with cancer prescribed ICI therapy have demonstrated conflicting results.
Using health administrative data from Ontario, Canada among adults diagnosed with cancer who had been prescribed ICI therapy and who had received an influenza vaccine from 2012 to 2019, we conducted a self-controlled case series study. The pre-vaccination control period started 42-days post-ICI initiation until 14-days prior to vaccination, the risk period was 1–42 days post-vaccination, and the post-vaccination control period was after the risk period until ICI discontinuation or a maximum period of two years. Emergency department (ED) visit(s) and/or hospitalization for any cause after ICI initiation was used to identify severe IRAEs. We fitted a fixed-effects Poisson regression model accounting for seasonality and calendar time to estimate relative incidence of IRAEs between risk and control periods.
We identified 1133 records of cancer patients who received influenza vaccination while prescribed ICI therapy. Most were aged ≥ 66 years (73 %), were male (63 %), had lung cancer (54 %), and had received ICI therapy with a programmed cell death protein 1(PD-1) inhibitor (91 %). A quarter (26 %) experienced an ED visit and/or hospitalization during the observation period. Rates of ED visits and/or hospitalizations in the risk vs. control periods were similar, with an incidence rate ratio of 1.04 (95 % CI: 0.75–1.45). Subgroup and sensitivity analyses yielded similar results.
Seasonal influenza vaccination was not associated with an increased incidence of ED visit or hospitalization among adults with cancer treated with ICI therapy and our results support further evidence of vaccine safety.
Estimating age-specific influenza-associated asthma morbidity in Ontario, Canada
2019, Respiratory Medicine
Citation Excerpt :
Influenza vaccination can be delivered in a variety of settings, and since there is no provincial registry recording influenza vaccinations, only a proportion of these are captured in health administrative databases. For that reason, Ontario health administrative data are not optimal for ascertaining individual vaccination status [25,26]. In a subset of the population for whom national health survey data were available, it was determined that 44% of individuals in Ontario who reported having asthma also reported having received an influenza vaccination in the prior 12 months [26].
There is a need to quantify the potential benefits of influenza-focused interventions in reducing asthma morbidity at a population level. This study aims to estimate age-specific annual excess asthma morbidity attributable to influenza in Ontario, Canada.
Weekly counts of hospitalizations, emergency department (ED) visits and outpatient physician office visits for asthma were obtained from health administrative data in Ontario from 2010 to 2015, for ages 0–14, 15–59 and 60+. Asthma morbidity was modelled as a function of influenza A and B activity using linear regression, controlling for seasonal and long-term trend, mean temperature and respiratory syncytial virus. Excess asthma morbidity attributable to influenza was calculated as the difference between full model predictions and model predictions with influenza A and B variables set to 0.
Annually, influenza was associated with the following rates of excess asthma morbidity, per 100,000 people with prevalent asthma: 12.5 hospitalizations for ages 15–59 (95% confidence interval (CI): 1.1–23.5); 35.7 hospitalizations for ages 60+ (95% CI: 3.3–67.1); 114.1 ED visits for ages 15–59 (95% CI: 46.9–181.6); 154.6 ED visits for ages 60+ (95% CI: 86.7–223.3); and 1025.7 outpatient physician office visits for ages 60+ (95% CI: 79.0–1877.3).
Influenza was associated with excess asthma hospitalizations and ED visits for ages 15–59 and 60+ and outpatient physician office visits for ages 60+. Individuals with asthma aged 15–59 and 60+ might be important targets for influenza-focused interventions, to reduce asthma morbidity at the population level.
The Relationship Between Cardiologist Care and Clinical Outcomes in Patients With New-Onset Atrial Fibrillation
2017, Canadian Journal of Cardiology
Citation Excerpt :
The proportion of patients who saw family physicians and noncardiology internal medicine specialists during the follow-up period was determined. Additional processes of care were identified using previously validated approaches (Supplemental Table S2).15,16 Specifically, the rates of cardiac procedures, including echocardiography, stress testing, coronary angiography, and coronary artery revascularization, were determined for the entire matched cohort.
Increased mortality is well described in patients with atrial fibrillation (AF), primarily related to death from cardiovascular causes. One may hypothesize that cardiology care may be associated with a reduction in cardiovascular deaths in patients with AF, thereby improving their overall survival. The aim of this study was to assess the association between cardiologist care and clinical outcomes, including all-cause mortality, in patients with new-onset AF.
This was a retrospective population-level, propensity score–matched cohort study of patients aged 20-80 years with new-onset AF presenting to an emergency department in Ontario, Canada between 2010 and 2012. Patients who saw a cardiologist within 1 year of the initial diagnosis were matched to patients who did not see a cardiologist. Linked administrative databases were used for cohort construction and allow for 1-year follow-up to assess for the clinical end points of death, hospitalization for AF, stroke syndromes, bleeding, and heart failure.
Cardiologist care was associated with a lower 1-year rate of death (5.3% vs 7.7%; hazard ratio, 0.68; 95% confidence interval, 0.55-0.84), despite an increased rate of hospitalizations for AF (17.9% vs 8.2%), stroke syndromes (1.7% vs 0.5%), bleeding (3.1% vs 2.0%), and heart failure (3.2% vs 1.4%).
Cardiologist care was associated with a reduction in death in patients with new-onset AF. Further study to obtain a greater understanding of the processes of care associated with the observed survival improvement is warranted.
L’augmentation de la mortalité est bien décrite chez les patients atteints de fibrillation auriculaire (FA), principalement liée au décès d’origine cardiovasculaire. On peut émettre l’hypothèse que les soins en cardiologie peuvent être associés à une réduction des décès d’origine cardiovasculaire chez les patients atteints de FA et, par conséquent, améliorer leur survie globale. L’objectif de la présente étude était d’évaluer l’association entre les soins des cardiologues et les résultats cliniques, y compris la mortalité toutes causes confondues chez les patients atteints d’une FA d’apparition récente.
Il s’agissait d’une étude de cohorte populationnelle rétrospective appariée par score de propension regroupant des patients de 20 à 80 ans atteints d’une FA d’apparition récente qui s’étaient présentés dans un service des urgences de l’Ontario, au Canada, entre 2010 et 2012. Nous avons apparié les patients qui avaient vu un cardiologue durant l’année du diagnostic initial aux patients qui n’avaient pas vu de cardiologue. Nous avons utilisé des bases de données administratives liées pour former la cohorte et prévu 1 an de suivi pour évaluer les critères de jugements cliniques de mortalité, d’hospitalisations en raison de FA, de syndromes d’accident vasculaire cérébral, d’hémorragies et d’insuffisance cardiaque.
Nous avons associé les soins des cardiologues à un taux de mortalité plus faible après 1 an (5,3 % vs 7,7 % ; rapport de risque, 0,68 ; intervalle de confiance à 95 %, 0,55-0,84), en dépit d’une augmentation du taux d’hospitalisation en raison de FA (17,9 % vs 8,2 %), des syndromes d’accidents vasculaires cérébraux (1,7 % vs 0,5 %), des hémorragies (3,1 % vs 2,0 %) et de l’insuffisance cardiaque (3,2 % vs 1,4 %).
Nous avons associé les soins des cardiologues à une réduction de la mortalité chez les patients atteints d’une FA d’apparition récente. Des études subséquentes pour acquérir une meilleure compréhension des processus de soins associés à l’observation d’une amélioration de la survie sont justifiées.
Comparison of self-report influenza vaccination coverage with data from a population based computerized vaccination registry and factors associated with discordance
2014, Vaccine
Citation Excerpt :
Furthermore, observational studies of influenza vaccine effectiveness and safety depend on accurate ascertainment of influenza vaccination status in the study population [4–7]. Potential sources of data for vaccine coverage include self-report, medical records and immunization registries [2,5–18]. Self-reported influenza vaccination status, usually collected in large population based health surveys, is commonly used to estimate vaccine coverage and to determine vaccine effectiveness [2,5,7–9].
We aim to compare influenza vaccination coverages obtained using two different methods; a population based computerized vaccination registry and self-reported influenza vaccination status as captured by a population survey.
The study was conducted in the Autonomous Community of Madrid (ACM), Spain, and refers to the 2011/12 influenza vaccination campaign.
Information on influenza vaccination status according to a computerized registry was extracted from the SISPAL database and crossed with the electronic clinical records in primary care (ECRPC). Self-reported vaccine uptake was obtained from subjects living in the ACM included in the 2011–12 Spanish National Health Survey (SNHS). Independent study variables included: age, sex, immigrant status and the presence of high risk chronic conditions. Vaccination coverages were calculated according to study variables. Crude and adjusted prevalence ratios were computed to assess concordance.
The study population included 5,245,238 adults living in the ACM in year 2011 with an individual ECRPC and 1449 adult living the ACM and interviewed in the SNHS from October 2011 to June 2012.
The weighted vaccination coverage for the study population according to self-reported data was 19.77% and 15.04% from computerized registries resulting in a crude prevalence ratio (cPR) of 1.31 (95% CI 1.20–1.44) so self-reported data significantly overestimated 31% the registry coverage. Self-reported coverages are always higher than registry based coverages when the study population is stratified by the study variables. Self-reported overestimation was higher among men than women, younger age groups, immigrants and those without chronic conditions. Both methods provide the most concordant estimations for the target population of the influenza vaccine.
Self-report influenza vaccination uptake overestimates vaccination registries coverages. The validity of self-report seems to be negatively affected by socio-demographic variables and the absence of chronic conditions. Possible strategies must be considered and implemented to improve both coverage estimation methods.
Risk of Guillain-Barré syndrome after seasonal influenza vaccination and influenza health-care encounters: A self-controlled study
2013, The Lancet Infectious Diseases
The possible risk of Guillain-Barré syndrome from influenza vaccines remains a potential obstacle to achieving high vaccination coverage. However, influenza infection might also be associated with Guillain-Barré syndrome. We aimed to assess the risk of Guillain-Barré syndrome after seasonal influenza vaccination and after influenza-coded health-care encounters.
We used the self-controlled risk interval design and linked universal health-care system databases from Ontario, Canada, with data obtained between 1993 and 2011. We used physician billing claims for influenza vaccination and influenza-coded health-care encounters to ascertain exposures. Using fixed-effects conditional Poisson regression, we estimated the relative incidence of hospitalisation for primary-coded Guillain-Barré syndrome during the risk interval compared with the control interval.
We identified 2831 incident admissions for Guillain-Barré syndrome; 330 received an influenza vaccine and 109 had an influenza-coded health-care encounter within 42 weeks before hospitalisation. The risk of Guillain-Barré syndrome within 6 weeks of vaccination was 52% higher than in the control interval of 9–42 weeks (relative incidence 1·52; 95% CI 1·17–1·99), with the greatest risk during weeks 2–4 after vaccination. The risk of Guillain-Barré syndrome within 6 weeks of an influenza-coded health-care encounter was greater than for vaccination (15·81; 10·28–24·32). The attributable risks were 1·03 Guillain-Barré syndrome admissions per million vaccinations, compared with 17·2 Guillain-Barré syndrome admissions per million influenza-coded health-care encounters.
The relative and attributable risks of Guillain-Barré syndrome after seasonal influenza vaccination are lower than those after influenza illness. Patients considering immunisation should be fully informed of the risks of Guillain-Barré syndrome from both influenza vaccines and influenza illness.
Canadian Institutes of Health Research.
Accuracy of self-reported human papillomavirus vaccine receipt among adolescent girls and their mothers
2012, Journal of Adolescent Health
The use of self-report of human papillomavirus (HPV) vaccination status has several implications for clinical care and research. Reports of HPV vaccination history of adolescent girls (ages: 14–17 years) and their mothers were compared with medical chart data to assess the accuracy of HPV vaccine recall.
Adolescent girls (N = 74) and their mothers independently completed questionnaires regarding HPV vaccination history, which were compared with medical chart data to assess the accuracy of HPV vaccine recall.
There were high levels of inaccuracy between actual HPV vaccination and self-report of vaccine receipt. Both mothers and daughters had poor recall of HPV vaccination, and were more likely to underestimate than overestimate the coverage. Girls who accurately reported their vaccination status were not more likely to have been sexually active in the past 2 months (p = .75).
These findings have clinical and research implications, as self-report is relied on to assess young women's vaccination status in research settings or in the absence of medical records. These data address the still prevalent concern that HPV vaccination encourages adolescent sexual behavior. It is unlikely that sexual behaviors will change as a result of vaccination in the large percentage of girls who cannot recall being vaccinated.

View all citing articles on Scopus

View full text

Using OHIP physician billing claims to ascertain individual influenza vaccination status

Abstract

Introduction

Section snippets

Study population and setting

Canadian Community Health Survey

Descriptive statistics

Agreement between OHIP claims and CCHS responses using kappa statistic

Discussion

Am J Prev Med

Am J Prev Med

Am J Prev Med

A dictionary of epidemiology