Elsevier

Vaccine

Volume 31, Issue 37, 20 August 2013, Pages 3928-3935
Vaccine

Self-report compared to electronic medical record across eight adult vaccines: Do results vary by demographic factors?

https://doi.org/10.1016/j.vaccine.2013.06.041Get rights and content

Highlights

  • Discrepancies exist between EMR and self-report.

  • This may increase as vaccines become available in more non-traditional settings.

  • Discrepancies exist by vaccine, particularly positive predictive value.

  • Minorities had higher disconcordance compared to whites.

  • Some discrepancies were found by education, income and employment.

Abstract

Immunizations are crucial to the prevention of disease, thus, having an accurate measure of vaccination status for a population is an important guide in targeting prevention efforts. In order to comprehensively assess the validity of self-reported adult vaccination status for the eight most common adult vaccines we conducted a survey of vaccination receipt and compared it to the electronic medical record (EMR), which was used as the criterion standard, in a population of community-dwelling patients in a large healthcare system. In addition, we assessed whether validity varied by demographic factors. The vaccines included: pneumococcal (PPSV), influenza (Flu), tetanus diphtheria (Td), tetanus diphtheria pertussis (Tdap), Human Papilloma Virus (HPV), hepatitis A (HepA), hepatitis B (HepB) and herpes zoster (shingles). Telephone surveys were conducted with 11,760 individuals, ≥18, half with documented receipt of vaccination and half without. We measured sensitivity, specificity, positive and negative predictive value, net bias and over- and under-reporting of vaccination. Variation was found across vaccines, however, sensitivity and specificity did not vary substantially by either age or race/ethnicity. Sensitivity ranged between 63% for HepA to over 90% (tetanus, HPV, shingles and Flu). Hispanics were 2.7 times more likely to claim receipt of vaccination compared to whites. For PPSV and Flu those 65+ had low specificity compared to patients of younger ages while those in the youngest age group had lowest specificity for HepA and HepB. In addition to racial/ethnic differences, over-reporting was more frequent in those retired and those with household income less than $75,000. Accurate information for vaccination surveillance is important to estimate progress toward vaccination coverage goals and ensure appropriate policy decisions and allocation of resources for public health. It was clear from our findings that EMR and self-report do not always agree. Finding approaches to improve both EMR data capture and patient awareness would be beneficial.

Introduction

Because immunization is crucial to the prevention of disease, having an accurate measure of vaccination status for a population can serve as an important guide in targeting prevention efforts.[1], [2] To monitor vaccination status, the United States conducts population-based vaccination coverage surveys, [3], [4] however, obtaining accurate assessment is difficult. Most people have attended multiple medical practices, leaving records scattered or incomplete. Time may also result in lapses in memory [5], [6], [7]. Several vaccinations, such as tetanus/diphtheria (Td), pneumococcal polysaccharide vaccine (PPSV), hepatitis A (HepA) and hepatitis B (HepB) series, may have been administered years before a survey is conducted [6]. Also, patients may affirm receipt of vaccines they believe they should have obtained or deny obtaining a vaccination that might indicate risky behavior [8], [9]. Lack of accurate data decreases the ability to interpret estimated coverage levels and may cause providers to miss opportunities to provide needed vaccines. Validity of self-report has been extensively studied for Influenza (Flu) and PPSV [9], [10], [11], [12], [13], [14], [15], but there is a paucity of literature on other vaccines (e.g., HepA, HepB) and relatively new vaccines such as Human Papillomavirus (HPV). Further, information on validity that is age and race/ethnicity specific has also had limited study [5], [7], [16]. In order to comprehensively assess the validity of self-reported adult vaccination status for the eight most common adult vaccines, we conducted a survey of vaccination receipt and compared it to the electronic medical record (EMR) in a population of community-dwelling patients in a large healthcare system. In addition, we assessed whether validity varied by demographic factors. The purpose of this paper is to report the concordance of data obtained through both methods of data collection.

Section snippets

Study setting and population

This study was conducted in an integrated health care delivery system with 21 primary care clinics, 30 specialty clinics and over 700 practicing physicians. The plan insures over one million people in an open-access system, allowing patients to obtain care within the medical group or the larger network. The vast majority of care is obtained within the network as nearly all services are covered. The majority of the population is white, employed, with education of high school or beyond. Eligible

Results

The goal of 11,200 completed surveys was based on 200 each from the 56 strata. We ended with 11,760 completed surveys: 10,670 toward goal and 1090 over. The overage occurred due to an initial flaw in the tracking program affecting all HepA (overage = 713). Overage was also due to multiple interviewers continuing to survey until informed that the goal was achieved. For non-Hispanic blacks 65+ and Hispanics with no evidence of PPSV or Flu vaccination, numbers in the base population were

Discussion

The purpose of this study was to examine self-reported vaccination status compared to EMR data. Considerable variation was found by vaccine, age and race/ethnicity. We also assessed rates of under-and over-reporting and net bias. Under-reporting was relatively low, except in the Hispanic strata, while over-reporting varied by vaccine. The net bias relative difference also varied widely. We found more favorable agreement statistics for PPSV and HPV. Tetanus had the largest net bias relative

Conclusions and future directions

Health system electronic databases provide ready capture of data on vaccinations, yet potential for misclassification cannot be ignored. As more vaccines are available in non-traditional settings, this problem is likely to become more common. We must continue to find ways for both health plans and patients to capture and communicate information on vaccination status to ensure surveillance efforts are as robust as possible.

Acknowledgements

The authors wish to thank Colleen King, Manager, and David Butani, Research Study Coordinator, from the HealthPartners Institute for Education and Research Data Collection Center, who trained and monitored the numerous interviewers and oversaw the data collection for over 11,000 surveys. We also thank the Centers for Disease Control and Prevention (CDC) for funding of the study and James Singleton for his editorial comments. The authors wish to acknowledge Sandy De Quesada, BA, HealthPartners

References (25)

Cited by (0)

View full text