Abstract
Background: Mandated vaccination for school attendance is a growing strategy internationally. Our aim was to investigate the effects of implementing school vaccination mandates on pediatric population vaccine coverage.
Methods: In this systematic review, we searched MEDLINE, Embase, CINAHL, the Education Resources Information Center (ERIC) and the PAIS Index for empirical studies of implementation of a primary or secondary school vaccination requirement published in any language through March 2019 with vaccination rates as an outcome. We sought additional studies by consulting experts, reference lists and grey literature sources. Included studies were too heterogeneous for meta-analysis; thus, we extracted data using a standardized rubric and synthesized the results narratively.
Results: Among the 4232 citations obtained, 20 studies met the inclusion criteria. Eighteen were conducted with US data, 1 with Italian data, and 1 with Australian data. Four studies examined school-entry mandates, and 16 examined adolescent requirements. An uncontrolled before–after design was used in 10 studies, cross-sectional analysis in 7, a retrospective cohort design in 2, and a prospective cohort in 1. In many cases, increased documentation of coverage followed the addition of new requirements. The exception to this was human papillomavirus vaccination mandates, which were highly controversial, in the United States. The studies contained notable risks of bias, with cointerventions rarely acknowledged or accounted for, and subpopulations often excluded. A substantial risk of ecological fallacy existed for most studies.
Interpretation: Vaccination mandates appear largely associated with increased vaccination coverage, but it is not possible to attribute causality to the mandate in most studies. High-quality implementation research that uses whole-population coverage data and takes into consideration cointerventions, confounders, clustering of unvaccinated populations and context is required.
Vaccination mandates for school attendance are ubiquitous in the United States and are growing as a strategy in Europe and Australia. Although the Canadian Medical Association passed a resolution in 2015 recommending that “governments authorize elementary and secondary schools to require a declaration of immunization status, to be followed by a conversation between public health officials and parents where children are shown to be inadequately immunized,”1 school vaccination legislation currently exists in just 2 Canadian provinces, Ontario2 and New Brunswick, 3 with intent to implement a policy in British Columbia recently announced.4 Mandate policies vary in requirements (documentation, education, vaccination), schedule (which vaccines, when), restrictiveness (exemption processes),5 and incentives (e.g., financial rewards) or penalties (e.g., fines, school exclusion). Debate regarding best practices for mandate policies tends to draw largely on ethical arguments6 regarding the optimal legislation to achieve maximum pediatric vaccine coverage with minimal violation of parental civil liberties, with some voices advocating strict policies permitting few exemptions, others favouring a more libertarian approach, and many aiming to strike a balance.7 Often in these debates, the effectiveness of vaccination mandate laws in increasing population vaccine coverage is assumed; however, recent systematic assessment of the literature regarding the impacts of school mandates on vaccine coverage in Canada and other wealthy countries has not, to our knowledge, been done.
To assess the effectiveness of school vaccination mandates in real-life settings, studies with appropriate methods and outcomes must be examined. Cross-sectional studies have documented associations between the existence (or strictness) of a child vaccination mandate and population coverage, suggesting that restrictive policies around exemptions from vaccination mandates may increase compliance.8–11 These studies alone, however, cannot determine causality and are vulnerable to ecological biases. Other studies have explored the influence of mandates on outcomes such as exemption rate12,13 and disease occurrence,14 but disease is affected by many factors including temporal trends, and overall exemptions may mask clustering of unvaccinated populations that may raise the risk of disease outbreak. A focused systematic review comparing studies assessing the preferred outcome of population vaccination coverage is required to assess evidence for or against implementing new vaccination mandate policies. Our aim in this analysis was to inform the ongoing discussion regarding optimal childhood vaccination policy by systematically identifying and synthesizing the existing evidence to answer the question: What is the impact of implementation of school vaccination mandate policies on school-age population vaccine coverage?
Methods
Research design
This systematic review with narrative synthesis is reported according to PRISMA guidelines.15 In designing and implementing this review, we referred to the Cochrane Handbook for Systematic Reviews of Interventions16 for guidance. The protocol for this nonclinical review of policy interventions was not registered.
Selection criteria
We sought studies published in any language, using any empirical method, to obtain evidence on the effects of implementation of school vaccination mandates on the outcome of population vaccine coverage. Appropriate comparison groups included same-time comparators in locations without mandate changes or pre–post intervention comparisons. Studies that focused on individual school rules rather than regional/government policy, mandates for nonpediatric or nonschool populations, or outbreak-specific policies were excluded, as were nonempirical papers and studies that examined only outcomes other than population vaccine coverage.
Data collection/search strategy
Two of the authors (C.V.-M. and D.G.) searched MEDLINE, Embase, CINAHL, the Education Resources Information Center (ERIC) and the PAIS Index in March 2019 to identify potentially relevant articles (search details provided in Appendix 1, available at www.cmajopen.ca/content/7/3/E524/suppl/DC1). Searches combined database-specific subject headings and keywords for the concepts vaccination, law/policy and schools. Our MEDLINE search strategy was peer reviewed through Peer Review of Electronic Search Strategies (PRESS).17 We conducted limited searches for grey literature in thesis and dissertation databases, and databases that include conference abstracts and working papers (Appendix 1). References were searched and experts consulted to identify studies, including grey literature, missed by database searching.
Screening and abstraction
Titles and abstracts were independently screened in duplicate by 2 authors (C.V.-M. and D.G.) for relevance. Full texts of potentially includable articles were obtained, and all of the authors reviewed them independently in triplicate. Discrepancies were resolved by discussion among the authors to reach consensus. Included articles were then subjected to a data-extraction process independently in duplicate by 2 authors (D.G. and J.A.B.) (see Appendix 2, available at www.cmajopen.ca/content/7/3/E524/suppl/DC1, for data extraction fields).
Risk of bias
Although we were unable to find a tool to assess the risk of bias that directly applied to all the included studies, 2 authors (J.A.B. and D.G.) independently and in duplicate assessed each included study for potential bias in methods using the bias categories from the ROBINS-I (Risk Of Bias In Non-randomised Studies – of Interventions) tool,18 adapted for relevance to this specific body of literature.
Data analysis
We synthesized the findings in a narrative manner using methods influenced by Popay and colleagues.19 Two authors (J.A.B. and D.G.) developed a preliminary textual synthesis of the characteristics and findings of the included studies, based on the tabulated extracted data. In an iterative manner, we explored relations within these data, grouping studies to examine the influence of heterogeneity (e.g., when looking by vaccine, target age group, location, data source, study design or type of mandate) on outcomes due to policy or setting details. We then incorporated our data on assessment of risk of bias into the descriptive synthesis of the included evidence to describe the robustness of the findings and temper the weight of the conclusions.
Ethics approval
As this study was solely literature based, it was not eligible for institutional ethics approval, and none was sought.
Results
Database searches resulted in 4232 unique citations to screen and assess for eligibility, and consulting experts and reference lists revealed 2 additional studies. After screening for relevance and applying inclusion and exclusion criteria, we selected 20 studies for inclusion in the review (Figure 1).
Heterogeneity
Of the 20 included studies, 18 were conducted with US data, 1 was from Italy,20 and 1 was from Australia.21 In 4 studies, the investigators looked at school-entry mandates implemented from 1970 to 2017,20–22 and the remaining 16 (all set in the US) examined adolescent mandates (sometimes referred to as “middle school” requirements) from the 1990s to 2015.23–38 An uncontrolled before–after design was used in 10 studies, cross-sectional analysis in 7 and retrospective cohort design in 2, and 1 study was a prospective cohort study. Population size ranged from 58323 to 954 973,22 and time frames ranged from single cross-sectional surveys27,31,38 to a 17-year span.22 Types of data included parental report, administrative data sets and clinician-verified records, and sources included various iterations of the US National Immunization Survey-Teen,24,25,30,33,34,37 state/national vaccine registries,20,22,35,36 school immunization databases,21,28,29 clinical registries,26,32 clinic and school sampling,27,38,39 random-digit-dial survey23 and the US Health Plan Employer Data and Information Set.31 The most commonly used data source, the US National Immunization Survey-Teen, is a random-digit-dial telephone survey of parents of adolescents aged 13–17 years in 50 US states and the District of Columbia that been conducted since 2006. Most of the included studies assessed coverage following the addition of new vaccines in a setting with existing mandates (e.g., addition of an adolescent pertussis vaccine dose), although 1 examined a new documentation mandate,21 1 examined a tightened exemption procedure,22 2 included education mandates,24,33 and 1 reported on added vaccines combined with stricter enforcement.20 Table 1 summarizes the methods and findings of included studies.
Risk of bias
Owing to the high degree of heterogeneity, it was difficult to quantitatively compare studies’ risk of bias. Drawing on the categories for assessment of risk of bias in the ROBINS-I v.19 tool,40 we systematically assessed all 20 included articles for potentially biasing limitations in the domains of confounding, comparison groups, data collection, lack of intervention detail and outcome assessment. We found that common biases to which this body of literature is vulnerable include confounding, selection bias, measurement bias and bias due to deviation from/variation in implementation of the interventions. All of the included studies were observational assessments of natural experiments rather than studies of interventions designed to be implemented in a controlled manner. Fifteen studies used ecological designs, which are prone to confounding and bias, including ecological fallacy, in which associations identified at a group level are extrapolated to apply to individuals41 and cannot be relied on as evidence of causality,42 as an increase or decrease in individual coverage may be due to factors other than the mandate. Several of the nonecological studies, however, recruited study populations that were unlikely to be representative of the larger population, which limited the external validity (generalizability) of the findings. Although some studies described cointerventions, others made no mention of common cointerventions (e.g., outreach programs to improve vaccination awareness, education and access, or changes in vaccine purchasing, coverage and distribution) that may accompany mandatory policies, and little effort was made across all studies to measure or account for the impact of such potential confounders. Studies with pre–post designs varied greatly in the length of baseline and follow-up data, and some had such short periods that a trend could not reliably be established. Any National Immunization Survey-Teen sample is vulnerable to response bias owing to the moderate response rate (55.5% for the landline sample and 29.5% for the cellphone sample in 2016).43 Landline-only samples (including all pre-2011 surveys) are vulnerable to additional selection bias, non–provider-verified data (including nearly half of National Immunization Survey-Teen responses) are vulnerable to recall bias and social desirability bias by the respondent, and using only provider-verified data risks additional selection bias. Finally, although implementation elements such as messaging parents and consistency of enforcement likely matter greatly in a policy’s success, details or measures of implementation factors were rarely mentioned in the studies and were never accounted for in analyses. Table 2 presents this assessment for each study individually, and discussion of these potentially biasing limitations is integrated into the findings below.
Study findings
School entry mandates (typically for age 5–7 yr and applying to a large array of vaccines scheduled from birth to school entry) were found to be associated with increased documentation and/or vaccination in diverse settings. The earliest study showed increased coverage for all vaccines, including one not required by the mandate, in the first year of a 1970 vaccine mandate compared to the previous year.39 Thompson and colleagues’21 evaluation of an Australian mandate for school entry vaccination certificates in the 1990s showed increased documentation for all students, with greatest effect among those not up to date with vaccines. Omer and colleagues22 studied changes to a Washington State vaccine mandate that introduced a requirement for a health care provider signature for exemptions in 2011 and found an increase in the proportion of students who were up to date for all vaccines after implementation. D’Ancona and colleagues20 reported preliminary numbers following Italy’s 2017 decision to add vaccination to an existing mandate and enforce it. They found a small increase in measles/mumps/rubella and polio vaccination among 7-year-old children. Jackson and Carpenter,39 Thompson and colleagues21 and D’Ancona and colleagues20 did not assess the statistical significance of the observed changes. All 3 studies were ecological studies and, thus, vulnerable to confounding, and the investigators reported varying numbers of cointerventions such as awareness campaigns, and reduction of cost and access barriers. The mandate change studied by Omer and colleagues22 was accompanied by other changes in vaccination education and access, which were not controlled for or assessed. Although the evidence for causality in this group of studies was not strong, all associations were positive.
Twelve of the included studies focused primarily or entirely on adolescent mandates (commonly for grades 5–9, or middle school populations) for vaccines other than human papillomavirus (HPV). These vaccines included hepatitis B, tetanus/diphtheria/acellular pertussis, meningococcal vaccines, measles/mumps/rubella and varicella. All 12 studies were conducted in the US, in jurisdictions with preexisting school entry mandates. These studies showed no increase in uptake associated with educational mandates,24 but vaccination mandates for these vaccines were positively associated with higher coverage after implementation,23–25,27–30,32,35,36,38 regardless of data source or study design. Effect sizes varied greatly; in some studies, larger increases were associated with mandates for vaccines whose coverage was lower before the intervention23,32 or for low-income students,25 and in 1 study, racial/ethnic disparities in coverage were smaller after the intervention.29 Spillover effects were observed from 1 mandated vaccine (e.g., tetanus/diphtheria/acellular pertussis) onto other adolescent vaccines (e.g., meningococcal vaccines) in 3 of the 4 studies in which this was examined.25,30,32 The 1 study that showed no spillover effects from tetanus/diphtheria/acellular pertussis mandates involved a small school-based sample that was unlikely to be representative of the entire population.38
The evidence on adolescent HPV mandates in the US told a different story. Four studies did not show an association between HPV vaccine education or mandates for girls and an increase in HPV vaccine coverage.24,26,33,34 Pierre-Victor and colleagues34 did find that, independent of mandates, physician recommendation and health care contacts were predictors of HPV vaccination. Thompson and colleagues37 studied a later-implemented mandate for both girls and boys in a state with high coverage in girls before the mandate and found that rates increased among boys but not girls following the mandate. The study did not disentangle the effect of the mandate and the expanded insurance coverage for boys, which happened at the same time. Three studies showed a small spillover effect of new adolescent mandates for other vaccines onto HPV vaccine uptake,25,30,35 but only if HPV vaccination was not mandated.30 This spillover effect was most pronounced among low-income students.25 The HPV mandate literature consisted of ecological studies and did not examine or control for other contextual or implementation factors, with the exception of the study by Cuff and colleagues,26 which was a single-centre study of well-child clinic patients and may not be representative of the larger population.
Interpretation
New or tightened requirements for vaccination of schoolchildren were usually associated with increased coverage of the affected populations, with effects appearing larger when preintervention vaccination rates were low. Mandates for HPV vaccination in the US, where there was a high degree of population hesitancy and politicization around the vaccine, were notably ineffective. Spillover effects indicate that health care interactions may be more important than the compulsory nature of mandates and notably had greater impact on HPV vaccine uptake than direct mandates for education or vaccination. The vast majority of the studies were ecological and, thus, vulnerable to confounding and ecological fallacy. Although ecological analyses are important for generating hypotheses and are widely used in epidemiology, they are vulnerable to identifying associations between factors that may be correlative, bidirectional, mediated by other factors, or confounded by unobserved cointerventions or population attributes. Such studies are therefore not typically considered capable of drawing causal conclusions. Few studies examined or accounted for the influence of common cointerventions such as improved access, education, insurance coverage, and changes in vaccine purchasing or costs. Furthermore, implementation and enforcement of vaccine mandates were not examined in most studies, yet poor implementation and/or uneven enforcement may render a policy ineffective.44
MacDonald and colleagues45 outlined 3 reasons jurisdictions implement vaccination mandates: failure of less-coercive methods of encouraging vaccination, outbreak of vaccine-preventable disease and as a final stage in a global disease-eradication project. Mandates considered in the current analysis — whether for documentation, education or vaccination — were enacted in 1 or both of the first 2 of these scenarios, when policy-makers decided such laws were ethically permissible given the safety profile of vaccine supplies and general population acceptance of vaccination.46 Although school vaccine mandates are commonly considered to have made a major, if not essential, contribution to US vaccine coverage, the causal relations between mandates and population vaccination remain unclear owing to myriad cointerventions and confounding factors. For example, in cases in which US insurance companies have been reluctant to cover nonmandated vaccines (e.g., HPV vaccine for boys), mandates can increase vaccination rates by acting indirectly on insurers through school vaccine requirements. Therefore, implementing a mandate in a setting in which public coverage for vaccines is already universally offered may not result in an increase in coverage comparable to that seen in jurisdictions with more privatized coverage.
Policy-makers must consider many factors, including the variety of possibilities for exemptions, penalties and rewards, and how the mandate may be implemented, when weighing implementation of new or revised mandates — issues Attwell and colleagues5 classified as matters of severity and enforcement. Given the potential for a policy to fail to gain compliance, as seen with the US HPV mandates for girls,24,26,30,33,34,37 such issues are real and present in today’s policy landscape. Our findings are largely congruent with those of Lee and Robinson,47 who found that, in most cases, childhood vaccination mandates through 2015 were associated with higher coverage in the US, with limited evidence of transferability to other settings. A review by the US Community Preventive Services Task Force that included studies published through 201248 advised that interventions such as reminder systems and school-based vaccination clinics were cost-effective interventions to promote pediatric vaccine coverage49 and that such strategies to increase awareness of and access to vaccination could be attempted before proposing or strengthening a mandate.
Limitations
No search is exhaustive, so although we endeavoured to be systematic, transparent and comprehensive in our data collection and inclusion screening, it is possible we may have missed studies that may have been eligible for inclusion. At least 2 reviewers assessed for inclusion at every stage of review, but errors in judgment are possible. In addition, with nearly half of the included studies published within the past 3 years, there may be new studies currently under way that would contribute valuable information to our findings. Older studies, particularly those conducted before the mid-1990s, would have been conducted in an era of substantially different disease prevalence, vaccine products, vaccine schedules, data sources, data management practices and public health policies, which may limit the transferability of their findings to contemporary settings.
Conclusion
Adding well-accepted vaccines to an existing mandate, introducing a mandate in concert with reduction of structural barriers to vaccination or adding documentation requirements all appear to be associated with increased vaccination and/or documentation in most cases. It is unclear, however, to what extent such increases are due to the compulsory nature of the policies or to cointerventions that increase access and awareness. Education or vaccination mandates for highly politicized vaccines are more risky and may fail to be followed by the desired increase or even decrease uptake relative to nonmandate jurisdictions. To further the science of vaccination levers, high-quality implementation research that uses whole-population coverage data and takes into consideration cointerventions, confounders, clustering of unvaccinated populations and context is required. Owing to the risk of backlash, in cases of highly politicized vaccines or jurisdictions without a tradition of mandates, other approaches such as improving access, education and documentation might be tried before moving to mandated vaccination.
Acknowledgements
The authors thank the peer reviewers, including the 2 CMAJ Open reviewers, who strengthened the manuscript, and Alex Goudreau, who provided Peer Review of Electronic Search Strategies (PRESS) for the MEDLINE search strategy.
Footnotes
Competing interests: Devon Greyson reports grants from the British Columbia Immunization Committee during the conduct of the study. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Devon Greyson and Julie Bettinger conceived of the study. Chris Vriesema-Magnuson and Devon Greyson conducted the literature searches. Devon Greyson drafted the manuscript. All of the authors analyzed the data, critically revised the manuscript for important intellectual content, approved the final version to be published and agreed to act as guarantors of the work.
Funding: Funding for this study was provided by the British Columbia Immunization Committee. Julie Bettinger is supported by a Michael Smith Foundation for Health Research Scholar Award.
Supplemental information: For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/7/3/E524/suppl/DC1.
References
- Copyright 2019, Joule Inc. or its licensors