Modern approaches to predicting vaccine hesitancy: A scoping review

1. Introduction

Even for highly effective and safe vaccines, vaccination rates range from approximately 50% to 90% in the United States [1], [2], and there are concerns about disparities in vaccination rates across different geographic units and populations [3], [4]. To help target resources to address these inequities, many studies constructed vaccine uptake indices (VUIs), defined as quantitative models that predict a vaccination rate within a given geographic area [5], [6], [7]. Such indices can address the need for equitable, targeted vaccination campaigns, which has been exemplified through the recent COVID-19 pandemic [4], [8], [9].

Vaccine uptake is closely related to the concepts of vaccine acceptance and vaccine hesitancy [10]. Vaccine uptake can be time-dependent [11], especially when a vaccine is in short supply, but at any given time point, there are often persistent inequalities in vaccination rates by group and geography [12], [13], [14], [15]. While the literature outside of COVID-19 is less advanced, development of VUIs could prove useful in the case of vaccination for other infectious diseases, such as human immunodeficiency virus (HIV), hepatitis C virus (HCV), or Epstein-Barr virus (EBV), where vaccine development continues to be of significant interest [16], [17], [18]. These tools could employ data on factors such as social stigma, perceived risk, and access to healthcare to help predict potential vaccine acceptance in a given area. Similarly, predicting vaccine acceptance may be valuable for vaccines that (1) are currently in early stages of development or clinical trials, including the HCV and EBV vaccines, (2) have been newly developed and distributed, such as the respiratory syncytial virus (RSV) vaccine [19], or (3) are mature vaccines with insufficient uptake, such as the influenza or human papillomavirus (HPV) vaccines [20]. Prediction of vaccination for both existing (e.g., influenza) and novel infectious diseases remains important and challenging [21], [22], [23]. Identifying regions likely to have increased vaccine hesitancy allows for targeted preventive strategies aimed at addressing barriers to acceptance. However, in current practice, VUIs are not used on a wide scale to predict vaccine acceptance in populations, and as a result, many studies utilize vulnerability indices to predict vaccine uptake and influence policy [24], [25].

A vulnerability index is a quantitative model that predicts the impact of an infection (e.g., COVID-19 infections or mortality) by area based on geospatial area and social risk, allowing for further analysis of inequities [26], [27]. Some vulnerability indices may be used to predict vaccine uptake outside of their intended applications. For instance, the Center for Disease Control’s Social Vulnerability Index (SVI) can be used to predict vaccination levels, but it was designed to assess the impact of natural disasters on a population by geospatial area [28]. It includes factors—such as the proportion of children, single-parent households, and households with access to a personal vehicle—that increase vulnerability during evacuation but likely do not significantly affect infectious disease risks [29]. Its updated form, the COVID-19 Community Vulnerability Index (CCVI), takes public health infrastructure into account but was designed to predict COVID-19 vulnerability trends and overall community risk rather than vaccination uptake by small-area analysis [30]. Because many vulnerability indices are not specifically designed to predict vaccination rates, introducing VUIs may facilitate equitable resource allocation with increased accuracy [8], [31], [32].

In this review, we survey the state of the literature regarding VUIs using the case of COVID-19. To our knowledge, this is the first study of its kind in the literature. Other meta-analyses and systematic reviews focus on either aggregating data on factors that contribute to COVID-19 vaccination inequities or creating a pooled vaccine acceptance rate from individual studies [33], [34], [35].

The overall goal of this study is to support the development of next-generation VUIs that address vaccination disparities by focusing on key areas identified in the current literature. We aim to determine the scope of the existing literature that explores quantitative indices that predict vaccination outcomes and/or social vulnerability. We evaluate whether the current literature: (1) makes use of quantitative models that predict vaccine uptake and social risk by geospatial area, (2) combines different sources of predictive variables or features of VUIs, including individual-level predictor variables [36], [37], geo-spatiotemporal analysis [38], [39], and vaccine agnosticity [21], [22], and (3) uses sociobehavioral frameworks to limit model misspecification [40], [41].

Due to limited data availability and the need for promptness of research during the COVID-19 pandemic, we hypothesized that most geospatial VUIs would directly predict vaccination using solely administrative data (i.e., vaccine uptake records from public health agencies), with limited use of models trained using data from infectious diseases beyond COVID-19 or frameworks to assist causal inference.

3. Results

Database searches retrieved 7,156 records (Fig. 1). After removing duplicates, 3,615 records remained for screening. Following the title and abstract screening, 345 reports were assessed for eligibility. A total of 229 reports met the inclusion criteria and were included in the scoping review. See Supplement 3 for a list of further reading and Supplement 4 for each included report in Supplement 1.

• Download figure
• Open in new tab

Fig. 1.

A PRISMA flow diagram depicting the scoping review process [46].

Of the 229 included studies, 203 (89%) were classified as VUIs and 26 (11%) were classified as vulnerability indices. The studies demonstrated a variety of methodological approaches, as presented in Table 1. Specifically, 151 VUIs (66%) utilized geospatial data, and of this group, 58 (38%) incorporated geo-spatiotemporal analysis. A total of 50 studies (22%) integrated data from multiple infectious diseases in their analyses, and 30 studies (13%) employed social and behavioral frameworks. Overall, 15 reports (7%) were classified as methodologically advanced VUIs, based on usage of at least three of the four following characteristics: a combination of individual- and population-level predictor variables, spatiotemporal analysis, vaccine-agnostic data usage, and health-related sociobehavioral frameworks.

The studies used different geospatial levels for their analyses (Table 2). Small-area analysis was the most commonly used, including county- or city-level data (64 [42%]) and ZIP-code- or neighborhood-level data (33 [22%]). Among the studies that applied social frameworks, the Health Belief Model was the most commonly used (16 [53%]). Other notable frameworks included the Theory of Planned Behavior (8 [27%]) and the Protection Motivation Theory (2 [7%]). Four studies (7%) used more than one of these frameworks to analyze vaccination uptake.

4. Discussion

This systematic scoping review analyzed VUIs and vulnerability indices in response to COVID-19. Out of the 229 studies reviewed, 203 (89%) were categorized as VUIs and 26 (11%) as vulnerability indices. The review revealed a significant methodological diversity in the development of VUIs (Table 1). Specifically, 124 studies (54%) used any form of individual-level data regarding vaccination behaviors as predictor variables. One hundred twenty-five of the VUIs (62%) utilized geospatial analysis, but only 49 (24%) made use of geo-spatiotemporal analysis. Additionally, most geospatial analysis was performed at the county or city level (64 [42%]) (Table 2). Only 50 studies (22%) used data from multiple infectious disease sources, and 30 VUIs (15%) employed social and behavioral frameworks. Most of the studies that used a social or behavioral framework used the Health Belief Model (16 [53%]).

The VUIs demonstrated various approaches in their construction. The use of geospatial data was prevalent, with most analyses performed at the county or city level, suggesting the importance of county-level administrative data in small-area analysis. However, the limited use of data sources from more than one infectious disease in constructing these indices suggests a gap in leveraging patterns of historical vaccination behaviors to predict vaccination for novel pathogens. Moreover, the sparse application of social and behavioral frameworks points to an underutilized avenue for facilitating causal inference with VUIs. This review also highlights the potential for studies that could integrate individual-level predictor variables with currently used designs that utilize aggregate-level data.

The review identified two primary types of study designs: Group A (cross-sectional surveys) and Group B (indices using administrative data). Group A studies, which constituted the majority of VUIs, focused on analyzing vaccine uptake within specific geospatial areas using individual-level variables such as vaccination attitudes and risk comfortability. Group B studies more often utilize machine learning, nonparametric modeling, or other novel statistical approaches that may better capture complexity of nonlinear relationships between epidemiological variables, but Group A studies place a valuable emphasis on using individual-level variables as predictors. The usage of simple machine learning models by Group A studies, such as multiple logistic regression, increases the overall interpretability of these VUIs, but this may come at the expense of accuracy [54].

This individual-focused approach allows for the identification of specific behavioral and psychological factors that may inform targeted interventions, often through using theoretical frameworks of vaccine hesitancy to reduce model misspecification [55]. However, Group B studies are still important in understanding the impact of social vulnerability on vaccination rates. These tradeoffs are important to consider, and an ideal VUI may make use of both individual-level and population-level variables when making predictions across geospatial areas and avoiding the ecological fallacy.

A minority of study designs did not fit in either category. For instance, some of the studies constructed temporal dynamics models that utilized vaccination data from Google Trends, Twitter, Reddit, or other internet- or social media-based sources to make predictions about vaccination rate by geospatial area [36], [37], [56]. These types of studies uniquely combine data from individuals as well as administrative public health data sources and are also methodologically diverse in terms of their underlying statistical modeling techniques.

We have also found that the utility of VUIs often overlaps with that of vulnerability indices. In practice, both types of models are generally used to predict vaccination given a geospatial area [8], [25], [31]. Some ecological study designs compare the performance of multiple vulnerability indices that are currently used in the field (such as the SVI and CCVI) to predict infection and/or mortality rates but do not always include an application to vaccination uptake [27], [57], [58]. Such studies were included as vulnerability indices, and we note that they are particularly important in comparing the uses of vulnerability indices in order to eventually predict vaccination.

The overlap of utility between the VUIs and vulnerability indices highlighted in our review suggests a potential to refine these tools for improved predictive power. Notably, the use of vulnerability indices (such as the SVI and CCVI) has shown promise in predicting vaccination rates, despite the primary design of such indices for other public health applications [6], [59], [60]. This highlights the potential of existing indices to contribute to next-generation VUIs.

We suggest that integrating both individual- and aggregate-level data alongside temporal and geospatial analyses could enhance the generalizability of these indices. The inclusion of historical vaccination data of other infectious diseases and the application of social and behavioral frameworks could address limitations of current models.

5. Conclusions

This scoping review of vaccine uptake indices found that most studies exhibited diverse characteristics of a methodologically advanced vaccine uptake index, though these characteristics were not frequently present together. These studies often utilize geospatial data, but less commonly (1) use individual-level data in tandem with administrative data, (2) integrate geospatial and temporal data, (3) utilize vaccination data sources from multiple infectious diseases, and (4) incorporate social and behavioral frameworks into their study designs. These findings suggest that future research could be performed to develop evidenced-based vaccine uptake indices for the purpose of public health intervention strategies aimed at facilitating equitable vaccination coverage.

Author Contributions

Conceptualization, K.G. and A.G.; Data curation, K.G., S.A., and A.G.; Formal analysis, K.G., S.A., and A.G.; Funding acquisition, K.G., R.H., and A.G.; Investigation, K.G. and S.A.; Methodology, K.G., S.A., R.H., and A.G.; Project administration, K.G.; Resources, K.G.; Software, K.G. and R.H.; Supervision, A.G.; Validation, K.G., S.A., R.H., and A.G.; Visualization, K.G.; Roles/Writing - original draft, K.G. and R.H.; Writing - review & editing, K.G., S.A., R.H., and A.G.

All authors attest they meet the ICMJE criteria for authorship.

Conflict of Interest Statement

The authors declare no conflict of interest. A.G. declares that the work does not relate to A.G.’s position at Amazon.

Supplements

Supplement 1: Search strings used for article retrieval

Supplement 2: Data extraction form

Supplement 3: Further reading of selected articles

Supplement 4: Complete list of articles included by the review