Correlates of COVID-19 vaccine acceptance, hesitancy and refusal among employees of a safety net California county health system with an early and aggressive vaccination program: Results from a cross-sectional survey
========================================================================================================================================================================================================================

* Nicole M. Gatto
* Jerusha E. Lee
* Donatella Massai
* Susanna Zamarripa
* Bijan Sasaninia
* Dhruv Khurana
* Kelsey Michaels
* Deborah Freund
* Judi Nightingale
* Anthony Firek
* for the Riverside University Health System (RUHS) Comparative Effectiveness and Clinical Outcomes Research Center (CECORC)-Claremont Graduate University (CGU) COVID-19 Research Group

## Abstract

Information on vaccine acceptance among healthcare workers is needed as health professionals provide front line care to COVID-19 patients. We developed and implemented an anonymous internet-based cross-sectional survey with direct solicitation among employees of a safety net health system. Items queried demographic and health-related characteristics, experience with and knowledge of COVID-19, and determinants of decisions to vaccinate. COVID-19 vaccine acceptance groups (acceptors, hesitant, refusers) were defined; an adapted version of the WHO vaccine hesitancy scale was included. The survey demonstrated good reliability (Cronbach’s alpha = 0.92 for vaccine hesitancy scale; 0.93 for determinants). General linear and logistic regression methods examined factors which were univariately associated with vaccine hesitancy and vaccine acceptance, respectively. Multivariable models were constructed with stepwise model-building procedures. Race/ethnicity, marital status, job classification, immunocompromised status, flu vaccination and childhood vaccination opinions independently predicted hesitancy scale scores. Gender, education, job classification and BMI independently predicted acceptance, hesitancy and refusal groups. Among hesitant employees, uncertainty was reflected in reports of motivating factors influencing their indecision. Despite a strong employee-support environment and job protection, respondents reported physical and mental health effects. Appreciation of varied reasons for refusing vaccination should lead to culturally sensitive interventions to increase vaccination rates in healthcare workers.

Keywords
*   vaccine hesitancy
*   COVID-19
*   pandemic
*   SARS-CoV-2
*   healthcare workers
*   vaccine acceptance
*   physicians
*   nurses
*   determinants

## 1. Introduction

The success of any vaccination program is dependent on a number of interconnected and interdependent actions. These include development of vaccine, testing for efficacy and safety, rapid distribution to the population and acceptance by recipients. The latter issue of vaccine uptake [1] is critical, and can be characterized as vaccine acceptance, refusal or hesitancy. Vaccine hesitancy is formally defined as a “delay in acceptance or refusal of vaccines despite availability of vaccine services”[2], and is acknowledged as complex and context-specific with the potential to vary across time and place. Factors that may influence vaccine acceptance include complacency, access to vaccination sources, confidence in the safety and efficacy and perceived intent of the agencies providing the vaccine[3].

Historically, themes have characterized perspectives among individuals who are vaccine hesitant in the United States (US)[4-6] and are not mutually exclusive. One is a distrust of government and a desire to retain individual liberties including the right to make their own decisions versus governmental-imposed actions [4,6]. Second, are philosophical beliefs that stress the purity of the human body [7,8], and further, are fears and controversies related to vaccine safety [6]. The spread of the more infectious SARS-CoV-2 delta variant in the US and the surge in new cases mainly in the unvaccinated demonstrates the liability of personal decisions on vaccine acceptance. Vaccine hesitancy is global and not limited to the current COVID-19 pandemic; it has, in part, been blamed for international resurgences of infectious disease outbreaks such as measles, which had previously been brought under control [7].

Since the emergence of the SARS-CoV-2 virus in late 2019 in Wuhan, China, there have been an excess of 216 million confirmed cases and 4.5 million deaths globally[9], with the US having the highest number of deaths at approximately 640,000 as of September 1, 2021 [10]. The pandemic has strained public health and medical systems, caused severe illness and death among a proportion of those infected overwhelming hospitals and healthcare workers, and caused major disruptions of daily life[11]. While non-pharmaceutical public health interventions (NPIs) aimed at slowing the spread of SARS-CoV-2 were key components of the early pandemic response, the availability of vaccines against COVID-19 beginning in January 2021 in the US has enhanced and strengthened primary prevention strategies. As such, vaccine uptake among populations eligible for vaccination is essential to ending the pandemic.

A consequence of the rapid development of successful vaccines during the COVID-19 pandemic has brought into refocus the issue of vaccine hesitancy, which the WHO has previously identified as one of the top ten global health threats of 2019 [12]. Surveys of the general US population between November 2020 and March 2021 prior to when the current study was conducted showed that the proportion who plan to be vaccinated has increased during this time. However, depending on the survey, 14-17% have expressed hesitancy and 10-15% report that they definitely won’t get vaccinated[13-16]. This raises concerns as to whether reaching herd immunity in the US will be achievable.

While the US population has been surveyed recurrently and assessments provide insights into factors influencing a person’s decision to accept vaccination [17-21], fewer studies of COVID-19 vaccine hesitancy have focused on healthcare workers and health system employees [22-24]. One survey between September and October 2020 found that 35% of health system employees expressed apprehension concerning the possibility of serious adverse events from vaccines, with 67% reporting they would delay COVID-19 vaccination if a vaccine became available[22]. A Kaiser Family Foundation/Washington Post survey in March 2021 indicated that 48% of healthcare workers surveyed had not been vaccinated with 12% undecided about vaccination and 18% not planning to get vaccinated[25]. A review of studies among healthcare workers which included several from the US identified common reasons provided by hesitant employees [26]. However, all surveys were conducted prior to vaccines becoming available in the US or within the first month after they were administered and suggested that the fraction of the population that is hesitant may have decreased over the one-year period covered by the review, indicating as did another study [27] that timing may have had an effect on hesitancy.

Information on vaccine acceptance, hesitancy and refusal among healthcare workers is needed as doctors, nurses, physician assistants, pharmacists and other health professionals provide front line care to COVID-19 patients. Frontline workers directly interact with patients, thereby having a high potential for exposure to the SARS-CoV-2 virus, emphasizing the essential need for protection through vaccination. In fact, healthcare workers were among the first groups to be eligible for COVID-19 vaccination in the US and California [11]. Additionally, health professionals serve as a direct and trusted source of information for patients [28], raising the question as to whether their opinions on vaccination could indirectly influence the medical advice they provide. Illustrating this concern, research among French physicians demonstrated high vaccination hesitancy translated into lower vaccine recommendations to their patients [29].

There is a present and unmet need to better understand vaccine uptake in healthcare workers and influences underlying their decisions. This is critical so that actions can be taken to both persuade them to be vaccinated and to retain their employment given potential shortages during times of spikes in the demand for hospital care. In response to this need, many states have begun to mandate vaccination among health workers [30]. As determined by earlier studies, vaccine acceptance or refusal among healthcare workers may or may not have distinct determinants from those for other populations and other types of vaccines [23,26,31]. Understanding determinants among healthcare workers and health system employees could lead to more focused worker- and patient-centered educational and other interventions, as employment retention and achieving herd immunity are critical to “ending” COVID-19 pandemic in these populations. This is particularly true in safety net medical centers that treat the most vulnerable who are at higher risk for COVID-19 complications and death. The Riverside University Health System (RUHS) medical center serves a large and highly disadvantaged predominantly multi-ethnic population. An early and aggressive COVID-19 vaccination program was initiated that virtually assured any employee access to the first vaccines. As such, this program essentially eliminated issues with employee access which have been acknowledged as barriers to vaccination [32]. Nevertheless, concern over continued hesitancy and refusal in this group remains significant particularly in light of SARS-CoV-2 variant surges.

We formed an interdisciplinary team of health professionals and designed a study with a two-fold objective. First, we assessed levels of vaccine uptake categorized as acceptance, refusal or hesitancy in RUHS employees using an anonymous internet-based cross-sectional survey with direct employee solicitation. Second, we sought to understand determinants of decisions to vaccinate and of vaccine hesitancy and refusal. We focused on potential factors for which educational and other interventions could be targeted. This research paper offers results of a survey of healthcare workers fielded after the emergency use authorization (EUA) and use of the Pfizer, Moderna and Johnson & Johnson vaccines in the US.

## 2. Materials and Methods

### 2.1 Study Design and Population

Beginning in November 2020, a collaboration was established between the Comparative Effectiveness and Clinical Outcomes Research Center (CECORC) at Riverside University Health System (RUHS) and Claremont Graduate University (CGU). RUHS is an integrated health network in Riverside County, California that includes a 439-bed county Medical Center, 10 federally qualified health centers, several primary and specialty clinics, and the departments of Behavioral and Public Health. RUHS is a safety net California county health system which serves the over 2.3 million residents of Riverside County.

A cross-sectional survey to assess vaccine hesitancy among RUHS Medical Center employees was developed using survey information from previously published surveys of US and Canadian adults [19-22]. The RUHS-CECORC/CGU team met regularly to review and revise the survey and to plan for its administration. The finalized survey instrument was adapted for administration via Survey Monkey and took about 10 minutes to complete (available in Supplemental Materials). All responses and demographic data were collected from the survey participants directly and we did not use any hospital, medical or employee records.

RUHS Medical Center employees were eligible and invited to participate in the survey by an initial email followed by three subsequent reminder emails. RUHS-CECORC staff and volunteers distributed recruitment flyers in person at the medical center three mornings a week as workers entered their place of work. A total of 2,983 employees were eventually provided the survey. All respondents consented to participate in the survey by clicking “next” after introductory text and instructions on how to complete the survey.

The study was reviewed by the RUHS Institutional Review Board and classified as exempt as all responses were collected in a de-identified manner.

### 2.2 Survey Development and Measures

#### 2.1.1. Demographic and health-related characteristics

Questions on demographic characteristics used common US Census formats for response categories. Information on current job status at RUHS was self-reported. Respondents were asked if they had ever been told by a doctor or other health care provider if they had one or more underlying health conditions which would put them at higher risk for severe COVID-19 including diabetes, hypertension, asthma, serious heart conditions, chronic lung disease, chronic kidney disease, liver disease or a weakened immune system [14,15]. Individual conditions were summed to calculate total number of underlying health conditions/comorbidities. We used self-reported height and weight to calculate body mass index (BMI) and categorize participants into underweight/normal, overweight or obese.

#### 2.1.2. Experience with COVID-19

Our survey assessed employees’ perception of their exposure to COVID-19 on a weekly basis (no direct exposure, minimal, moderate or high exposure). Items asked whether the respondent or anyone they knew had ever tested positive for COVID-19. Given the relationship to the person, participants were asked to describe the severity of their symptoms [No Symptoms, Mild (symptoms but no shortness of breath), Moderate (visited doctor but no hospital stay), Severe/Critical (hospital stay), Death]. The impact of the pandemic on the respondent’s employment/income, mental and physical health, and ability to carry out normal activities was evaluated using a Likert scale with response options “severely decreased”, “decreased”, “no effect”, “improved”, “don’t know”.

Two sets of items assessed knowledge of COVID-19. Each correct response to a question which included six true/false items about COVID-19 disease was summed to create a disease knowledge scale (possible range 0-6). A second question asked respondents to identify common symptoms of COVID-19 from among 14 presented; each of ten correctly selected items were summed to create a symptom knowledge scale (possible range 0-10). For both scales, a higher score indicated greater knowledge.

#### 2.1.3. COVID-19 vaccine acceptance groups

Based on responses to three items about intent to receive a COVID-19 vaccination, we defined three groups (vaccine acceptors, hesitant, refusers) as follows. Respondents who reported having been vaccinated against COVID-19 (either fully or partially) or who planned to be vaccinated were categorized as vaccine acceptors. Those who reported not currently being vaccinated and were uncertain whether they would be vaccinated when an opportunity arises either now or at a future date were categorized as vaccine hesitant. Respondents who reported not currently being vaccinated, did not plan to be vaccinated when an opportunity arises, and would not consider vaccination at a later date were categorized as vaccine refusers.

#### 2.1.4. Vaccine hesitancy

To measure vaccine hesitancy, we included an adapted [22] version of the validated WHO SAGE working group vaccine hesitancy scale [2] for use in adults and implemented among health system workers [22]. The vaccine hesitancy scale is constructed using responses to eleven items which asked participants to rate their opinion on a Likert scale with response options “strongly disagree”, “disagree”, “neutral”, “agree”, “strongly agree”. For three items, a “strongly agree” response indicated a higher level of vaccine hesitancy. Other items were reverse-coded so as for their interpretation to be consistent. Scores on the scale ranged from 11-55 with higher scores indicating greater vaccine hesitancy.

#### 2.1.5. Determinants of vaccination

Depending on responses to the three items querying COVID-19 vaccination intention, participants were directed in the survey to a slightly different version of questions asking them about influences or potential influences of their decision to be vaccinated. Participants were presented with seventeen different determinants ranging from contextual influences (i.e., historical, socio-cultural, political, economic or health system/institutional factors), to individual and group influences (i.e., personal perception or social/peer environment), to vaccine-specific issues (i.e., directly related to COVID-19 vaccination). Determinants related to financial incentives were formulated based on prior studies [33-36]. All other items were modeled after Reiter et al. 2020 [20]; Pogue et al. 2020 [19]; Taylor et al. 2020 [21]. Respondents were asked to rank the level of influence on their decision to be vaccinated using a Likert Scale with response options “definitely would not”, “probably would not”, “not sure”, “probably would”, “definitely would”.

#### 2.1.6. Other items

We believed it likely that individuals who did not get a flu vaccine or have their children vaccinated [37,38] were also more likely to be vaccine refusers. Thus, questions about annual influenza vaccination were included for comparison to determine whether respondents took the flu vaccine as recommended (yes, no, skip some years). Respondents were asked to rank in order of importance a series of reasons understood to motivate flu vaccination including the safety and effectiveness of the vaccines; allergies to the vaccine; desire not to infect co-workers, patients or family members. In addition, opinions about the importance of childhood vaccinations were collected among participants with children.

### 2.3 Reliability of survey

Constructed scales for knowledge of COVID-19 in the final survey were assessed for intra-rater reliability using test-retest data from five non-survey participants to calculate intraclass correlations (ICCs) of averaged scale values for each rater. The ICC for the COVID-19 symptom knowledge scale was very good (ICC = 0.87) and for the COVID-19 disease knowledge scale was excellent (ICC > 0.99).

We examined the internal consistency of the vaccine hesitancy scale and the determinants of vaccination items by calculating Cronbach’s alpha among all respondents who completed the survey. Both the vaccine hesitancy scale and the determinants items demonstrated excellent internal consistency (standardized alpha = 0.92 and 0.93, respectively).

### 2.4 Statistical Analysis

Descriptive statistics (means, frequencies) for survey participants were summarized overall and compared between groups (vaccine acceptors, hesitant, refusers) using chi-square tests, Fisher’s exact test for categorical and t-tests or ANOVAs for continuous variables. Statistical tests were two-tailed.

We used general linear regression models to examine factors that were associated with vaccine hesitancy; in these models, the score on the vaccine hesitancy scale was the dependent outcome variable. Estimated β coefficients and standard errors (SE) of β expressed the average point difference in the vaccine hesitancy scale associated with a given variable compared to the reference group of that variable. β coefficients > 1 indicated greater hesitancy for the group compared with the reference; β coefficients < 1 indicated less hesitancy. We used multinomial logistic regression to identify predictors of COVID-19 vaccine acceptance comparing refusers and (separately) hesitant employees to acceptors. Estimated odds ratios (ORs) and 95% confidence intervals (CIs) expressed the increased or decreased likelihood associated with a given variable of being vaccine hesitant or a vaccine refuser compared with being an acceptor. Some variables were re-categorized due to small numbers of responders to that question. In model building approaches, all variables with global p < 0.10 in univariate analyses or for which the literature supported a relationship with vaccine hesitancy [6,8] were entered into a preliminary multivariable model. We then used stepwise procedures to retain variables with p < 0.05 in final multivariable linear and logistic models.

Responses to each of the seventeen different determinants influencing participants’ decision to be vaccinated were compared between the three groups of participants (vaccine acceptors, hesitant, refusers) using Kruskal-Wallis tests with Bonferroni adjustment to account for multiple testing (with the adjusted α set at 0.003). Effects of the COVID-19 pandemic on employment, income and health were similarly compared between the three groups without Bonferroni adjustment. Analyses used SAS version 9.4 (SAS Institute Inc., Cary, NC, USA) or RStudio version 1.3 1093 (2009-2020 RStudio, PBC, Boston, MA, USA).

## 3. Results

The survey was administered from March 15 – April 26, 2021 to 2,983 RUHS Medical Center employees; 791 surveys were returned for a 27% response rate. After excluding 2 records because most responses were blank or in one case, appeared to be fictitious, 789 remained. Respondents were predominantly female (79.2%), between the ages of 30-64 (83.7%), non-Hispanic white (37.7%) or Hispanic (36.8%) and had a self-reported education level of a college degree or higher (59.7%). Of those responding to the survey, 755 (95.6%) answered questions enabling a categorization into groups of vaccine acceptors, hesitant, or refusers. Vaccine hesitant and refusers were more likely to be women and to have an annual household income of less than $50,000. Refusers were more likely to be in the 30-49-year age range. A greater proportion of vaccine hesitant had less than a college degree, and both hesitant and refusers had lower proportions with a college degree or higher (Table 1).

View this table:
[Table 1.](http://medrxiv.org/content/early/2021/09/20/2021.09.14.21263588/T1)

Table 1. Characteristics of RUHS Medical Center Employees (n=789) who Participated in COVID-19 Vaccine Survey, by Vaccine Acceptance Status (Acceptors, Hesitant, Refusers)

Job classifications of survey respondents were generally reflective of the overall makeup of medical center employees of 45% nurses, 5% physicians, 19% ancillary and 30% non-medical personnel, based on RUHS human resources data. Higher proportions of nurses, nursing assistants and medical assistants were among the vaccine refusers and higher proportions of administrative, non-clinical staff were among the vaccine hesitant. Physicians and allied health personnel were more likely to be in the vaccine acceptor group. Respondents were approximately evenly distributed by level of exposure to COVID-19 on a weekly basis (Table 1). Eighty-nine percent either had themselves or knew someone who had tested positive for COVID-19.

Twenty-seven percent of respondents reported having one chronic condition, with hypertension being the most common (24.5%), followed by asthma (16.7%) and diabetes (7.5%), and 72% had BMIs in the overweight or obese range, based on self-reported height and weight. Vaccine hesitant and refusers were less likely to have hypertension than vaccine acceptors (p = 0.01), but individuals who were vaccine hesitant were more likely to have asthma (p = 0.05).

Seventy-nine percent of respondents received the flu vaccine each year, which differed markedly between groups, with 85% of acceptors, 52% of hesitant and 17% of refusers reporting flu vaccination annually. Among respondents with children, 93% overall reported believing in the importance of having their children vaccinated against childhood diseases. This differed significantly between groups with nearly all of vaccine acceptors (97%), 86% of vaccine hesitant, yet just over half (52%) of vaccine refusers reporting affirmatively.

Respondents as a group were knowledgeable about both COVID-19 symptoms and disease, with average scores on corresponding scales near the upper bound of the total possible. Vaccine hesitant had lower knowledge of COVID-19 disease compared with acceptors and refusers (p=0.0003).

Scores on the vaccine hesitancy scale tracked, as anticipated, based on the three categories of respondents. Overall, respondents’ scores on the vaccine hesitancy scale were 21.7 out of 55, with scale scores differentiating well between the three groups (acceptors, hesitant, refusers) (Table 1). Scores were lowest among vaccine acceptors, while vaccine refusers had the highest scores, indicating the greatest hesitancy. An average of 10 points separated the vaccine hesitant from the vaccine acceptors, and an average of 18 separated the vaccine refusers from the vaccine acceptors (p <0.0001).

Eleven of the 23 characteristics were individually associated with vaccine hesitancy as assessed with the vaccine hesitancy scale: gender, race/ethnicity, marital status, educational level, annual household income, job classification, weekly exposure to COVID-19, annual flu vaccination, importance of childhood vaccinations and COVID-19 disease knowledge (Table 2). Interestingly, several factors were not correlated with hesitancy including age, testing positive or knowing someone who tested positive for COVID-19, BMI, or knowledge of COVID-19 symptoms. Despite well-characterized associations with medical complications and poor outcomes [39], no specific chronic condition nor the total number of chronic conditions were associated with vaccine hesitancy in our healthcare worker population. In multivariable models, race/ethnicity, marital status, job classification, immunocompromised status, annual flu vaccination and importance of childhood vaccinations were significant independent predictors of vaccine hesitancy. Asian respondents were more likely to be hesitant than non-Hispanic whites (β = 1.63; p = 0.06) and immunocompromised persons were more hesitant than those who were not (β = 2.36; p = 0.03). Compared to doctors, physician assistants and nurse practitioners, all other job classifications had higher vaccine hesitancy scores.

View this table:
[Table 2.](http://medrxiv.org/content/early/2021/09/20/2021.09.14.21263588/T2)

Table 2. Univariate and multivariate associations (β [SE (β)]; p-value) between potential predictors of the vaccine hesitancy scale from general linear regression models

Acceptance of the flu vaccine was predictive of vaccine hesitancy, as employees who reported regularly receiving the flu vaccine scored an average of 8.23 points higher on the vaccine hesitancy scale (p < 0.0001) compared to those that did not, and persons who skipped some years scored an average of 4.49 points higher (p < 0.0001).

Employees who did not believe it was important to have their children vaccinated against childhood diseases scored nearly 13 points higher on the vaccine hesitancy scale compared to those who did have this belief (p < 0.0001), and those who were not sure about childhood vaccinations scored over 9 points higher (p < 0.0001). Respondents who were previously married were less likely to be hesitant than those who were currently single (β = -2.35; p = 0.02)

Several characteristics were individually associated with vaccine acceptance status including age, gender, educational level, job classification, BMI, testing positive or knowing someone who tested positive for COVID-19, hypertension, asthma, COVID-19 disease knowledge, annual flu vaccination, and importance of childhood vaccinations (Table 3). ORs estimated for the latter two had very wide confidence intervals due to the small number of observations in cells despite re-categorizing; as such, these variables were not further pursued in models building approaches. In multivariable models, gender, educational level, job classification and BMI were significantly independently predictive of vaccine acceptance, hesitancy and refusal. Compared with women, men were significantly less likely to be vaccine hesitant (OR = 0.12; 95% CI = 0.02, 0.91). Persons with a college degree or higher were 55% less likely to be vaccine hesitant compared to those with less than a college degree (OR = 0.44; 95% CI = 0.22, 0.89) and employees with a BMI classified as obese were 65% less likely to be vaccine hesitant compared to those with a BMI in the normal/underweight range (OR = 0.35; 95% CI = 0.14, 0.90). Non-clinical staff were more than twice as likely to be vaccine hesitant than clinical employees (OR = 2.31; 95% CI = 1.16, 4.59). Associations for these characteristics were apparent and in the same direction for vaccine refusers with the exception of BMI but did not achieve statistical significance likely because of the small numbers of refusers.

View this table:
[Table 3.](http://medrxiv.org/content/early/2021/09/20/2021.09.14.21263588/T3)

Table 3. Univariate and multivariate associations (ORs, 95% CIs) between potential predictors of the vaccine acceptance, hesitancy or refusal from multinomial logistic regression models

Employees at RUHS Medical Center who accepted COVID-19 vaccination reported several influences of their decision to be vaccinated, including protection of the vulnerable, encouragement from family members or colleagues, and advice from a health care worker. They also prioritized safety and efficacy of the vaccine and job or other requirements as determining factors. For those employees who accepted vaccination, financial incentives, raffles, social media and religious leaders were not motivating factors (Figure 1). Among RUHS employees who were hesitant, uncertainty was also prevalent in their reports of motivating factors influencing their indecision to accept or refuse vaccination, with many responses tending to concentrate along the middle of the Likert scale. Employees who refused vaccination, on the other hand, showed a very different pattern in their responses. In this group, all possible reasons were ranked as not impacting a potential decision to be vaccinated. Additionally, refusers indicated that financial incentives would not increase their likelihood of being vaccinated.

![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/09/20/2021.09.14.21263588/F1.medium.gif)

[Figure 1.](http://medrxiv.org/content/early/2021/09/20/2021.09.14.21263588/F1)

Figure 1. Influences of Decision for COVID-19 Vaccination with Average Score on Likert Scale for Each Response, by Group.

Despite a strong employee-support environment and job protection, respondents reported that the COVID-19 pandemic had affected their physical and mental health and well-being as well as their ability to carry out normal activities (Figure 2). Paradoxically, vaccine refusers reported their physical health was less affected than that reported by acceptors or hesitant. Both family income and employment were less impacted for all groups.

![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/09/20/2021.09.14.21263588/F2.medium.gif)

[Figure 2.](http://medrxiv.org/content/early/2021/09/20/2021.09.14.21263588/F2)

Figure 2. Reported effects of the COVID-19 pandemic on employment, income, health and normal activities, by group.

## 4. Discussion

In our comprehensive survey of employees of a large safety net county health system conducted between March and April 2021 when COVID-19 vaccinations were being administered in the US, we found that 9.4% overall were hesitant and 5.3% refused to be vaccinated, which is consistent with studies that have estimated 8-18% hesitancy [3]. These proportions contrast markedly from general population estimates of 20-30% resistant or refusing vaccination [13-16,23].

Similar to previous research among US healthcare workers [22,40-45], our data suggest that employees who hesitated or refused to be vaccinated against COVID-19 were more likely to be women, of younger ages, and to have lower levels of education and income. Also analogous to other studies that found less hesitancy among healthcare workers in direct patient care roles[22,42-45], we observed that employees with high or moderate weekly exposure to COVID-19 such as from working in the COVID ward or ICU in any capacity were less likely to be hesitant. Unlike some previous surveys [22,42-45] [46] we did not find lower hesitancy among Asian healthcare workers. One possible explanation for this difference is that the makeup of our study population within the Asian racial group is not comparable to those of previous research, which may be supported by another California-based study [23]. Our survey did not query the specific Asian designation i.e., Filipino, Chinese etc. nor did previous studies provide data to assess this.

One of our objectives was to potentially identify reasons underlying vaccine hesitancy and refusal in healthcare workers which could be targeted by interventions. As suggested by others [47], our findings provide the assessment of barriers to vaccination adoption in an organizational setting which can be used to identify evidence-based strategies to increase COVID-19 vaccine uptake. Based on responses from employees who were hesitant to be vaccinated, we suggest campaigns which focus on providing information and reassurances[48] regarding the safety and efficacy of COVID-19 vaccines. Efforts directed at building trust in vaccine manufacturers could also be useful as well as vaccine mandates as a condition of employment. We see an opportunity in educational programs given that employees who were vaccine hesitant had lower knowledge of COVID-19 disease, despite employment within a health system. Our data further suggest that messaging from colleagues or family members, within social media networks or by religious leaders is less likely to sway hesitant health system employees to be vaccinated. This finding can be supported by a similar study [41] which found that social media is not perceived a valuable source of vaccine information for healthcare workers given social media networks are platforms found to disseminate extensive vaccine misinformation[49]. The response pattern among those who refused to be vaccinated indicates that converting these persons will likely be more challenging as no one reason emerged as a potential candidate for a targeted intervention. Similar to previous research [42,45,46], acceptance of annual flu vaccination and the importance of childhood vaccination were significant predictors of COVID-19 vaccine hesitancy and refusal in our population. These likely reflect longer standing beliefs which may be more difficult to modify. Therefore, in order to address vaccine hesitancy and increase COVID-19 vaccine uptake, it will be important to examine beliefs of healthcare workers with pre-existing concerns about vaccines in general.

Response bias may be a consideration as participants in the survey did so voluntarily. We suspect that both vaccine hesitant and refusers are higher within our employees as these groups may not have responded to the survey since they may hold underlying concerns and suspicions related to the intention of the questions despite reassurance of anonymity. Additionally, these two groups may not have completed a survey because of perceived stigmas associated with or the social undesirability of their responses. On the other hand, those that have accepted vaccination may find little personal value in completing a survey that largely has no direct relevance to their decision. In sum, it is likely that vaccine hesitant and refusers are underestimated in our study, and those who did participate may not be representative of all RUHS health system employees.

It is important to contextualize the survey with the historical events occurring contemporaneously with when data were being collected in March and April 2021. Globally, countries marked the 1-year anniversary of the COVID-19 pandemic and the WHO released a report on the potential origin of the virus in China [2]. Debate about the origin of the SARS-CoV-2 virus continued to evolve and was driven by deep-seated political beliefs. In the US, three vaccines had EUA by the FDA, two had been in use since January 2021. The Biden administration announced the purchase of additional Johnson & Johnson vaccines in order to expand supply to have sufficient vaccines for all US adults by the end of May 2021. This preceded the pause in use of their vaccine over concerns of blood clotting. Pfizer & Modera mRNA vaccines were found to prevent infection not just illness, and both manufacturers began trials on children aged 6 months – 11 years. In California, vaccine eligibility was expanded to additional groups, and some “lockdown” measures were relaxed as some counties moved into less restrictive tiers. Changing public health recommendations as well as the scientific complexity related to the novel coronavirus and our understanding as to how the virus adapts impacted the perceived confidence of the general population for vaccination. Different from previous studies among healthcare workers, by the time our survey was administered, vaccination against COVID-19 was not a hypothetical. The majority of published US studies[22,24,42,44,45] that surveyed healthcare workers were during periods when vaccines were still under development; others coincided with early vaccination efforts of healthcare workers and nursing home residents[40,41,43,46]. Moreover, our Medical Center had an early and robust employee focused vaccination program at the time the survey was administered.

The findings of this study should be interpreted cautiously given that they are based on data from 71 employees whose survey responses indicated they were hesitant to be vaccinated and 40 whose responses indicated they refused vaccination. As with other cross-sectional surveys, we do not have longitudinal data to examine if positions about COVID-19 vaccination changed over time among our health system employee cohort. Thus, the survey responses reflect a “snapshot” of opinions at one point in time and should be contextualized as described above. We plan a follow-up survey of employees to examine change in vaccine acceptance. One major strength of our study over previous research is our multivariable modeling approach which identifies factors that are independent predictors of vaccine hesitancy and refusal. This is useful because individual factors are often correlated, eg., education, income, and job title, and may be reflective of a common underlying construct. An additional strength of our work is our implementation of two measures of vaccine hesitancy, one derived from survey items and a second from a validated vaccine hesitancy scale [2]. Furthermore, we demonstrated that essential components of our survey instrument were reliable.

## 5. Conclusions

Vaccine hesitancy including refusing vaccination is a major global concern and is not novel to the COVID-19 pandemic [4-6]. Continued attempts to reassure people of the efficacy and safety of vaccines and to accept vaccination for influenza and other seasonal virus infections has been an ongoing public health effort for decades [50]. This study demonstrates that healthcare workers are not immune to concerns related to vaccination. A troubling finding of our study is the effect of the pandemic on wellbeing and work performance within health system employees despite strong benefit support, salary protection, healthcare benefits and continued employment assurance. Implications of the reported short-term effects and potential for long-term consequences merit further investigation.

Unvaccinated persons are both victims and culprits of SARS-CoV-2. The notion that the virus would be completely controlled with high vaccination rates has been challenged by the emergence of more infectious variants like delta together with significant proportions of vaccine hesitant persons in populations. This further raises concerns that additional variants may challenge the immune protection now afforded by current vaccines. Therefore, it is important to understand the motivations and beliefs of those not accepting vaccinations and to develop interventions that may increase acceptance, particularly among healthcare workers who are in positions to influence others. We point to our observation that diverged from most previous work, of greater hesitancy among Asian health system employees as a reason why a “one-size fits all” approach will not suffice as others similarly advocate [51]. Future research surveys and other quantitative analyses should be accompanied by qualitative research aimed at discovering more information on why individuals have refused vaccinations and more importantly, what can be done about it. We recommend that focus groups of refusers and vaccine hesitant healthcare workers lead to a thoughtful and deep probing of reasons and a full discussion of strategies suggested by participants that could result in their being vaccinated. Consideration should be given to the diversity within ethnic and racial groups as relates to cultural practices and beliefs about vaccination. An appreciation that there may be varied reasons for refusing vaccination should lead to more particularized culturally sensitive interventions to successfully increase vaccination rates.

## Supporting information

Supplemental Materials [[supplements/263588_file02.pdf]](pending:yes)

## Data Availability

The data presented in this study are available on request from the corresponding author.

## Supplementary Materials

The following are available online at [www.mdpi.com/xxx/s1](http://www.mdpi.com/xxx/s1), Survey Instrument.

## Author Contributions

Conceptualization, N.M.G., J.E.L., S.Z., B.S., D.F., A.F.; methodology, N.M.G., J.E.L., S.Z., B.S., D.K., D.F., A.F.; software, N.M.G., D.M., J.E.L., D.K., S.Z., B.S.; validation, N.M.G., formal analysis, N.M.G., D.M., J.E.L., D.K.; resources, J.N., A.F.; data curation, S.Z., D.K.; writing—original draft preparation, N.M.G., A.F.; writing—review and editing, N.M.G., J.E.L., D.M., B.S., D.K., K.M., D.F., J.N., A.F.; visualization, N.M.G.; supervision, N.M.G., D.F., A.F. All authors have read and agreed to the published version of the manuscript.

## Funding

This research received no external funding.

## Institutional Review Board Statement

The study was reviewed by the RUHS Institutional Review Board and classified as exempt as all responses were collected in a de-identified manner.

## Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

## Data Availability Statement

The data presented in this study are available on request from the corresponding author.

## Conflicts of Interest

The authors declare no conflict of interest.

## Acknowledgments

We thank employees of RUHS for their participation in this study.

*   Received September 14, 2021.
*   Revision received September 14, 2021.
*   Accepted September 20, 2021.


*   © 2021, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  1.Piltch-Loeb, R.; DiClemente, R. The Vaccine Uptake Continuum: Applying Social Science Theory to Shift Vaccine Hesitancy. Vaccines 2020, 8, doi:10.3390/vaccines8010076.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/vaccines8010076&link_type=DOI) 

2.  2.World Health Organization. Report of the SAGE Working Group on Vaccine Hesitancy WHO: GENEVA, 01 October 2014.
    
    

3.  3.Larson, H.J.; Jarrett, C.; Eckersberger, E.; Smith, D.M.D.; Paterson, P. Understanding vaccine hesitancy around vaccines and vaccination from a global perspective: A systematic review of published literature, 2007–2012. Vaccine 2014, 32, 2150-2159, doi:[https://doi.org/10.1016/j.vaccine.2014.01.081](https://doi.org/10.1016/j.vaccine.2014.01.081).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2014.01.081&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24598724&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F09%2F20%2F2021.09.14.21263588.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000335632200004&link_type=ISI) 

4.  4.Gostin, L.O. Jacobson v Massachusetts at 100 years: police power and civil liberties in tension. Am J Public Health 2005, 95, 576-581, doi:10.2105/AJPH.2004.055152.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2105/AJPH.2004.055152&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15798112&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F09%2F20%2F2021.09.14.21263588.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000227998700011&link_type=ISI) 

5.  5.Schwartz, J.L. New media, old messages: themes in the history of vaccine hesitancy and refusal. Virtual Mentor 2012 Jan 1, 50–55, doi: doi: 10.1001/virtualmentor.2012.14.1.mhst1-1201..
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/virtualmentor.2012.14.1.mhst1-1201&link_type=DOI) 

6.  6.Wolfe, R.M.; Sharp, L.K. Anti-vaccinationists past and present. BMJ 2002, 325, 430, doi:10.1136/bmj.325.7361.430.
    
    [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMjUvNzM2MS80MzAiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wOS8yMC8yMDIxLjA5LjE0LjIxMjYzNTg4LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

7.  7.Benecke, O.; DeYoung, S.E. Anti-Vaccine Decision-Making and Measles Resurgence in the United States. Global pediatric health 2019, 6, 2333794×19862949, doi:10.1177/2333794×19862949.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/2333794x19862949&link_type=DOI) 

8.  8.Pelčić, G.; Karačić, S.; Mikirtichan, G.L.; Kubar, O.I.; Leavitt, F.J.; Cheng-Tek Tai, M., Morishita, N.; Vuletić, S.; Tomašević, L. Religious exception for vaccination or religious excuses for avoiding vaccination. Croat Med J 2016, 57, 516–521, doi:10.3325/cmj.2016.57.516.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3325/cmj.2016.57.516&link_type=DOI) 

9.  9.World Health Organization. Novel Coronavirus (2019-nCoV) Situation Report - 22. 2020.
    
    

10. 10.Johns Hopkins University & Medicine. Coronavirus Resource Center. Available online: [https://coronavirus.jhu.edu/](https://coronavirus.jhu.edu/) (accessed on March 2020).
    
    

11. 11.Cooper, M. Tracking the Impact of the Coronavirus on the U.S. The New York Times Last Updated March 15, 2020, 9:37 p.m. ET 2020.
    
    

12. 12.World Health Organization. Ten threats to global health in 2019. Available online: [https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019](https://www.who.int/news-room/spotlight/ten-threats-to-global-health-in-2019) (accessed on August 1).
    
    

13. 13.File, T.M.; Abinash. Around Half of Unvaccinated Americans Indicate They Will “Definitely” Get COVID-19 Vaccine. America Counts: Stories Behind the Numbers 2021.
    
    

14. 14.Funk, C.T.A. Growing Share of Americans Say They Plan To Get a COVID-19 Vaccine – or Already Have. Pew Research Center 2021.
    
    

15. 15.Huetteman, E. Covid Vaccine Hesitancy Drops Among All Americans, New Survey Shows. Kaiser Health News 2021.
    
    

16. 16.Stobbe, M.F.; H.. AP-NORC poll: A third of US adults skeptical of COVID shots. AP News 2021.
    
    

17. 17.Fisher, K.A.; Bloomstone, S.J.; Walder, J.; Crawford, S.; Fouayzi, H.; Mazor, K.M. Attitudes Toward a Potential SARS-CoV-2 Vaccine : A Survey of U.S. Adults. Annals of internal medicine 2020, 173, 964–973, doi:10.7326/m20-3569.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/m20-3569&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F09%2F20%2F2021.09.14.21263588.atom) 

18. 18.Kreps, S.; Prasad, S.; Brownstein, J.S.; Hswen, Y.; Garibaldi, B.T.; Zhang, B.; Kriner, D.L. Factors Associated With US Adults’ Likelihood of Accepting COVID-19 Vaccination. JAMA network open 2020, 3, e2025594, doi:10.1001/jamanetworkopen.2020.25594.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2020.25594&link_type=DOI) 

19. 19.Pogue, K.; Jensen, J.L.; Stancil, C.K.; Ferguson, D.G.; Hughes, S.J.; Mello, E.J.; Burgess, R.; Berges, B.K.; Quaye, A.; Poole, B.D. Influences on Attitudes Regarding Potential COVID-19 Vaccination in the United States. Vaccines 2020, 8, doi:10.3390/vaccines8040582.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/vaccines8040582&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33022917&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F09%2F20%2F2021.09.14.21263588.atom) 

20. 20.Reiter, P.L.; Pennell, M.L.; Katz, M.L. Acceptability of a COVID-19 vaccine among adults in the United States: How many people would get vaccinated? Vaccine 2020, 38, 6500–6507, doi:10.1016/j.vaccine.2020.08.043.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2020.08.043&link_type=DOI) 

21. 21.Taylor, S.; Landry, C.A.; Paluszek, M.M.; Rachor, G.S.; Asmundson, G.J.G. Worry, avoidance, and coping during the COVID-19 pandemic: A comprehensive network analysis. Journal of anxiety disorders 2020, 76, 102327, doi:10.1016/j.janxdis.2020.102327.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.janxdis.2020.102327&link_type=DOI) 

22. 22.Gadoth, A.; Halbrook, M.; Martin-Blais, R.; Gray, A.; Tobin, N.H.; Ferbas, K.G.; Aldrovandi, G.M.; Rimoin, A.W. Cross-sectional Assessment of COVID-19 Vaccine Acceptance Among Health Care Workers in Los Angeles. Annals of internal medicine 2021, 174, 882–885, doi:10.7326/m20-7580.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/m20-7580&link_type=DOI) 

23. 23.Grumbach, K.; Judson, T.; Desai, M.; Jain, V.; Lindan, C.; Doernberg, S.B.; Holubar, M. Association of Race/Ethnicity With Likeliness of COVID-19 Vaccine Uptake Among Health Workers and the General Population in the San Francisco Bay Area. JAMA internal medicine 2021, 181, 1008–1011, doi:10.1001/jamainternmed.2021.1445.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamainternmed.2021.1445&link_type=DOI) 

24. 24.Meyer, M.N.; Gjorgjieva, T.; Rosica, D. Trends in Health Care Worker Intentions to Receive a COVID-19 Vaccine and Reasons for Hesitancy. JAMA network open 2021, 4, e215344, doi:10.1001/jamanetworkopen.2021.5344.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2021.5344&link_type=DOI) 

25. 25.KFF/Post Survey of Frontline Health Care Workers Finds Nearly Half Remain Unvaccinated. Kaiser Family Foundation/The Washington Post 2021.
    
    

26. 26.Biswas, N.; Mustapha, T.; Khubchandani, J.; Price, J.H. The Nature and Extent of COVID-19 Vaccination Hesitancy in Healthcare Workers. Journal of community health 2021, 1–8, doi:10.1007/s10900-021-00984-3.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s10900-021-00984-3&link_type=DOI) 

27. 27.Halbrook, M.; Gadoth, A.; Martin-Blais, R.; Gray, A.N.; Kashani, S.; Kazan, C.; Kane, B.; Tobin, N.H.; Ferbas, K.G.; Aldrovandi, G.M.; et al. Longitudinal assessment of COVID-19 vaccine acceptance and uptake among frontline medical workers in Los Angeles, California. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America 2021, doi:10.1093/cid/ciab614.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciab614&link_type=DOI) 

28. 28.Katzman, J.G.; Katzman, J.W. Primary Care Clinicians as COVID-19 Vaccine Ambassadors. Journal of primary care & community health 2021, 12, 21501327211007026, doi:10.1177/21501327211007026.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/21501327211007026&link_type=DOI) 

29. 29.Wilson, R.J.I.; Vergélys, C.; Ward, J.; Peretti-Watel, P.; Verger, P. Vaccine hesitancy among general practitioners in Southern France and their reluctant trust in the health authorities. International journal of qualitative studies on health and well-being 2020, 15, 1757336, doi:10.1080/17482631.2020.1757336.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/17482631.2020.1757336&link_type=DOI) 

30. 30.Newitt, P. 8+ states mandating vaccines for healthcare workers, state employees. Becker’s ASC Review 2021.
    
    

31. 31.Karafillakis, E.; Larson, H.J. The paradox of vaccine hesitancy among healthcare professionals. Clinical microbiology and infection : the official publication of the European Society of Clinical Microbiology and Infectious Diseases 2018, 24, 799–800, doi:10.1016/j.cmi.2018.04.001.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.cmi.2018.04.001&link_type=DOI) 

32. 32.World Health Organization WHO technical advisory group on behavioural insights and sciences for health. Behavioural considerations for acceptance and uptake of covid-19 vaccines. Meeting Report., WHO: Geneva, 15 October 2020.
    
    

33. 33.Achat, H.; McIntyre, P.; Burgess, M. Health care incentives in immunisation. Australian and New Zealand journal of public health 1999, 23, 285–288, doi:10.1111/j.1467-842x.1999.tb01257.x.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1467-842X.1999.tb01257.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10388173&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F09%2F20%2F2021.09.14.21263588.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000080940500013&link_type=ISI) 

34. 34.Higgins, S.T.; Klemperer, E.M.; Coleman, S.R.M. Looking to the empirical literature on the potential for financial incentives to enhance adherence with COVID-19 vaccination. Preventive medicine 2021, 145, 106421, doi:10.1016/j.ypmed.2021.106421.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ypmed.2021.106421&link_type=DOI) 

35. 35.Lin, C.J.; Nowalk, M.P.; Zimmerman, R.K. Estimated costs associated with improving influenza vaccination for health care personnel in a multihospital health system. Joint Commission journal on quality and patient safety 2012, 38, 67–72, doi:10.1016/s1553-7250(12)38009-4.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/s1553-7250(12)38009-4&link_type=DOI) 

36. 36.Wilson, F.A.; Wang, Y.; Stimpson, J.P. Universal paid leave increases influenza vaccinations among employees in the U.S. Vaccine 2014, 32, 2441–2445, doi:10.1016/j.vaccine.2014.02.084.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2014.02.084&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24631069&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F09%2F20%2F2021.09.14.21263588.atom) 

37. 37.Paterson, P.; Chantler, T.; Larson, H.J. Reasons for non-vaccination: Parental vaccine hesitancy and the childhood influenza vaccination school pilot programme in England. Vaccine 2018, 36, 5397–5401, doi:10.1016/j.vaccine.2017.08.016.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2017.08.016&link_type=DOI) 

38. 38.Santibanez, T.A.; Nguyen, K.H.; Greby, S.M.; Fisher, A.; Scanlon, P.; Bhatt, A.; Srivastav, A.; Singleton, J.A. Parental Vaccine Hesitancy and Childhood Influenza Vaccination. Pediatrics 2020, 146, doi:10.1542/peds.2020-007609.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTc6IjE0Ni82L2UyMDIwMDA3NjA5IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDkvMjAvMjAyMS4wOS4xNC4yMTI2MzU4OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

39. 39.Kompaniyets L, P.A.; Goodman AB; Rosenblum HG; Belay B; Ko JY; et al. Underlying Medical Conditions and Severe Illness Among 540,667 Adults Hospitalized With COVID-19, March 2020–March 2021.. Prev Chronic Dis 2021, 18, doi:[http://dx.doi.org/10.5888/pcd18.210123](http://dx.doi.org/10.5888/pcd18.210123).
    
    

40. 40.Amin, D.P.; Palter, J.S. COVID-19 vaccination hesitancy among healthcare personnel in the emergency department deserves continued attention. The American journal of emergency medicine 2021, doi:10.1016/j.ajem.2021.01.089.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ajem.2021.01.089&link_type=DOI) 

41. 41.Kociolek, L.K.; Elhadary, J.; Jhaveri, R.; Patel, A.B.; Stahulak, B.; Cartland, J. Coronavirus disease 2019 vaccine hesitancy among children’s hospital staff: A single-center survey. Infection control and hospital epidemiology 2021, 42, 775–777, doi:10.1017/ice.2021.58.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1017/ice.2021.58&link_type=DOI) 

42. 42.Kuter, B.J.; Browne, S.; Momplaisir, F.M.; Feemster, K.A.; Shen, A.K.; Green-McKenzie, J.; Faig, W.; Offit, P.A. Perspectives on the receipt of a COVID-19 vaccine: A survey of employees in two large hospitals in Philadelphia. Vaccine 2021, 39, 1693–1700, doi:10.1016/j.vaccine.2021.02.029.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2021.02.029&link_type=DOI) 

43. 43.Schrading, W.A.; Trent, S.A.; Paxton, J.H.; Rodriguez, R.M.; Swanson, M.B.; Mohr, N.M.; Talan, D.A. Vaccination rates and acceptance of SARS-CoV-2 vaccination among U.S. emergency department health care personnel. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine 2021, 28, 455–458, doi:10.1111/acem.14236.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/acem.14236&link_type=DOI) 

44. 44.Shaw, J.; Stewart, T.; Anderson, K.B.; Hanley, S.; Thomas, S.J.; Salmon, D.A.; Morley, C. Assessment of U.S. health care personnel (HCP) attitudes towards COVID-19 vaccination in a large university health care system. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America 2021, doi:10.1093/cid/ciab054.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciab054&link_type=DOI) 

45. 45.Shekhar, R.; Sheikh, A.B.; Upadhyay, S.; Singh, M.; Kottewar, S.; Mir, H.; Barrett, E.; Pal, S. COVID-19 Vaccine Acceptance among Health Care Workers in the United States. Vaccines 2021, 9, doi:10.3390/vaccines9020119.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/vaccines9020119&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33546165&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F09%2F20%2F2021.09.14.21263588.atom) 

46. 46.Ciardi, F.; Menon, V.; Jensen, J.L.; Shariff, M.A.; Pillai, A.; Venugopal, U.; Kasubhai, M.; Dimitrov, V.; Kanna, B.; Poole, B.D. Knowledge, Attitudes and Perceptions of COVID-19 Vaccination among Healthcare Workers of an Inner-City Hospital in New York. Vaccines 2021, 9, doi:10.3390/vaccines9050516.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3390/vaccines9050516&link_type=DOI) 

47. 47.Finney Rutten, L.J., Zhu, X.; Leppin, A.L.; Ridgeway, J.L.; Swift, M.D.; Griffin, J.M.; St Sauver, J.L.; Virk, A.; Jacobson, R.M. Evidence-Based Strategies for Clinical Organizations to Address COVID-19 Vaccine Hesitancy. Mayo Clinic proceedings 2021, 96, 699–707, doi:10.1016/j.mayocp.2020.12.024.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.mayocp.2020.12.024&link_type=DOI) 

48. 48.Goldstein, S.; MacDonald, N.E.; Guirguis, S. Health communication and vaccine hesitancy. Vaccine 2015, 33, 4212–4214, doi:10.1016/j.vaccine.2015.04.042.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.vaccine.2015.04.042&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25896382&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F09%2F20%2F2021.09.14.21263588.atom) 

49. 49.Puri, N.; Coomes, E.A.; Haghbayan, H.; Gunaratne, K. Social media and vaccine hesitancy: new updates for the era of COVID-19 and globalized infectious diseases.
    
    

50. 50.Schmid, P.; Rauber, D.; Betsch, C.; Lidolt, G.; Denker, M.L. Barriers of Influenza Vaccination Intention and Behavior – A Systematic Review of Influenza Vaccine Hesitancy, 2005 - 2016. PloS one 2017, 12, e0170550, doi:10.1371/journal.pone.0170550.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0170550&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F09%2F20%2F2021.09.14.21263588.atom) 

51. 51.Chou, W.S.; Budenz, A. Considering Emotion in COVID-19 Vaccine Communication: Addressing Vaccine Hesitancy and Fostering Vaccine Confidence. Health communication 2020, 35, 1718–1722, doi:10.1080/10410236.2020.1838096.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/10410236.2020.1838096&link_type=DOI)