ABSTRACT
Background The U.S. has experienced an unprecedented number of shelter-in-place orders throughout the COVID-19 pandemic. There is limited empirical research that examines the impact of these orders. We aimed to rapidly ascertain whether social distancing; difficulty with daily activities (obtaining food, essential medications and childcare); and levels of concern regarding COVID-19 changed after the March 16, 2020 announcement of shelter-in-place orders for seven counties in the San Francisco Bay Area.
Methods We conducted an online, cross-sectional social media survey from March 14 – April 1, 2020. We measured changes in social distancing behavior; experienced difficulties with daily activities (i.e., access to healthcare, childcare, obtaining essential food and medications); and level of concern regarding COVID-19 after the March 16 shelter-in-place announcement in the San Francisco Bay Area and elsewhere in the U.S.
Results The percentage of respondents social distancing all of the time increased following the shelter-in-place announcement in the Bay Area (9.2%, 95% CI: 6.6, 11.9) and elsewhere in the U.S. (3.4%, 95% CI: 2.0, 5.0). Respondents also reported increased difficulty with obtaining food, hand sanitizer, and medications, particularly with obtaining food for both respondents from the Bay Area (13.3%, 95% CI: 10.4, 16.3) and elsewhere (8.2%, 95% CI: 6.6, 9.7). We found limited evidence that level of concern regarding the COVID-19 crisis changed following the shelter-in-place announcement.
Conclusion These results capture early changes in attitudes, behaviors, and difficulties. Further research that specifically examines social, economic, and health impacts of COVID-19, especially among vulnerable populations, is urgently needed.
Introduction
The coronavirus disease 2019 (COVID-19) pandemic began when clusters of “pneumonia of unknown etiology” were identified in December 2019. 1-5 By May 2020, there were over 3 million confirmed cases globally. One-third of these confirmed cases occurred in the United States (U.S.), with over 60,000 recorded deaths to date. 6,7
In the absence of vaccines or treatments, 8 the primary defense has been to reduce the risk of SARS-CoV-2 exposure through non-pharmaceutical interventions (NPIs) such as school closures, social distancing, isolation and quarantine, and use of personal masks. 9-13 NPIs were shown to be effective during the 2003 severe acute respiratory syndrome coronavirus (SARS-CoV) outbreak, 14 and quickly became the cornerstone of mitigation and intervention strategies for COVID-19 globally. 15-17 However, the extent and level of enforcement of these measures vary widely. 9
On March 19, 2020, California was the first U.S. state to enact a statewide shelter-in-place order, 18 following an announcement on March 16, 2020 of shelter-in-place orders for seven San Francisco Bay Area counties effective on 12:01 AM on March 17, 2020. 19 In the following weeks, 42 states and the District of Columbia passed shelter-in-place orders. 20 Potential subsequent SARS-CoV-2 wintertime outbreaks may necessitate repeated intermittent social distancing orders into 2022. 17 Given the unprecedented nature of these orders in the U.S., it is critical that we understand the impact of shelter-in-place orders on the public’s behaviors and perceptions.
For the present study, we employed convenience sampling to rapidly ascertain and summarize how levels of social distancing, difficulty related to daily activities such as obtaining food, essential medications and childcare, and levels of concern regarding the COVID-19 crisis changed after the March 16, 2020 shelter-in-place announcement among respondents living in the seven affected California counties compared with respondents living elsewhere in the U.S.
Methods
Study Sample
We conducted a cross-sectional, online survey with convenience sampling through three social media platforms (NextDoor, Twitter, and Facebook) starting on March 14, 2020 through April 1, 2020. Twitter and Facebook posts were shareable to facilitate snowball sampling. We included all respondents who completed at least 80% of the survey and excluded those missing both zip code and GeoIP location and those outside of the U.S.
Data Collection
The 21-item survey collected information regarding shelter-in-place behaviors, experienced difficulty with daily activities, level of concern, demographic characteristics, and location. Demographic information included gender (female, male, other); race/ethnicity (white, Asian/ Pacific Islander, Hispanic/Latino, Black or other); year of birth was used to create age categories (25 years or less; 26 – 45; 46 – 65; older than 65 years); education (less than high school, high school or GED, some college, bachelor’s degree); and health insurance (yes, no, don’t know). Respondents reported the number of children (<18 years) and adults over age 65 years in their household. Participants were informed of the purpose, risks, and benefits of the study.
Shelter-in-place announcement
We focused the analysis on the implications of shelter-in-place orders announced for six San Francisco Bay Area counties (San Francisco, Santa Clara, San Mateo, Marin, Contra Costa, and Alameda) and separately for Santa Cruz County made mid-day on March 16, 2020; hereafter, referred to collectively as “seven Bay Area counties”. We classified survey responses collected before March 16, 2020 as having occurred before the shelter-in-place announcement. We did so in order to more precisely identify responses that occurred before the announcement, as we anticipated that some respondents were aware of or suspected the shelter-in-place announcement several hours before it occurred. We differentiated survey respondents living in the seven affected Bay Area counties from those residing elsewhere in the U.S. using self-reported zip codes. For invalid or missing zip codes, we assigned participants’ locations based on latitude and longitude (i.e., GeoIP location, an estimation of the respondent’s location based on their IP address).
Level of concern, social distancing behaviors, and difficulties
We considered three outcomes: social distancing behaviors (all of the time, most of the time, some of the time, none of the time); experienced difficulties with daily activities (access to healthcare, childcare, transportation, job loss, or difficulty obtaining essential items including food, medications, and hand sanitizer); and level of concern regarding the COVID-19 crisis (extremely concerned, very concerned, moderately concerned, somewhat concerned, not at all concerned).
Statistical Analysis
We first summarized demographic characteristics for survey respondents living in the seven Bay Area counties affected by the shelter-in-place announcement on March 16, 2020 compared to respondents living elsewhere within the U.S.
Changes Before and After the Shelter-in-Place Announcement
We used linear probability models to assess changes in levels of social distancing, the proportion of respondents experiencing difficulty with daily activities, and level of concern regarding the COVID-19 crisis after versus before the shelter-in-place announcement separately for respondents in the seven Bay Area counties and for respondents elsewhere in the U.S. Beta coefficients were transformed to reflect percent changes in each response level after the announcement was made.
Difference-in-Differences Estimates
We used a difference-in-differences (DID) approach with linear probability models to estimate the impact of the shelter-in-place announcement. 21,22 The DID estimator compared the change in responses after versus before March 16, 2020 among respondents in the Bay Area versus elsewhere in the U.S. The DID approach assumes that any changes that occurred outside of the Bay Area reflect background or secular trends. Under the assumption that these trends would have been parallel among respondents in the Bay Area and elsewhere had the shelter-in-place announcement not occurred, the resulting DID estimates correspond to the change in each outcome attributable to the announcement itself in the Bay Area. We calculated DID estimates in the study population overall, and within subgroups defined by gender, age, and household composition (at least one child at home, at least one adult > 65 years).
Sensitivity Analyses
We considered three alternative approaches for the DID analysis. First, we compared responses from the entire state of California to those of respondents elsewhere in the U.S. Because the announcement was highly publicized on mainstream news media channels and social media platforms, survey respondents living in California outside of the seven Bay Area counties may have modified their behaviors. We therefore repeated the main analysis comparing respondents in California to respondents in other U.S. states. Second, a similar announcement also occurred in Washington state on March 16, 2020. Therefore, we repeated the main analysis with respondents from Washington state combined with Bay Area respondents. Finally, we repeated the main analysis excluding responses after March 19, 2020 – when California announced a state-wide shelter-in-place order.
We conducted all statistical analyses using R version 3.2.3 (R Foundation for Statistical Computing, Vienna, Austria). This study was approved by the Institutional Review Board at Stanford University.
Results
In total, 22,913 respondents started the survey. We excluded 4,031 respondents who completed less than 80% of the survey, 1,136 respondents with no geolocation data, and 203 international respondents. The final analytic sample included 17,543 respondents of whom 4,161 (24%) were from the seven Bay Area counties. Among respondents from the Bay Area, 2,951 (70.9%) completed the survey prior to March 16, 2020. Among respondents living elsewhere in the U.S., 8,410 (62.8%) completed the survey prior to March 16, 2020 (Table 1). Overall, the majority of respondents were younger than 66 years (N = 90%), and the majority (84%) had earned at least a bachelor’s degree. The majority of respondents were female (72%), and most (96%) had some form of health insurance. Approximately 41% of respondents indicated living with at least one child under the age of 18 years and 19% indicated living with at least one adult over the age of 65 years.
Respondents from the Bay Area were less likely to identify as non-Hispanic white as compared with other respondents (73.6% versus 86.0%) and less likely to identify as Black (0.7% versus 1.4%). Respondents from the Bay Area were more likely to be Asian or Pacific Islanders (15.1% versus 4.5%) or Hispanic/Latino (4.9% versus 4.1%). Respondents from the Bay Area were also less likely to be under age 36 years (21.1% versus 31.0%) and slightly more likely to be over age 65 years (13.9% versus 8.6%). The distribution of participants by gender, educational attainment, and household composition was similar among respondents from the Bay Area and respondents living elsewhere. We noted only minor differences between respondents who completed the survey before or after March 16, 2020 in the Bay Area or elsewhere, except for the percentage of respondents who were female and living outside of the Bay Area which was substantially lower before March 16, 2020 versus afterwards (52.5% versus 79.1%). (Table A1 in the Supplementary Materials)
Changes Before and After the Shelter-in-Place Announcement
In Table 2, we present the change in level of social distancing, difficulties experienced, and level of concern following the March 16, 2020 shelter-in-place announcement for respondents from the Bay Area and respondents living elsewhere. In general, we observed similar trends in the two groups. We found an increase in the proportion of respondents practicing social distancing all of the time after the shelter-in-place announcement in the Bay Area (9.2%, 95% CI: 6.6, 11.9) and elsewhere (3.4%, 95% CI: 2.0, 5.0). We also observed increases in the proportion sheltering in place most of the time among survey respondents from the Bay Area (5.7%, 95% CI: 2.3, 9.0) and elsewhere (8.5%, 95% CI: 6.8, 10.3). The proportion of respondents sheltering in place some of the time and none of the time decreased both among respondents from the Bay Area and elsewhere.
Respondents also reported more difficulty associated with activities such as obtaining food, hand sanitizer, and medications after the March 16, 2020 shelter-in-place announcement versus before. The increase in difficulty was largest for obtaining food for both respondents from the Bay Area (13.3%, 95% CI: 10.4, 16.3) and elsewhere (8.2%, 95% CI: 6.6, 9.7). Similarly, both groups reported greater difficulty obtaining hand sanitizer. Greater difficulty with wages was reported more frequently by respondents from the Bay Area following the shelter-in-place announcement (4.7%, 95% CI: 2.6, 6.8) and even more so by respondents living elsewhere (6.4%, 95% CI: 5.2, 7.6). Respondents in both groups were also more likely to report difficulty related to job loss following the announcement (Bay Area: 1.2%, 95% CI: 0.5, 1.8; Elsewhere: 1.6%, 95% CI 1.1, 2.0).
We observed only small changes in level of concern regarding the COVID-19 crisis after the March 16, 2020 shelter-in-place announcement among respondents in the Bay Area. Among respondents living elsewhere, we observed a decrease in the proportion of respondents reporting they were “extremely concerned” after the announcement (−4.1%, 95% CI: - 5.7, - 2.4).
Difference-in-Differences Estimates
In Table 3, we present DID estimates for the change in the proportion of respondents who were social distancing all of the time after the shelter-in-place announcement in the Bay Area versus elsewhere.
Overall, the proportion of respondents social distancing all of the time increased after the announcement in the Bay Area versus elsewhere (5.8%, 95% CI: 2.8, 8.8). Relative increases were greatest among men (9.3%, 95% CI: 3.2, 15.4), adults between the ages of 46 and 65 years (6.7%, 95% CI 1.8, 11.7), and respondents from households with children.
We calculated DID estimates for experienced difficulties in the Bay Area versus elsewhere following the shelter-in-place announcement. We noted the strongest differences for difficulty obtaining food (5.2%, 95% CI: 1.8, 8.5), followed by difficulty with transportation (2.2, 95% CI: - 1.5, 5.9) (Figure 1, Table A2). We observed limited evidence of increased difficulty with healthcare, obtaining hand sanitizer, or obtaining medications.
In Table 4 we present DID estimates for the change in the proportion of respondents who were extremely concerned following the shelter-in-place announcement in the Bay Area versus elsewhere. Overall, the proportion of respondents who reported extreme worry did not increase after the announcement for most groups, with the exception of those aged 46-65 years (8.03, 95% CI 2.03, 14.0) and respondents living with at least one child (6.20, 95% CI 0.62, 11.8). The proportion reporting extreme worry decreased in some groups including men, those under age 25, and those living outside the Bay Area.
Sensitivity Analyses
The overall pattern of results in all three sensitivity analyses was consistent with our main analysis. Effect estimates were slightly attenuated when we compared California respondents to respondents elsewhere in the U.S., or when we combined Washington state respondents with Bay Area respondents (Supplemental tables A3, A4, A5), and slightly accentuated when we excluded survey responses after March 19, 2020.
Discussion
We examined changes in attitudes and behaviors as a result of the COVID-19 crisis in a cross-sectional convenience sample of 17,543 respondents recruited through three social media platforms. Differences in key demographic characteristics (level of insurance, educational attainment, race/ethnicity) preclude generalization of our findings to the Bay Area or to the U.S. more broadly. Nevertheless, the study data capture how social distancing behaviors, difficulties with daily activities, and levels of concern regarding how COVID-19 may have changed in the days that immediately preceded and immediately followed the announcement of the nation’s first shelter-in-place order for seven Bay Area counties. This announcement occurred at a point where the seriousness of the COVID-19 pandemic for the U.S. was increasingly recognized, but the eventual impact on cities such as New York, New Orleans, and Detroit was yet to be realized. 23-26 As such, the results of this study offer some insight into our collective disposition towards the pandemic at a unique point in history as the very first shelter in place policy decisions were made.
Overall, we found that participants’ behaviors and attitudes regarding the COVID-19 pandemic evolved even within our brief survey period. After the shelter-in-place orders were announced for the Bay Area, social distancing increased. Increases in level of social distancing were more pronounced among respondents in the Bay Area versus those living elsewhere in the U.S., adults older than 46 years, and those living with children or an adult over age 65 years. This pattern may be explained by early suspicions that older adults were most vulnerable to COVID-19. 27
Consistent with media reports of food shortages, 28 respondents were most likely to report difficulty obtaining food, with increases in difficulty obtaining food more pronounced in the Bay Area following the shelter-in-place announcement. Increases in difficulty with access to healthcare, hand sanitizer, and transportation were similar among respondents in the Bay Area versus those living elsewhere. We detected the early impacts on job loss and wages, which were followed by a national surge in unemployment after the study period. 29,30 We anticipate that our findings may further underestimate the impacts of shelter-in-place on job loss and wages given the high levels of educational attainment in our study population, as may respondents may have been able to transition more easily to remote work. 31
Finally, we found that approximately one-third of respondents were “extremely concerned” about the COVID-19 crisis, although we found little evidence to support the idea that levels of concern increased – among respondents in the Bay Area or elsewhere – following the announcement of shelter-in-place orders. This raises the interesting question as to whether announcements regarding COVID-19 lead to increased or decreased levels of concern and anxiety that should be considered further in more representative study populations and as the pandemic continues to evolve.
Limitations
Despite the large number of survey respondents, older adults, Black respondents, and men were underrepresented in this convenience sample. Similarly, household structure of respondents suggests that a large number of respondents did not have children or elderly family members that may have required extra care. Recruitment was convenience sampling via three social media websites. Snowball sampling (through re-posts on Facebook and Twitter) may have further propagated participation among a more homogenous group of respondents. Our results therefore likely underrepresent the true extent of challenges associated with the pandemic across the U.S. and precludes meaningful examination of the early impacts of shelter-in-place orders on economically marginalized and vulnerable population subgroups. 32,33
The cross-sectional nature of this study represents an additional limitation. Because we did not observe changes in social distancing, experienced difficulties, and levels of concern in individuals over time, it is possible that our findings are explained at least in part by compositional effects (i.e., systematic differences in respondents who completed the survey before and after March 16th). Reassuringly, we found limited evidence of systematic differences in measured characteristics before and after the March 16th cutoff with the exception of the gender breakdown among respondents who resided outside of the Bay Area.
Finally, the announcement of shelter-in-place orders for the seven Bay Area counties was covered extensively in the national media, which makes spillover effects of the announcement to survey respondents living outside of the Bay Area – particularly elsewhere in California –likely. The assumptions of DID are therefore unlikely to be met, and our estimates are more appropriately interpreted as summary measures of the change in the Bay Area relative to the change elsewhere in the U.S. rather than causal estimates of the impact of the announcement. However, in sensitivity analyses to examine spillover in Washington State and California, we found similar pattern of findings across subgroups of interest.
Conclusions
We found evidence of increased social distancing and difficulty with daily activities such as food and transportation in the wake of the announcement of the nation’s first shelter-in-place orders, particularly among respondents in the Bay Area. Levels of concern remained fairly consistent throughout the study period among respondents in the Bay Area and elsewhere. Given that our study population was highly educated, concentrated in one of the more affluent areas in the U.S., and queried relatively early in the COVID-19 pandemic, we anticipate that our findings underestimate substantially the impact of county- and statewide shelter-in-place orders. As such, our study represents a first step towards understanding the social attitudes and consequences of this crisis. Further research that specifically examines social, economic, and health impacts of COVID-19 especially among vulnerable populations is needed.
Data Availability
Data will be provided upon request.
Funding
EL is supported by the NIH (grants DP2CA225433 and K24AR075060). MVK is supported by the National Institute on Drug Abuse (T32DA035165). LMN is supported by the Clinical and Translational Science Award Program of the National Institutes of Health’s National Center for Advancing Translational Science (UL1 TR001085). The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.