Abstract
Without vaccines, non-pharmaceutical interventions have been the most widely used approach to controlling the spread of COVID-19 epidemics. Various jurisdictions have implemented public health orders as a means of reducing effective contacts and controlling their local epidemics. Multiple studies have examined the effectiveness of various orders (e.g. use of face masks) for epidemic control. However, orders occur at different timings across jurisdictions and some orders on the same topic are stricter than others. We constructed a county-level longitudinal data set of more than 1,400 public health orders issues by California and its 58 counties pertaining to its 40 million residents. First, we describe methods used to construct the dataset that enables the characterization of the evolution over time of California state- and country-level public health orders dealing with COVID-19 from January 1, 2020 through September 30, 2020. Public health orders are both an interesting and important outcome in their own right and also a key input into analyses looking at how such orders may impact COVID-19 epidemics. To construct the dataset, we developed and executed a search strategy to identify COVID-19 public health orders over this time period for all relevant jurisdictions. We characterized each identified public health order in terms of the timing of when it was announced, went into effect and (potentially) expired. We also adapted an existing schema to describe the topic(s) each public health order dealt with and the level of stricture each imposed, applying it to all identified orders. Finally, as an initial assessment, we examined the patterns of public health orders within and across counties, focusing on the timing of orders, the rate of increase and decrease in stricture, and on variation and convergence of orders within regions.
Public Availability
We are making the dataset publicly available as part of our commitment to open data and combating the COVID-19 pandemic (http://sc-cosmo.org). We intend to update the dataset at least monthly. We are hopeful that others will use our methodological approach to characterize county-level or even city-level public health orders over time across additional states and would be happy to discuss this further via email (jeremygf{at}stanford.edu). A description of the contents and linkages of released files are provided in Appendix Table 1.
Number of orders by month
Public Health Order Searches
We developed and executed a search strategy to identify COVID-19 public health orders over the analytic time period applied to all California counties and the state itself. We identified website(s) for each county on which public health orders were released and supplemented this with web searches for press releases and media that combined county name, terms relating to COVID-19, and terms relating to public health orders and the topics they dealt with [1-59] (see also Appendix Tables 2 and 3). Search terms used in Google included combinations of each county’s name and any of the following: declares emergency; bans gatherings; closes March; reopens April; reopens May; reopens June; Stage 2; Stage 3; recloses July; face coverings.
Number of orders by topic and percent of orders by topic
Number of orders by county by month
We recorded URLs and the date that each document was accessed, also archiving documents whenever possible. As some counties maintain only their most recent public health order on their websites (e.g., Solano County), we used the Wayback Machine to examine the orders contained on each such website at all points in time at which orders were added or changed [60]. In addition to gathering county public health orders, we also searched for and compiled the timing and topics of state-level public health guidance that applied to some or all of counties (e.g., all counties in the state, counties on a watch list, etc.) as this sometimes, explicitly or implicitly, substituted for or superseded county orders. Finally, when we encountered them, we separately noted instances where other county ordinances and orders may have related to COVID-19 but were not explicitly public health orders; we used this last category of information for narrative purposes only, excluding it from our dataset and analyses described below as we could not be sure that our search strategy would systematically identifying them in an equivalent way across time and counties.
Characteristic Extraction
All public health orders and guidance had the following characteristics: 1) county(s) to which they pertained; 2) date of announcement; 3) date of effect; and 4) date of expiry1 (which could be either an explicit date or ongoing).
Classification of Public Health Order Topics
We began the schema for order classification using topics found in the COVID-19 US State Policy Database [61]. To these, we added 3 topics to reflect more thoroughly the actions found in the California county public health orders: moratorium on gatherings; quarantine for residents; and lab reporting & contact tracing. The complete list of topics and their general definition are provided in Appendix Table 4. Based on full textual review by a study investigator, we classified each identified public health order as to whether it pertained to each of the 14 topics (e.g., face masks, gathering size, etc.). In the event of ambiguity of whether an order dealt with a given topic, a second study investigator reviewed the text and discussion to reach consensus was used to reach a final determination. A composite category of current level of county closure was added to the list which combined initial closures, reopenings, and reclosures (see below for further detail).
Classification of Stricture on Each Public Health Order Topic
The primary focus of the classification of stricture was to gauge how public health orders relating to specific topics became more or less strict within each county over time. For example, instead of attempting to rank whether closure of indoor dining in a given county was more or less strict than closure of nail salons in another county, we focused on how the set of business types that were closed in a given country widened or narrowed over time. For certain topic areas (e.g., state of emergency), stricture was binary – whether an order about it existed at a given point in time or not. For others, stricture had ordinality (e.g., the increasing number of people allowed at gatherings or events). In addition to applying a within-county numeric scale, specific free text information was extracted to capture the details of how the order’s stricture on a given topic was determined. Lists of scales and their definitions are provided in Appendix Table 5. Based upon the scales we used, a study investigator classified the strictness of each order for each category it pertained to. Again, in the event of ambiguity, a second study investigator reviewed and discussed the particular example until consensus was reached. Since public health orders dealt with closures, reopenings, and reclosures, we constructed a combined variable on current strictness of closure/openness (which essentially added closure, reopening, and reclosure with a floor of 0 for completely open to account for differences in the ordinal scales we used). The main determinant of this variable’s value became the state-level tiered system upon the system’s rollout on August 31st with the exception that counties could opt to remain more strictly closed than the state system required [62].
Descriptive Analysis of Public Health Orders and Their Topics and Stricture
We identified and characterized 1,414 state- and county-level public health orders related to COVID-19 that were issued between January 1, 2020 and September 29, 2020. Of those, 1,276 dealt with the key topics of physical distancing, stay at home, quarantine instructions for residents, quarantine instructions for visitors, reopening, reclosing, lab and contact tracing, and mask restrictions.
Counts of orders
For the date ranges reviewed, we determined the number of COVID-19 public health orders promulgated. The months with the largest numbers of orders corresponded to rises in detected case rates.
Counts of topics covered by orders
For the date ranges reviewed, we determined the number of order-topics (i.e., each time any COVID-19 public health order dealt with a given topic). Like counts of orders, we treated the topics covered by state guidance and orders in two ways: including them as a county orders when relevant or excluding them. We also calculated the percent of each specific order type as a function of total orders. Since some orders dealt with multiple topics in a single order, the percentages sum to greater than one. A large fraction of orders dealt with which venues would be open or closed and/or the levels of their closure/openness.
Counts of orders by county
For the date ranges reviewed, we determined the number of COVID-19 public health orders promulgated by each county and examined the distribution of these counts across counties. Since state guidance and orders could substitute for or supersede county orders, we generated county-level counts and other descriptive statistics in two ways, including relevant state orders as a county order when it applied and excluding state orders.
Daily rates of public health order/topic stricture by county
We employed graphical analysis to characterize the timing and relative degree of stricture of each county’s orders related to each of the 14 topics. Additionally, we employed graphical analysis to characterize the relative degree of stricture within California regions. One particularly interesting feature that varies across counties is the use of face masking orders and the use of closure orders in the late spring and summer. In many counties, closure order strictness increased prior to masking order strictness. However, once masking order strictness increased, some counties appear to have moved to re-open while others maintained or increased closure strictness. The use of different order types as substitutes for one another or as complements is an important area for further research.
Discussion and Conclusions
Without vaccines, non-pharmaceutical interventions have been the most widely used approach to controlling the spread of COVID-19 epidemics. Various jurisdictions have implemented public health orders as a means of reducing effective contacts and controlling their local epidemics. While multiple studies have examined the effectiveness of various orders (e.g. use of face masks) for epidemic control [63-69], a complete picture of the magnitude and mechanisms of effectiveness of orders and regulation will likely require geographical and temporal variation in the contents of orders and their strictness. Current data sets tend to report on orders and regulations on a national and/or state level with current county-level efforts not necessarily standardized in their review and curation nor in their characterization of the topics and level of strictness contained in each order [70-72].
We constructed a county-level longitudinal data set of public health orders issues by California and its 58 counties pertaining to its 40 million residents. We observe both spatial and temporal patterns in the timing and strictness of orders on a variety of topics across counties and over time. We have made the dataset publicly available as part of our commitment to open data and combating the COVID-19 pandemic (http://sc-cosmo.org).
There are many potential uses of the data set we have constructed and released. It can be used in a variety of quasi-experimental designs to estimate the effects of such orders on epidemic health outcomes and on non-health outcomes (e.g., unemployment), potentially refining or extending prior studies in this area. It can be used to examine mechanisms like behavioral responses to such orders and compliance using either surveys such as those whose results are made available by Facebook or measured behavioral responses like those made available by Google’s Mobility Trends [73-74].
The approach we have employed for California is not exclusively applicable to California, and our hope is that other researchers in other jurisdictions may build up similar data sets and make them publicly available, leveraging the methods and approach we have taken. We are eager to assist in such efforts to the extent possible.
In conclusion, our data set provides an open source and useful contribution to analyses of important issues related to the COVID-19 pandemic. Its methods may be used more broadly to characterize policy responses. Illustrating its utility, a descriptive examination of public health orders at the county-level in California that accounts for the timing of the orders, the topics the orders address, and their level of strictness highlights intriguing patterns like the use of masking orders as a complement to other types of closures in some jurisdictions, presumably attempting to further reduce COVID-19 incidence, and as a substitute to other types of closure in other jurisdictions, presumably attempt to maintain COVID-19 at low incidence while mitigating the effects of health orders on the local economy. Further analyses of such phenomena are highly relevant and urgently needed as the pandemic continues.
Data Availability
We are making the complete dataset publicly available as part of our commitment to open data and combating the COVID-19 pandemic. We intend to update the dataset at least monthly as we continue our ongoing review and extraction of public health orders.
Appendix
State- and County-Level COVID-19 Public Health Orders in California: Constructing a Dataset and Describing Their Timing, Content, and Stricture
Details of released data files
Websites accessed for searches of public health orders (main governmental sites)
Additional websites accessed for county and state public health orders
List of topics covered with definition
Descriptions of value definitions for each ordinal public health order category
Footnotes
↵1 Expiry dates were mainly used by counties in the beginning of the COVID-19 epidemic and were typically overwritten by additional orders before their expiration.
References
