Abstract
We study the impact of college reopening in Fall 2020 on county-level COVID-19 cases and deaths using the information of 1,076 U.S. colleges. We match college and county characteristics using several methods and calculate the average treatment effects of three teaching modalities: in-person, online, and hybrid on COVID-19 outcomes up to two months after college reopening. In pairwise comparison, colleges reopened with in-person teaching mode were found to have about 36% point more cases within 15 days of reopening, compared to those reopened online, and the gap widens over time at a decreasing rate. Death rates follow the pattern with a time lag. However, colleges with hybrid mode catch the pattern of in-person mode after some time. We also find that greater endowment and student population, and fewer republican votes in the county are major predictors of choosing remote teaching modes over in-person.
JEL codes A23, I18, I23.
1 Introduction
The COVID-19 pandemic and its containment measures have unprecedentedly affected human health and economic activities (e.g., McKibbin and Fernando, 2020; Atkeson, 2020). Countries across the world have implemented partial or full business and school closures to mitigate the infection spread (Panovska-Griffiths et al., 2020; Di Domenico et al., 2020). Many U.S. colleges temporarily closed or switched to online in the spring semester, and over six out of ten colleges reopened in the fall semester with an in-person or a combination of in-person and online teaching plans (Gallagher and Palmer, 2020; Picault, 2021). College students mainly fall in the age cohort of 18 to 29 years, which has a lower death rate (0.4%) from COVID-19 (Wrighton and Lawrence, 2020), but a greater chance of socialization than the older age cohorts. Although colleges have taken different measures–including symptom screening, contact tracing, cleaning and disinfecting, and mandatory mask policies–there is a chance that reopening colleges may spread the virus among faculty and staff serving students, and in the region around the college (Hubler and Hartocollis, 2020).
Given the substantial risk of infection spread from college campuses to the community, a timely policy question is, if and to what extent, switching to a more remote teaching modality help in controlling communicable diseases. Although the severity of the pandemic somewhat reduced by the end of 2021, a study that analyzes the effect of different teaching modalities may offer directions in similar crises in the future—such as an expansion of a new variant of COVID-19. The current study applies a quasi-experimental approach to find the impacts of teaching modality on infection spread onto the neighboring areas. Intuitively, we match college and county characteristics to find a pair of similar colleges, but one chose in-person and the other online. Then we measure how choosing an in-person mode might have affected COVID-19 cases and deaths in the neighborhood as opposed to choosing an online mode. We perform similar exercises for two other pairs: in-person versus hybrid, and hybrid versus online. Results provide evidence that college reopening, especially with in-person teaching mode, increases COVID-19 cases and deaths over time.
Most studies so far have looked into the impact of elementary and secondary school closure, optimal reopening plan, and teaching strategy (e.g., Di Domenico et al., 2020; Panovska-Griffiths et al., 2020; Gandolfi, 2020; Cohen et al., 2020). Walke, Honein, and Redfield (2020) discuss the possible issues of U.S. college reopening and the prevention measures required on campuses. In a study close to ours, Andersen et al. (2020) use an event study under difference-in-difference setting to investigate college openings’ association with cellphone-tracked human mobility and COVID-19 cases in the counties of the campuses. Their preliminary results suggest that reopening increased county COVID-19 cases by 2.7 per 100 people, of which 0.042 per 100 thousand people resulted in death, and in-person teaching created greater visits and higher cases during the reopening. However, most of these studies do not consider the fact that the choice of reopening and teaching modality is endogenous to observed cases. We use different matching methods to re-weight the sample based on the college’s probability of selecting an instructional mode.
2 Method
We are interested in finding the effects of teaching modality on two COVID-19 outcomes: (1) new daily COVID-19 cases reported in the county, and (2) new COVID-19 related deaths in the county. The impact of teaching modality is assessed on county-level outcomes (and not on the college level) because students may spread the virus to non-students and immunocompromised people in the community, which is a critical policy concern. First, we specify a simple relationship between college reopening in the fall semester and the outcome as below, where, Yi is a COVID-19 outcome, i = 1, …, N indicate colleges and Ti is a binary treatment variable that represents teaching modality. We use three specifications of Ti separately: (1) in-person=1, online=0, (2) in-person=1, hybrid=0, and (3) hybrid=1, online=0 for pairwise comparison of teaching modalities. The vector of control variables includes observed college and county level covariates such as college endowments or percent of the population who stayed at home, and β is a vector of respective parameters. Our parameter of interest is α that shows the impact of a mode of instruction on an outcome variable, e.g., whether switching from online to in-person has an impact on the COVID-19 outcomes. The equation above would have been identified if colleges were homogeneous and were randomly assigned a teaching mode (Barnow, Cain, and Goldberger, 1981). In practice, the treatment assignment is not random, because colleges choose the mode of teaching based on their distance education capacities, observed COVID-19 cases, and many other characteristics. The probability of adopting a teaching modality should considerably vary across colleges. Intuitively, if we could select two colleges with identical institutional and regional features, except one went in-person and the other online, we could measure the difference in COVID outcomes induced by the teaching modality.
There are some other estimation issues. First, many areas had high COVID-19 cases in Spring and Summer semesters. Due to the exponential nature of the disease spread, places with more initial cases will experience and faster growth. Moreover, teaching modality in the fall semester may depend on existing cases, thus generating a reverse causality. Second, there can be observed and unobserved college and county features that affect both teaching modality and COVID-19 outcomes, creating an omitted variable bias. Third, a measurement error may occur because infections are reported less during the weekend and more on weekdays, and the incubation or survival period varies. Finally, a county may have multiple colleges that determine the presence of students in the region. We take several measures to address these problems. For instance, we control for a number of observed college and county characteristics, and use percentage changes of the outcome variables to eliminate college and county-level unobserved fixed effects. We also control for the presence of other colleges in a county. Attempts to address endogeneity are described below.
One source of identification is that different colleges reopened on different dates between July and October of 2020. Therefore, we can isolate the effect of reopening on COVID-19 from the aggregate trend. We also implement various matching methods described below. To deal with the measurement and threshold selection issue, we use two-week intervals based on recommended quarantine period: 0-15 days after reopening, 15-30 days after, 30-45 days after, and 45-60 days after reopening, and run separate cross-section regressions for each subsample. The benchmark of percentage calculation is 15-0 days before reopening; for example, and so on. We chose two weeks intervals because doing so gives COVID-19 enough time to appear in the outcomes, and we did not go beyond 60 days because the effects may be contaminated by other exogenous factors.
We employ five methods and check if the results are robust: (1) bivariate Ordinary Least Squares (OLS) where β = 0, (2) multivariate OLS where β ≠ 0, (3) propensity score matching (PSM), (4) Nearest Neighbor matching (NN), (5) and Kernel Multivariate Distance Matching (KMDM). The first two methods are conventional. The latter three methods focus on finding “statistical twins” for each observation in the treatment group from the control group with similar values of the covariates. The average treatment effects can be calculated as the mean of differences between the observed values in the treatment group and the imputed counterfactual values. We present a non-technical version of the matching methods below.
Rosenbaum and Rubin (1983, 1984) discuss propensity score matching (PSM) to adjust the probabilities for the differences in pre-treatment variables. A propensity score is the conditional probability of receiving the treatment given the pretreatment variables, i.e., P(Ti = 1| Xi). The probabilities of treatments generated are used to create weights that adjust the pre-treatment imbalances. For example, if a college has a high probability of choosing online than in-person, greater weights are assigned on in-person than online during the estimation. PSM is usually calculated using a logistic regression (for a recent technical discussion, see Imbens, 2000; McCaffrey et al., 2013).
Although finding a match with PSM is relatively easier, many studies argue that matching with propensity scores can be misleading as it ignores the multidimensional differences between two observations and simplifies them into one dimension—the score (e.g., King and Nielsen, 2019). The nearest neighbor (NN) matching adopts a more multidimensional approach and uses some (default being one) closest observations in the control group. In case of ties, NN uses all ties as matches. Conversely, a single control observation can be used many times with replacement. Matching with more than one continuous covariate may induce bias, so we apply a bias-adjusted NN matching proposed by Abadie and Imbens (2011).
Our final matching technique, MDM, is similar to NN, but uses a certain bandwidth for multivariate matching instead of comparing with the closest neighbor. Simulation studies show that MDM tends to outperform PSM because it approximates fully blocked randomization that is relatively more efficient (King and Nielsen, 2019). We apply KMDM where observations in treatment and control groups are matched using Mahalanobis distance, and use the Epanechnikov kernel function to assign larger weights to controls with smaller distances (see Jann (2017) for further discussion on the advantages of MDM). Following Huber et al. (2015), we choose 1.5 times the 90% quantile of the non-zero distances in finding a pair.
3 Data and descriptive statistics
We collected characteristics of four-year and two-year undergraduate degree-granting colleges and universities randomly chosen from the National Center for Education Statistics (NCES, 2020). Most of the colleges available in the data are public or private non-profit. Data on teaching modality and the official start date of the fall 2020 semester were manually collected between July and December of 2020 by authors from respective college websites1. Over 90% of the colleges mentioned their latest teaching modality between June and September 2020. Information for the remaining colleges was obtained from contemporary campus news or local news. Some examples of search phrases we used are, “XXX university fall 2020 covid information reopening plan new” or “XXX college restarts in-person class in fall 2020” or “XXX institute fall 2020 plan president announcement updated”. We categorize fall reopening plans into three types: (1) in-person, (2) online, and (3) hybrid. Colleges in the “in-person” group started Fall 2020 semester with face-to-face classes and open residence halls, and may include some online delivery materials; colleges in the “online” group primarily offered online classes with some exceptions for lab components and may have some students on campus; and colleges in the “hybrid” group either divided the class into online and in-person section, or switched the teaching mode on a rolling basis, or offered courses with both in-person and online access. County characteristics were extracted from the American Community Survey (ACS, 2020). Merging all these data sets eventually gives 1,076 colleges in all U.S. states and Washington D.C.
We obtain daily new COVID-19 cases and deaths at the county level from the New York Times (NYTimes, 2020). We also merged the data with individual mask policies in county and state from HealthData.gov (2020). Finally, information on county-level shares of republican votes in Presidential Election 2016 is included from the MIT Election Data and Science Lab (MIT, 2018) to take residents’ perception of COVID-19 risk into account (e.g., Tyson, 2020).
Figure 1 simply plots the changes in COVID-19 cases and deaths as a percent of 15 pre-semester days grouped by college teaching modality. We took the percentage changes to remove the college and county-level idiosyncratic features. Different colleges started the Fall 2020 semester on different dates from the end of July to October. Figure 1 matches the reopening dates, and suggests that COVID-19 cases were exponentially growing in the Fall semester regardless of teaching modality, and COVID-related deaths followed the pattern. However, colleges teaching online are located in counties that had slower growth of cases after the college reopening date, on average. Cases were slowly growing for colleges with hybrid teaching mode as opposed to colleges with in-person mode. After 30 days, the pattern of cases is reflected in the pattern of deaths. Counties where colleges taught in hybrid or in-person mode experienced a rise in COVID-related deaths, but counties where colleges taught online experienced a fall. However, an econometric analysis is required to find the impact of teaching modality.
Table 1 provides variable descriptions and sources, and Table 2 presents summary statistics of variables. The earliest reopening date for the Fall semester is July 20th in our sample whereas the latest date is October 24th. Our analysis also controls for the average percentage of the county population that stayed home daily, collected from the Bureau of Transportation Statistics (BTS, 2020). Omitting stay-home makes the effects of a teaching mode on COVID-19 outcomes biased, because COVID-19 outcomes may increase if fewer people stay home regardless of campus reopening or teaching modality. A variable on the existing mask ordinance was also included for the same reason. These variables are further used to find the determinants of teaching modality and matching. The matching process should only contain variables that are measured at baseline because variables measured at around the treatment may be influenced or modified by treatment (Austin, 2011). Therefore, we selected the determinants from the latest academic year available before the Fall 2020 semester. We also included total COVID-19 cases and deaths by the end of the Spring 2020 semester because it was the last regular semester that gave the college administration a signal about the disease situation in the community. Some studies find an association between temperature and COVID-19 spread (e.g., Livadiotis, 2020), so the average temperature during the reopening month was included in the model.
A required condition for estimating treatment effects is distributional similarities of pre-treatment covariates across the treatment groups McCaffrey et al. (2013). Table 2 indicates considerable variation across teaching groups. The difference requires the use of matching in our analysis. Tests for the distributional equality of matching variables between the treatment and control group are placed in the appendix.
4 Results
Tables 3 and 4 respectively show the average treatment effects of teaching modalities on COVID-19 cases and deaths. We begin with simple bivariate OLS to test the correlation, then control for other covariates, and then use three matching models. Table 3 suggests that the effect of in-person teaching on COVID-19 cases, as opposed to online, is positive and significant for all time intervals. For example, the bivariate model shows that cases increase by 36.78% point in the first 15 days after reopening, and keep increasing at a decreasing rate in later periods. The hybrid teaching mode also has a positive significant difference from online. The gap between the effects of in-person and online stays when we control for other covariates, but becomes statistically insignificant between hybrid and online. Matching with propensity scores does not indicate a significant advantage of any teaching modality in later periods. A significant effect in later periods is important from a policy perspective because the difference of cases in the first 15 days can be influenced by more tests before reopening with in-person teaching mode. Matching with NN somewhat corroborates the result from the simple OLS. A more robust method like KMDM suggests a statistical difference in COVID-19 cases between in-person and online groups, and to some extent, hybrid and online groups. The gap between in-person and hybrid remains statistically insignificant. To summarize, colleges that chose in-person over online experienced a greater increase in COVID-19 cases in their respective counties. A hybrid instructional mode may create a difference from in-person at the beginning, but may not sustain for longer periods.
Table 4 presents similar results for COVID-19 deaths. MDM captures a statistically significant difference between in-person and online modes, which also grows over time at a decreasing rate. For example, teaching in-person instead of online would result in 33% point more deaths in 30-45 days after reopening compared to the pre-semester 15 days. Other results are not significant at the 1% level, but it is interesting that the gap between hybrid and online groups closes in the 45-60 days interval, while the in-person group remains statistically different from them. As seen earlier, no difference in deaths was found for the hybrid versus online model.
Notice that the magnitude of the average treatment effects increases at a decreasing rate. This occurs because college reopening increases average cases and deaths regardless of teaching modality. Although online mode raises the rates slower than in-person, the gap starts to shrink after 45 days, possibly because college students are likely to socialize on campus outside the classroom even if the classes are online.
The dataset further allows us to explore another aspect of teaching modality during a pandemic. We use logistic regressions to find the predictors of a teaching modality. The results in Table 5 show that colleges with greater enrollment are less likely to choose in-person than online (column 1), and in-person than hybrid (column 3), holding other variables constant. A percent increase in endowment per student decreases the log odds ratio of choosing in-person over online by 0.28, and that of choosing hybrid over online by 0.26, given the other predictors. Public colleges are less likely to choose an in-person mode as opposed to online, but the coefficient is significant only at the 10% level. A greater population in the county might have made colleges choose some remote teaching components. Prevailing mask ordinance has a negative association with going in-person, and more republican votes have a positive association with choosing in-person over hybrid or online. A negative association of in-person mode with deaths in the Spring 2020 semester is noticeable, but not statistically significant. However, there is some positive association between the percentage of the Black population in the county and choosing an in-person mode over hybrid. The variable representing the percentage of people who stayed home is not statistically significant in these results, but it allows us to control for the mobility in the area along with controlling for the total population. To summarize, greater enrollment, endowment per student, county population, fewer republican supporters in the county, and prevailing individual mask ordinance are major predictors of adopting online or hybrid teaching modality over in-person.
5 Concluding remarks and implications
Our findings have several implications for higher education and health policy. First, results from logistic regression suggest that colleges with smaller endowments per student are less likely to choose online over in-person teaching mode, after controlling for other factors. Moreover, the average treatment effects show adopting an in-person mode instead of an online mode may increase COVID-19 cases. Therefore, colleges with small endowments need special policy attention to combat a disease-induced crisis.
Second, the analysis also suggests that campus reopening has a positive impact on COVID-19 cases across all teaching modes, and both new cases and new deaths tend to increase for colleges teaching in person as opposed to colleges teaching online. Since the treatment effects find counterfactuals, one can point out that the risk of spreading a communicable disease can be partially mitigated with an initiative from the colleges by increasing distance education elements in classes.
Finally, counties with more republican votes in Presidential Election 2016 were less likely to choose online or hybrid modes over in-person, ceteris paribus. The finding indicates that political affiliation might play a role in the containment of COVID-19 or any other rapidly communicable diseases.
Our study offers critical insights about choosing a teaching modality amid a disease-induced pandemic. Future research can look into the short-run and long-run effects of different teaching modalities on student learning outcomes.
Data Availability
All data used are publicly available and do not include any individual level information.
Acknowledgments
Authors are grateful for comments and suggestions from the participants of the American Economic Association and Southern Economic Association meetings. There is no conflict of interest. No funding was received for this project.
Appendix
Footnotes
A revised version as of December 2021.
↵1 A list of colleges and their webpages are publicly available in school_info.txt on https://github.com.