Abstract
Importance The COVID-19 pandemic has resulted in increased use of virtual care, however, few studies have looked at the association between virtual primary care visits and other healthcare use.
Objective To determine whether there was an association between a high proportion of virtual visits in primary care and more emergency department visits
Design A cross-sectional study, using routinely collected data
Setting Ontario, Canada
Participants Ontario residents alive on March 31st 2021 and family physicians with at least 1 visit claim between February and October 2021.
Exposure Family physicians stratified by the percentage of total visits that were virtual (phone or video) between February and October 2021
Main outcome(s) and measure(s) We calculated the emergency department visit rate for each stratum of family physician virtual care use. We used multivariable logistic regression models to understand the relative rate of patient emergency department use after stratifying for rurality and adjusting first for patient characteristics and then the 2019 emergency department visit rate.
Results We analyzed data for 15,155 family physicians and 12,951,063 Ontarians attached to these physicians. The mean number of emergency department visits was highest among patients whose physicians provided only in-person care (470.3 ± 1918.8 per 1,000) and was lowest among physicians who provided >80 to <100% care virtually (242.0 ± 800.3 per 1,000). After adjustment for patient characteristics patients seen by physicians with >20% of visits delivered virtually had lower rates of emergency department visits compared to patients of physicians who provided >0%-20% virtually (e.g. >80 to <100% vs >0%-20% virtual visits in Big Cities, Relative Rate (RR) 0.80 [95%CI 0.76-0.83]). This trend held across all rurality strata and after adjustment for 2019 emergency department visit rates. In urban areas, there was a gradient whereby physicians providing the highest level of virtual care had the lowest emergency department visit rates.
Conclusions and Relevance Physicians who provided a high proportion of care virtually did not have higher emergency department visits than those who provided the lowest levels of virtual care. Our findings refute hypotheses that emergency department use is being driven by family physicians providing more care virtually.
Question Do family physicians who provide more care virtually have higher emergency department visit rates among their patient panel?
Findings In this cross-sectional study from Ontario, Canada, we examined data from February to October 2021 for 12,951,063 patients attached to 15,155 family doctors and found that physicians who provided a high proportion of virtual care did not have higher emergency department visits than those who provided the lowest levels of virtual care. This finding remained true after adjusting for patient characteristics.
Meaning Our findings refute hypotheses that emergency department use is being driven by family physicians providing more care virtually.
Introduction
The COVID-19 pandemic resulted in a seismic shift in how primary care was delivered. Shortly after a global pandemic was declared in March 2020, the amount of care delivered virtually—by phone or video—increased 56-fold, comprising over 70% of all primary care visits.1 More than two years into the pandemic, virtual care continues to comprise a substantial portion of visits.2 Research has found that most patients feel comfortable with virtual care, like its convenience, and want it to continue.3-6 However, there are concerns about the impact of virtual primary care visits on care quality including on patient safety,7 equity in access,8,9 effective chronic condition management,10 and healthcare utilization.11 Effects have been hard to untangle due to the simultaneous impact of the pandemic on care.
The proportion of primary care delivered virtually varies by jurisdiction and practice and is influenced by factors such as reimbursement and overhead, patient and provider access to technology and infrastructure, provider access to personal protective equipment, provider health concerns, and patient and provider preference.12-17 There have been concerns that in some practices, the proportion of virtual visits is too high resulting in an increase in other health system use. However, few studies have looked at the association between virtual primary care visits and other healthcare use. One such study found high practice telehealth use was associated with a small increase in emergency room visits and hospitalizations for ambulatory care sensitive conditions compared with low practice telehealth use.11
In Ontario, Canada, provincial authorities had initially directed family physicians to adopt a virtual-first approach to seeing patients as a safety precaution when COVID-19 case counts were high. This approach meant physicians were to perform the initial assessment remotely and then bring patients into the office for an in-person visit only if warranted. However, by fall 2021, authorities were concerned that many family physicians were doing too much virtual care and that this was making it difficult for patients to access in-person care, resulting in increased emergency department use. In October 2021, the provincial medical officer of health and the regulatory college issued a joint statement encouraging more use of in-person care.18 It is unclear, however, whether high use of virtual care was driving perceived increases in emergency department visit volumes.
We undertook a cross-sectional study using routinely collected data to understand whether there was an association, ecologically and at a practice-level, between a high proportion of virtual visits in primary care and more emergency department visits.
Methods
Context and Setting
Ontario is Canada’s largest province with a population of 14.8 million in 2021.19 Physician and hospital visits, including emergency department care, are fully insured and free at the point-of-care for all permanent residents through the Ontario Health Insurance Plan (OHIP). Just over 80% of the population is formally enrolled to a family physician practicing in a Patient Enrolment Model where, depending on the model of care, between 15% to 70% of payment is via capitation (adjusted for patient age and sex) with some fee-for-service and incentive payments.20 The remaining population either are unattached or see a family physician practicing fee-for-service. Approximately one quarter of the population lives in rural communities with fewer than 30,000 people, where health services are structured differently.21 Rural emergency departments are staffed by family physicians and although emergency department volumes are relatively low, visit rates are much higher than in urban areas, due in part to the limited availability of other after-hours care.22 Temporary virtual care billing codes were introduced for phone and video visits in Ontario on March 14, 2020 and, during the study time period, paid the same amount as an equivalent in-person visit. Initially, the same billing code was used for phone and video.
Study Design
We examined primary care and emergency department visit trends between January 2019 and October 2021. We then conducted a cross-sectional, population-based analysis using linked health administrative data to examine the association between the proportion of care delivered virtually at the primary care physician-level and emergency department visit rates between February and October 2021 in Ontario, Canada.
Datasets were linked using unique encoded identifiers and analyzed at ICES. ICES is an independent, non-profit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyze health care and demographic data, without consent, for health system evaluation and improvement.
Study Population
We examined visit trends for all residents in Ontario. For the cross-sectional analysis, we included family physicians with at least 1 home, office, or virtual visit claim between February and October 2021. We included permanent residents of Ontario who were alive as of March 31st 2021. We assigned Ontario residents to a family physician using enrolment tables from the Ontario Ministry of Health; for those not enrolled, we assigned them to the family physician who billed the maximum value of 30 common primary care fee codes billed within the previous 2 years (eExhibit 1). We excluded residents who we could not assign to a family physician or whose family physician did not have any visit claims during the study period.
We categorized physicians as practicing traditional fee-for-service or in one of the following three types of Patient Enrolment Models: Enhanced fee-for-service (Comprehensive Care Model, Family Health Group) where fee-for-service comprises approximately 80% of income; Non-team capitation (Family Health Network, Family Health Organization) where capitation comprises approximately 70% of income; and Team-based capitation (Family Health Team) where capitation comprises approximately 70% of income and physicians also have access to an interprofessional team funded through the Ministry of Health.
Analysis
Trends in primary care and emergency department visits
We used physician billing claims to count primary care visits weekly by type: office, home, virtual and total. We used the National Ambulatory Care Reporting System to assess emergency department visits weekly (total and stratified by triage level). For context, we included data on COVID-19 case counts in Ontario using publicly available COVID-19 surveillance data from Public Health Ontario.23
Cross-sectional description of physicians by virtual care use
For each family physician, we calculated the total volume of visits that were in-person (office and home) and virtual (phone and video) between February and October 2021. We then calculated the percent of all visits that were delivered virtually during that period and stratified physicians into the following groups a priori: 0% virtual (100% in-person), >0-20%, >20-40%, >40-60%, >60-80%, >80-<100%, and 100% virtual.
We described patient and physician characteristics as of March 31st, 2021 within each stratum of virtual care use. We assessed physician characteristics using the provider databases. We assessed patient age, sex, and postal code using the provincial registry of all individuals eligible for OHIP coverage. We used postal code and 2016 Canadian census data to derive neighbourhood-level income quintile. We assessed rurality using patient and physician postal code and the Rurality Index of Ontario (RIO) score, where areas with a score of 0 are considered big cities, 1-9 are small cities, 10-39 are small towns, and 40 or more are rural areas.24 Patient postal code and the Ontario Marginalization Index25,26 were used to assess neighbourhood-level material deprivation and ethnic diversity. We assessed whether patients registered for OHIP within the last ten years, a proxy for recent immigration. Finally, we used the Johns Hopkins Adjusted Clinical Group method to assess patient co-morbidity and morbidity (ACG® System Version 10).27 We used the Johns Hopkins ACG® System Aggregated Diagnosis Groups to categorize comorbidity as 0 (no comorbidity), 1-4 (low comorbidity), 5-9 (moderate comorbidity) and 10+ (high comorbidity). Morbidity was assessed using corresponding Resource Utilization Bands (RUBs) categorized as 0-1 (non-user/healthy user), 2 (low morbidity), 3 (moderate morbidity) and 4+ (high morbidity).28
Variation in virtual care use
For physicians in a Patient Enrolment Model, we examined the variation in virtual care use between physicians in the same practice group and between physicians in different practice groups, stratified by the type of Patient Enrolment Model. For this analysis, the percentage of total visits that were virtual was modeled as a continuous variable for each physician. To understand how much of the total variance in virtual visits was attributable to physician group and practice type, we calculated an intraclass correlation coefficient from a three-level (patient, physician and group), intercept-only mixed linear model with virtual visit rate as the outcome.
Cross sectional association between virtual care and other health care use
We examined healthcare use of patients attached to family physicians within stratum of virtual care use. The primary outcome was the rate of emergency department visits per 1,000 patients. We compared this across levels of family physician virtual care use between February and October 2021. For each stratum, we assessed the number of emergency department visits for every patient of physicians in the stratum and calculated the average number per patient. We also assessed the following contextual information: i) the percent of patients with a primary care visit, ii) the average number of primary care visits, and iii) the percentage of visits to the usual family physician between February and October 2021. Usual family physician was the physician a patient saw the most over the two years prior to March 31, 2021. Secondary outcomes included the percent of patients with an ambulatory care sensitive condition-related hospital admission, derived from the Canadian Institute for Health Information’s Discharge Abstract Database (eExhibit 2), and specialist visits, ascertained from physician billings. We examined bivariate associations overall and separately for each rurality stratum.
Next, we wanted to understand whether the associations observed in 2021 reflected historical associations that pre-dated the pandemic and widespread adoption of virtual care. To do so, we examined the absolute and relative differences in the emergency department visit rate between February and October 2021 and the corresponding period (February to October) in 2019 for each stratum of family physician virtual care use.
Finally, we aimed to understand the relationship between physician virtual care use and patient emergency department visits after adjustment for potential confounders. We constructed two multivariable logistic regression models with physician-level virtual care proportion as a 5-level categorical exposure and volume of patient emergency department visits as the outcome. In Model 1, we adjusted for patient age, sex, neighbourhood income quintile, recent registration, co-morbidity, and healthcare use, and stratified by rurality. Model 2 included the same covariates and further adjusted for the emergency department visit rate in 2019, to account for potential pre-existing patterns. All analyses were performed in SAS EG 7.1.
Sensitivity Analysis
We performed sub-group analyses where we used a previously-published algorithm29 to restrict our cohort to comprehensive primary care physicians, active as of March 31, 2019, the most recent year where the data for the algorithm was available.
Ethics Approval
The use of the data in this project is authorized under section 45 of Ontario’s Personal Health Information Protection Act (PHIPA) and does not require review by a Research Ethics Board.
Results
Primary Care and Emergency Department Visits Over Time
Total primary care visits dropped at the onset of the pandemic but returned to average pre-pandemic levels by fall of 2020 (Figure 1a). The proportion of virtual primary care visits peaked in the first two weeks of the pandemic at 82% but was 49% by October 2021. Emergency department visits dropped at the start of the pandemic and remained lower than 2019 volumes throughout the study period (Figure 1b). Trends in ED visit rates were similar for Canadian Triage and Acuity Scale (CTAS) levels 2,3, and 4 (Appendix, eExhibit 3) with dips visually corresponding to increases in COVID numbers (Figure 1c). Periods with a higher proportion of virtual primary care visits visually related to higher numbers of COVID cases and lower ED visit rates (Figures 1 c and d).
Primary Care Visits, Emergency Department Visits, and Percent of Total Primary Care Visits Delivered Virtually, January 2019 to October 2021.
(A): Weekly primary care visits, by type of visit, over time. (B): Comparison of weekly emergency department visits during the Pre-COVID-19 period (January 2019 – October 2019) and the COVID-19 period (January 2020 to October 2021). (C): Comparison of weekly emergency department visits and COVID-19 case counts between January 2020 and October 2021. (D): Comparison of weekly emergency department visits and percent of primary care visits that were virtual between January 2020 and October 2021.
Physician and Patient Characteristics
We analyzed data for 15,155 family physicians who had at least one office, home, or virtual claim between February and October 2021 and 12,951,063 Ontarians who were attached to these physicians (Table 1). The majority of physicians provided between 40-80% of care virtually; only 400 physicians provided 100% of care virtually while 2,691 provided between >80 to <100% of care virtually. A higher proportion of the physicians who provided >80% virtual care were 65 years or older, female, and practicing in big cities. Most physicians who provided 100% of care virtually worked traditional fee-for-service and had a panel size of less than 100; more of their patients were recent registrants and lived in a neighbourhood in the lowest income quintile and a neighbourhood that was more ethnically diverse (Appendix, eExhibit 4). Patient co-morbidity and morbidity were similar across strata.
Physician and Practice Characteristics by the Percent of Care Provided Virtually, between February 1 and October 31, 2021, in Ontario, Canada.
Variation in the proportion of virtual care
Among physicians practicing in a patient enrolment model, we found substantial variation in the proportion of virtual care provided both between and within groups of physicians (Figure 2). The variation was not explained by practice model (Interclass Correlation Coefficient for practice model = 0.5%); physician group accounted for almost one-third of the observed variation between all physicians (Interclass Correlation Coefficient for physician group = 30.5%).
Variation in the Percent of Virtual Visits between Physicians in the Same Group and between Physicians in Different Groups, Stratified by Type of Patient Enrolment Model, February and October 2021
The black line represents the mean ratio for the practice group. Each group can have 3 or more mos. Each dot represents a physician. Physicians within the same practice group are represented on the same vertical line. We calculated an intraclass correlation coefficient from a three-level, intercept-only mixed linear model to understand how much of the total variance in virtual visits was attributable to physician group and practice type. We found the variation was not explained by model of care (ICC: 0.5%), whereas a high proportion of variation was explained by specific practice group the physician belonged to (ICC:30.5%).
Health Service Utilization
Compared to patients whose physicians provided 40-60% of care virtually, patients whose physicians provided >80% of care virtually had a higher mean number of primary care visits per patient and lower continuity; they did not have substantially different referral rates to specialists or rates of hospital admission for ambulatory sensitive conditions (Table 2).
Patient Health Service Utilization Stratified by the Rostered Physician’s Percent of Care Provided Virtually During The Eight-Month Period Between February 1 and October 2021
The mean number of emergency department visits was highest among patients whose physicians provided only in-person care (470.3 ± 1918.8 per 1,000) and was lowest in the >80-<100% virtual care group (242.0 ± 800.3 per 1,000) (Table 2). Emergency department use decreased as physician percent of virtual care increased, except among physicians who provided 100% of care virtually where ED rates were 287.1 ± 1048.9 per 1000. Emergency department visits in 2019, prior to the COVID-19 pandemic, demonstrated a very similar pattern. Between 2019 and 2021, there was an overall 13% decrease in the mean number of emergency department visits (2019: 309.6 ± 965.4, 2021: 268.9 ± 903.2). Visits decreased across all levels of physician virtual care use, apart from a slight increase among patients attached to physicians who provided 0% virtual care; the absolute and relative decrease was smallest among patients of physicians who provided 100% virtual care followed by those whose physicians provided 80-<100% virtual care. The crude association between the proportion of care delivered virtually and ED use was consistent by rural strata (eExhibit 7).
Regression Findings
Regression modeling excluded patients of physicians providing 0 or 100% virtual care (n= 1,395 and 400, respectively). We found that after adjustment for patient characteristics, patients seen by physicians who had more than 20% visits virtually had lower rates of emergency department visits when compared to patients of physicians who provided the least percentage of virtual care (>0-20%) (Figure 3a). For example, patients who lived in big cities who provided >60-80% of care virtually had almost 20% lower rate of emergency department visits compared to those who were seen by a physician providing >0-20% of care virtually. This trend held across all region types. In both big and small cities, we observed a gradient such that patients whose corresponding physicians provided the highest level of virtual care had the lowest rates of emergency department visits. These patterns remained even after further adjusting the model for patient emergency department visit rates in 2019 (Figure 3b).
Adjusted Relative Rates of Patient Emergency Department Visits by Percent of Total Visits Delivered Virtually by Family Physicians, Stratified by Rurality. Reference = Physicians Where Virtual Care Comprises between >0 to 20% of Total Visits.
Model 1: Adjusted for patient age, sex, neighbourhood income quintile, recent registration, co-morbidity and morbidity.
Model 2: Adjusted for patient age, sex, neighbourhood income quintile, recent registration, co-morbidity, morbidity and rate of emergency department visits in 2019.
Sensitivity analysis
Overall, results were consistent in the sensitivity analysis limited to comprehensive family physicians (n=9,462) (eExhibit 5 and eExhibit 6).
Discussion
The expansion of virtual care during the pandemic opened new modes of access for patients and allowed family physicians to care for people while reducing the risk of COVID transmission. Nonetheless, some policymakers became worried that virtual care was being used inappropriately, leading to an increase in emergency department use. Our findings refute this hypothesis. First, we examined population-based trends in emergency department and primary care visits before and after the pandemic. We found emergency department visit rates were lower than in the pre-pandemic period; increases in emergency department use seemed to coincide with decreasing COVID cases and did not coincide with more virtual primary care. Second, we conducted a cross-sectional analysis of 12,951,063 million patients attached to 15,155 family doctors who practiced between February and October 2021. We found that, at the population level, physicians who provided a high proportion of virtual care did not have higher emergency department visits than those who provided the lowest levels of virtual care. This finding remained true even after adjusting for patient characteristics with differences largely following pre-pandemic patterns.
Comparison with other studies
Prior to COVID-19, several studies suggested virtual care could reduce emergency department and other hospital use, specifically for rural populations,30 older populations,31 and following a natural disaster.32 Following COVID-19, a US study examining virtual care use and hospital visits for ambulatory-care sensitive conditions (ACSC) found that practices with high virtual care use had a small increase in ACSC visits compared to those with medium virtual care use, but differences disappeared when acute and chronic ACSC were evaluated separately.11 Other studies suggest that virtual care use during the COVID-19 pandemic was higher among patients who were sicker, suggesting it supported care continuity when COVID-19 cases were high.1,33 In US studies, virtual care often includes a high portion of video unlike in Ontario where most virtual care was delivered by phone.3
Strengths and Limitations
We conducted a population-based analysis that included data on all family physicians in Canada’s largest province who were practicing in the study period. We examined emergency department use along with other health service use and looked back to the pre-pandemic period to understand if associations may have pre-dated the pandemic. Our findings were consistent when we limited our analysis to physicians meeting criteria for practicing comprehensive care. We examined virtual care use using new billing codes but unfortunately these do not distinguish between video and phone visits. Billings also do not capture other aspects of care such as email with patients or care provided by non-physician team members. We examined a nine-month period of the pandemic and findings may not reflect evolving practice patterns. Finally, our study is limited to administrative data and does not shed light on why there are differences in the amount of care delivered virtually, appropriateness of virtual care for specific circumstances, and whether that care meets patients’ needs.
Implications for Policy
The pandemic resulted in the widespread adoption of virtual care, which is here to stay. In Ontario, billing for virtual care that was introduced on an emergency basis in the pandemic will become permanent in October 2022 with new billing codes that provide lower remuneration for phone appointments compared to in-person visits. However, two years into the pandemic, there continue to be news reports of emergency departments being overwhelmed and some continue to speculate that one contributing factor is patients not being able to see their physician in-person.34 Mixed methods research is needed to better understand patients’ experience accessing their family physicians, reasons for seeking care in the emergency department, their views on virtual care, and drivers of physician-level variation in virtual care provision. Researchers and policy-makers should be mindful of different patient subgroups where virtual care can either facilitate access (e.g. in rural areas) or be a barrier (e.g. for those with language or sensory barriers).
Conclusion
We found that, at the population level, physicians who provided a high proportion of virtual care did not have higher emergency department visits than those who provided the lowest levels of virtual care. Further, during the first 18-months of the pandemic, emergency department visit rates were lower than pre-pandemic levels and periods where emergency department visit rates were highest did not coincide with higher rates of virtual care use. Our findings refute hypotheses that emergency department use is being driven by family physicians providing more care virtually.
Data Availability
The data sets from this study are held securely in coded form at ICES. Data-sharing agreements prohibit ICES from making the data sets publicly available, but access may be granted to those who meet pre-specified criteria for confidential access, available at www.ices.on.ca/DAS. The complete data set creation plan, and underlying analytic code are available from the authors upon request, understanding that the programs may rely upon coding templates or macros unique to ICES.
Appendix
Billing Codes Used to Determine the Usual Family Physician for Not Enrolled Patients.
Ambulatory Care Sensitive Condition Related Hospital Admissions Used as Secondary Outcomes to Examine Healthcare Utilization of Patients Attached to Family Physicians within Each Stratum of Virtual Care Use.
Weekly Emergency Department Visits by Canadian Triage and Acuity Scale (CTAS) Level, January 2019 to October 2021.
Patient Characteristics by the Percent of Physician Primary Care Provided Virtually, between February 1 and October 31, 2021, in Ontario, Canada.
Physician and Practice Characteristics by the Percent of Care Provided Virtually, Comprehensive Family Physicians, between February 1 and October 31, 2021, in Ontario, Canada.
Characteristics of Patients Attached to Comprehensive Family Physicians by the Percent of Physician Primary Care Provided Virtually, between February 1 and October 31, 2021, in Ontario, Canada.
Health Service Utilization of Patients Attached to Comprehensive Family Physicians Stratified by the Attached Physician’s Percent of Care Provided Virtually, between February 1 and October 2021, in Ontario Canada.
Patient Health Service Utilization Stratified by the Attached Physician’s Percent of Care Provided Virtually, between February 1 and October 2021, in Ontario Canada, by Rurality.
STROBE Statement—Checklist of items that should be included in reports of cross-sectional studies
Footnotes
Funding statement This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health (MOH) and Ministry of Long-Term Care (MLTC). This study also received funding from Ontario Health, the Canadian Institutes of Health Research funding reference number SOP 162662, and the INSPIRE Primary Health Care Research Program which is funded through the Health Systems Research Program of the Ontario Ministry of Health (MOH). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES, Ontario Health, or the Ontario MOHLTC is intended or should be inferred.
Parts of this material are based on data and information compiled and provided by Ontario Ministry of Health (MOH) and the Canadian Institute for Health Information (CIHI). The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.
Drs. Kiran and Glazier are supported as Clinician Scientists by the Department of Family and Community Medicine at the University of Toronto and at St. Michael’s Hospital. Dr. Kiran is the Fidani Chair of Improvement and Innovation in Family Medicine at the University of Toronto. Michael Green is supported by the Brian Hennen Chair in Family Medicine at Queen’s University.
Conflicts of interest The authors declare no conflicts of interest
Data sharing: The data sets from this study are held securely in coded form at ICES. Data-sharing agreements prohibit ICES from making the data sets publicly available, but access may be granted to those who meet pre-specified criteria for confidential access, available at www.ices.on.ca/DAS. The complete data set creation plan, and underlying analytic code are available from the authors upon request, understanding that the programs may rely upon coding templates or macros unique to ICES.
References
