Using Primary Care Text Data and Natural Language Processing to Monitor COVID-19 in Toronto, Canada

1 Introduction

The coronavirus disease 2019 (COVID-19) pandemic is caused by the viral respiratory pathogen severe acute respiratory syndrome coronavirus-2 (SARS-CoV2). Globally it is estimated that over 500 million SARS-CoV2 infections have occurred and over that 6.2 million people have died because of COVID-19 [World Health Organization: URL accessed April 27, 2022 https://covid19.who.int/]. In many countries COVID-19 remains a persistent threat to human health/well-being, with the capability of inducing continued human, social, and economic loss.

Currently, a confluence of viral, socio-behavioural, and economic factors create a landscape which potentially exacerbates viral transmission and places populations at continued risk of COVID-19 infection. Outside of human control, there exists considerable uncertainty regarding the natural evolution of the SARS-CoV2 virus, and novel variants of concern continue to emerge [Darby and Hiscox, 2021, Lauring and Malani, 2021, Walensky et al., 2021]. When dealing with novel COVID-19 variants of concern, the science regarding transmission characteristics, protection conferred from available vaccines or past infection, and clinical presentation/severity resulting from infection are necessarily incomplete/evolving. From a sociological perspective, vaccine hesitancy, misinformation regarding risks/benefits of vaccines, and the “anti-vax” movement threaten efforts to achieve herd immunity against COVID-19 [Callaghan et al., 2021, Forman et al., 2021, Machingaidze and Wiysonge, 2021]. Further, global inequities create a situation where many nations lack the economic resources necessary to procure and distribute vaccines for their populations [Padma et al., 2021]. As the pandemic extends, there is a risk that continued public adherence with non-pharmaceutical interventions aimed at mitigating viral transmission will wane (e.g. mask wearing, social distancing, and “lockdowns”) [Crane et al., 2021, Doogan et al., 2020]. As a result of these aforementioned factors, and a multitude of others, in many regions across the globe COVID-19 may remain a continued threat to community health.

Working from a hypothesis that we will continue to live with the threat of COVID-19 for the foreseeable future, it becomes paramount that governments and public health authorities develop technologies to rapidly identify and respond to viral re-emergence in their local communities. Tools should assist decision makers determine when/how to promote and re-implement non-pharmaceutical interventions, when to increase procurement/distribution of vaccine inventories and other COVID-19 anti-viral medications, and when/how to expand hospital/ED/ICU capacity. Throughout the pandemic public health researchers have sought novel scientific technologies/data-sources which may act as leading indicators of viral activity, including: cell phone mobility data [Buckee et al., 2020], search history and social media posts [Lampos et al., 2021], data from contact tracing application [Budd et al., 2020], environmental data sources (e.g. COVID-19 RNA from wastewater sources) [Peccia et al., 2020], and health system electronic medical record data [Chapman et al., 2020, Liu et al., 2021].

In this study we seek to investigate whether expressive clinical narrative data, passively captured from primary care patient electronic medical records, can be used as an indicator of COVID-19 viral activity in Toronto, Canada. We apply a rule-based COVID-19 natural language processing (NLP) system to infer the COVID-19 positivity status of primary care text records. Preliminary study objectives relate to the identification of COVID-19 entities in clinical text streams, assessment of COVID-19 positivity status at a document/patient level, and exploration of the agreement in COVID-19 positivity status across mined primary care text streams. The primary objective of the study involves the construction of a primary care NLP-derived COVID-19 indicator series, and assessment of its correlation with other available COVID-19 series (obtained from Toronto Public Health), including: 1) lab confirmed COVID-19 cases, 2) COVID-19 hospitalizations, 3) COVID-19 ICU admissions, and 4) COVID-19 intubations.

2 Methods

3 Results

3.4 A COVID-19 Primary Care NLP-Derived Time Series Analysis and External Validation

To investigate the temporal dynamics of COVID-19 in primary care electronic medical records over the study period we discretize time into 53 weekly bins and sum the number of COVID-19 positive indications/utterances in each week (from either the lab text, health condition diagnosis text or clinical notes). We visualize the resulting primary care NLP-derived series using several time series plots. We further compare the primary care NLP-derived series against other externally generated COVID-19 time series collected/curated by Toronto Public Health. We descriptively assess the correlation between our NLP-derived primary care COVID-19 series, and four public health series: 1) lab confirmed COVID-19 cases, 2) COVID-19 hospitalizations, 3) COVID-19 ICU admissions, and 4) COVID-19 intubations. While the primary care derived series appear strongly correlated with each of the Toronto Public Health COVID-19 series (using naïve Pearson correlation); in many cases, the series track each other strongly, with one series leading/lagging the other during different time periods of the 2020 pandemic year.

• Download figure
• Open in new tab

Figure 1:

Time series plot of the composite count of the number of COVID-19 positive utterances from each of the three electronic medical records text streams (lab text, health condition diagnosis text and clinical notes) versus the number of lab confirmed COVID-19 cases in Toronto, Canada (data extracted from Toronto Public Health).

• Download figure
• Open in new tab

Figure 2:

Time series plot of the composite count of the number of COVID-19 positive utterances from each of the three electronic medical records text streams (lab text, health condition diagnosis text and clinical notes) versus the number of COVID-19 related hospital admissions in Toronto, Canada (data extracted from Toronto Public Health).

• Download figure
• Open in new tab

Figure 3:

Time series plot of the composite count of the number of COVID-19 positive utterances from each of the three electronic medical records text streams (lab text, health condition diagnosis text and clinical notes) versus the number of COVID-19 related ICU admissions (data extracted from Toronto Public Health).

• Download figure
• Open in new tab

Figure 4:

Time series plot of the composite count of the number of COVID-19 positive utterances from each of the three electronic medical records text streams (lab text, health condition diagnosis text and clinical notes) versus the number of COVID-19 related intubations in Toronto, Canada (data extracted from Toronto Public Health).

4 Discussion

In this study we applied a rule-based NLP system to identify COVID-19 positive utterances over millions of clinical text documents, gathered from hundreds of thousands of primary care patient records. The NLP system was originally developed by the United States Veteran’s Affairs heath system, and we transported the technology to primary care electronic medical records from Toronto, Canada. The rule-based NLP system identified hundreds of distinct COVID-19 entities across three diverse clinical text streams: lab text, health condition diagnosis text and clinical notes. Thousands of primary care patients were identified using the NLP system as having at least one positive COVID-19 document classification over our study timeframe (i.e. January 1, 2020 through December 31, 2020).

The study findings corroborate the important role that primary care physicians have played during the pandemic. Buried within the primary care electronic medical record, there exist thousands of instances where COVID-19 related entities are mentioned in the context of medical communication, disease prevention, disease management and care-planning. A key finding relates to the diversity of COVID-19 entities discovered and the heterogeneity in which these entities are recorded in modern electronic medical record systems. Study findings suggest that COVID-19 utterances are being recorded in numerous areas of the primary care electronic medical record system where clinical narrative data are captured, including: lab text, health condition diagnosis text and clinical notes. Further, there does not appear to be strong agreement between patient-level COVID-19 positivity status across these clinical text data sources. This finding suggests that COVID-19 related information is not captured/recorded in a single primary care electronic medical record field, and that studies employing primary care electronic medical records to identify COVID-19 positivity status would be wise to mine all available data sources/streams available, as reliance on a single source is likely to miss important COVID-19 utterances mentioned elsewhere in the patient medical record.

A primary objective of this study was to investigate the temporal dynamics of inferred COVID-19 positivity status from primary care electronic medical records. The estimated COVID-19 primary care NLP-derived time series exhibit temporal dynamics which have strong face validity over the study period. Early in 2020 (prior to the WHO declaring COVID-19 a global pandemic) COVID-19 lab confirmed infections and hospitalizations were low (near zero) in Toronto – this is corroborated in our analyses. In March/April 2020 we observed a sharp increase in COVID-19 positive documents identified in the primary care electronic medical record – the timing being consistent with the first wave of COVID-19 infection in Toronto, Canada. COVID-19 positive document counts displayed a low/minima in the summer of 2020 (as Toronto moved out of pandemic wave-01, following weeks of governmental imposed lockdowns). As lockdown measures were relaxed and public adherence to non-pharmaceutical interventions waned through the fall of 2020, we observed a sharp increase in the number of COVID-19 positive documents/patients recorded in the latter part of 2020 (which is consistent with the commencement of the wave-02 COVID-19 infection surge in Toronto). We plotted our COVID-19 NLP derived series against other important COVID-19 indicator series independently generated by Toronto Public Health using public-health/hospital data sources, including: 1) lab confirmed COVID-19 cases, 2) COVID-19 hospitalizations, 3) COVID-19 ICU admissions, and 4) COVID-19 intubations. On external validation, our primary care NLP-derived COVID-19 series correlated strongly with other important public health series. Strong empirical correlations are encouraging, and future work should continue to investigate whether primary care electronic medical record data can be used in the quest to uncover leading indicators of COVID-19 infection/hospitalization – as this would greatly assist public health officials and hospital executives attempting to plan for future waves of COVID-19 infection.

A unique aspect of our study is the use of free text primary care electronic medical record data to investigate COVID-19 positivity status, and temporal dynamics of COVID-19 over the initial pandemic year. Few other studies/jurisdictions have utilized primary care electronic medical record data for COVID-19 phenotyping or for the identification of leading indicators of COVID-19 infection/hospitalization. The work by Chapman et al. [2020] provides a working tool for performing COVID-19 biosurveillance using hospital electronic medical record data. Liu et al. [2021] developed a COVID-19 “hot-spot” identification algorithm based on the presence/absence of 10 clinical indicators, in the context of a large integrated health system in California, USA. The authors evaluated their algorithm using similar design/methods as our study and found their method to be a strong leading indicator of COVID-19 cases/hospital-admissions. The ability to reconstruct their bespoke algorithm in our primary care setting would be challenging and would require a combination of remapping clinical disease codes to our nomenclature and the use of NLP named entity recognition tools to extract clinical indicators not collected using our standardized nomenclatures. Many other studies have investigated COVID-19 phenotyping using electronic medical record data. Certain exceptional studies include Brat et al. [2020] and Klann et al. [2021]; however, these studies were developed in the context of hospital electronic medical records, and the availability of requisite data sources for phenotype construction in primary care settings may be limited.

Our study is not without limitations. A chief limitation of our study is that we employ a rule based COVID-19 phenotype algorithm which has undergone minimal internal validation in our local setting (see Appendix A). A challenge with internal validation of COVID-19 phenotyping algorithms relates to sampling design. As illustrated in Appendix A, evaluation of algorithm operating characteristics based on a random sample of clinical notes results in the identification of only a small number of true positive COVID-19 documents (and similarly a small number of algorithm predicted positive documents); hence, internal estimates of sensitivity and positive predictive value are imprecise. Estimation of algorithm operating characteristics from large samples are associated with increased costs (financial costs, time costs and human-resource costs), however, are necessary to achieve precise estimates of sensitivity and positive predictive value. Alternatively, sequential or stratified sampling designs may provide more efficient estimates of algorithm operating characteristics as compared to simple random sampling designs. That said, the original authors did perform their own validation of the COVID-19 biosurveillance tool and report exceptional sensitivity (94.2%) and positive predictive value (82.4%) [Chapman et al., 2020]. Qualitatively, our small validation study suggests similarly effective performance in an entirely different study setting/context. A central objective of our study involved a rigorous external validation of the COVID-19 biosurveillance system against important public-health/hospital-based indicators of COVID-19 activity in Toronto, Canada. Our study findings have face validity, as counts of documents reporting positive utterances of COVID-19 correlate strongly against externally generated COVID-19 indicator series collected from Toronto Public Health. Another limitation of our study relates to the retrospective design and delay/lag in data reception, curation and reporting in our local jurisdiction. The utility of our findings would be enhanced if more timely access to primary care electronic medical record data existed in our jurisdiction, which would facilitate using our biosurveillance tool for near real-time monitoring/surveillance of COVID-19 in Toronto, Canada.

5 Conclusions

In this study, we applied a rule-based NLP system for COVID-19 biosurveillance to our primary care electronic medical records in Toronto, Canada. The method scaled to millions of primary care clinical text documents, and hundreds of thousands of patient records. We identified hundreds of unique COVID-19 entities, and thousands of COVID-19 positive clinical documents, across three primary care clinical text streams. The resulting primary care COVID-19 NLP-derived time series correlated strongly against other important COVID-19 indicator series, externally generated from hospital/public-health data sources. Future work should continue to investigate whether leading indicators of COVID-19 cases or hospitalizations may be uncovered using readily available, passively collected, high quality primary care electronic medical record data.