Has the pandemic enhanced and sustained digital health-seeking behaviour? A big data interrupted time-series analysis of Google Trends
======================================================================================================================================

* Robin van Kessel
* Ilias Kyriopoulos
* Brian Li Han Wong
* Elias Mossialos

## Abstract

**Background** Due to the emergency responses early in the pandemic, the use of digital health in healthcare increased abruptly, yet it remains unclear whether this introduction was sustainable on the long term. We explore trends in digital health-seeking behaviour as proxy for readiness to adopt digital health as a mainstream form of healthcare.

**Methods** We use weekly Google Trends data from February 2019 to August 2021 in Canada, United States, United Kingdom, New Zealand, Australia, and Ireland. We used five keywords to monitor online search interests in Google Trends: *online doctor, telehealth, online health, telemedicine*, and *health app*. Data are analysed using an interrupted time-series analysis with break-points on 11 March 2020 and 20 December 2020.

**Results** Digital health searches immediately increased in all countries after the pandemic announcement. There was some variance in what keywords were used per country. However, searches declined after this immediate spike, sometimes towards pre-pandemic levels. The exception is the search volume of *health app*, which showed to either remain stable or gradually increase during the pandemic.

**Interpretation** Our findings suggest that digital health-seeking behavioural patterns associated with the pandemic are currently not sustainable. Further building of digital health capacity and development of robust digital governance and literacy frameworks remain crucial to more structurally facilitate digital health transformation across countries.

**Funding** Not applicable.

## Introduction

On 11 March 2020, the World Health Organization formally announced that COVID-19 was classified as a pandemic, which disrupted healthcare services worldwide. Evidence from a systematic review shows a median 42% decrease in healthcare visits, 28% decrease in admissions, 31% reduction in diagnostics, and 29% reduction in therapeutics, resulting in an overall 37% decrease of utilisation of healthcare services (1). Digital health usage has exploded during the early stages of the COVID-19 pandemic as traditional health services were disrupted (2–4). Digital health generally refers to the use of internet solutions, big data, and communications technologies to collect, share and manage health information to improve both individual and public health, as well as identify symptoms, plan treatment, monitor key health parameters, and monitor progress and treatment effects (5,6). Even though digital health saw tremendous uptake early in the pandemic (4), no information exists on whether this uptake was sustainable (7), which we investigate in this article.

Due to the sudden disruption of traditional health services, health ecosystems may have digitalised predominantly out of a need to survive rather than a desire to innovate, highlighted by the emergency transition to a digital paradigm and subsequent changes when the pandemic started (2). Implementing digital health solutions is complex and relies on many institutional factors, cultural and behavioural traits and health system characteristics (7,8). For instance, the design process of digital health solutions is indicative of which populations it will be able to reach and, equally importantly, what population groups will experience difficulties in accessing and using the tool (3,9). Policy environments are also vital to laying the foundation of how conducive a health system is to adopting a digital health solution and how health professionals are trained in the field of digital health (3,9,10). Finally, the readiness and willingness of digital health users are crucial elements in adopting digital health solutions (2,8). A recent analysis of digital skills in the European Union – frequently regarded as a leading region in terms of digital skills – found clear discrepancies across the European region (11). Certain population groups also seem to fare more favourable in a digital world: people who are younger, higher educated, male, live in urban regions, and are either a student or employed consistently report higher internet access and digital skills (11– 14). These findings suggest that − if digital health services were structurally introduced now − only certain population groups could fully benefit from these services. Paradoxically, population groups that have higher needs for healthcare and could potentially benefit most from these innovations are the ones that would experience the highest barriers to access (9).

With traditional health services having regained their ability to function, patients can increasingly decide whether they prefer to seek traditional or digital healthcare, which often starts with online search engines (15). Although the internet cannot substitute health professionals as a sole health information source due to a combination of the prominence of misinformation and a lack of health, digital, and science literacy (13,16–18), search engine data can be instrumental in understanding the general preference of populations in exploring the possibility of using digital health (19,20). While online searches can be information- or curiosity-driven, previous research has shown that online search behaviour is strongly correlated with the actual (healthcare) needs of the population and forms an integral part of a pathway to actual communication with healthcare providers (19–22).

This article aims to further explore the digital readiness for structurally implementing digital health solutions by investigating the health-seeking behaviour of the general population by analysing Google Trends™ data. These data showcase how popular specific search terms were in certain countries. The main advantage of Google Trends™ is that it uses the revealed and not stated users’ preferences (23), making it possible to obtain data that would be difficult to collect otherwise. The advantage of revealed preference data is that it is based on actual decisions, meaning there is no need to assume that participants will respond to simulated situations as is the case with stated preference data. As a result, revealed preference data are characterised by high reliability and face validity (24). Google Trends™ have shown to be a viable tool to understand, monitor, and even forecast information-seeking trends and public interest and is becoming an increasingly popular method for assessing population preferences in health research (19,25,26).

## Methods

Google Trends™ is the principal tool used to study trends and patterns of search engine queries using Google (19). It is an openly accessible big data tool that provides real-time and archived information on Google queries from 2004 onwards (23). Data are available in realtime, solving issues that arise with conventional, time-consuming survey methods (23). So far, it has been applied to various health-related topics such as mental health, vaccine hesitancy, and infodemic surveillance (22,27,28).

All Google Trends data points are normalised and scaled (23), meaning the number of searches for a specific term is divided by the total number of searches for all topics at a particular location and within the specified timeframe, resulting in a normalised score. All normalised scores are scaled to have a maximum score of 100 points. Scaled scores (known as relative search volumes) range from 0 to 100 points and represent search interest relative to the month with the highest search interest. Google adjusts relative search volumes for internet access and population size. We followed the established methodological guidelines for using Google Trends in infodemiology and infoveillance (23).

### Keyword, Timeframe, and Country Selection

Based on the keywords used in recent systematic reviews in the field of digital health (29– 31), we used five keywords to monitor online search interests in Google Trends: *online doctor, telehealth, online health, telemedicine*, and *health app*. While we also explored *digital health, digital therapeutics, telecare, telemonitoring*, and *virtual health*, these terms’ search volumes were negligible (< 1). As such, we did not include them in our analyses. We extracted weekly relative search volume data for Canada, the United States, the United Kingdom, New Zealand, Australia, and Ireland from 1 February 2019 to 1 August 2021 (n=780 country-weeks). The six countries were chosen because they share English as their dominant language and provide a varied representation in policy landscape regarding digital health (9,10). Google is used for 87% to 93% of the online search queries in the countries under study (32–34), meaning our data accurately captures the preferences of the population of the countries under study. As such, we can accurately measure public interest in digital health in the midst of a global pandemic (25).

### Empirical approach

To assess both short- and long-term effects of the pandemic on digital health search behaviour, we used a single-group interrupted time-series design (35,36), which is suitable for evaluating population-level effects of critical events (e.g., interventions, policy innovations, or public health crises) that have occurred at a clearly defined point in time (37).

Our design adopts a segmented regression analysis approach that allows for the identification of three effects by comparing pre- and post-event trends (37,38): (i) the preevent slope, indicating how the outcome of interest was changing prior to the occurrence of the critical event; (ii) the change in intercept, identifying the immediate change following the critical event; and (iii) the post-event slope, capturing the gradual change in the period after the critical event. In this article, the critical events are the start of the pandemic on 11 March 2020 and the announcement of the vaccines on 20 December 2020. The start of the pandemic was chosen as a critical event because of the shock it brought to health system functioning and the public attention it raised about the health risks associated with COVID-19 (1). The vaccine announcement was chosen as a second critical event as this signalled the start of a new period where it was possible to partially return to the pre-pandemic status quo (39).

This approach addresses internal validity constraints and represents a methodologically robust design for measuring the impact of critical events (40), and has been widely used in empirical work in the field of public health (41–43). We also computed seven-day moving averages for active COVID-19 cases and COVID-19 deaths using Oxford Government Policy Tracker data (44). These moving averages were added as covariates in the segmented regression model, as trends in COVID-19 cases and deaths may be associated with (digital) healthcare-seeking behaviour during the pandemic. Our segmented regression models were fit using Newey-West standard errors, which are designed to account for autocorrelation and potential heteroskedasticity in time-series data (45). We conducted a post-estimation analysis to capture the exact post-intervention trend.

To ensure that we fit a model that accounts for the correct autocorrelation structure, we performed Cumby-Huizinga tests for autocorrelation for each individual segmented regression model (46). The appropriate lag was determined by performing Cumby-Huizinga tests on segmented regression models without lag. Finally, we conducted a sensitivity analysis to verify the robustness of the segmented regression model, consistent with a placebo intervention using the time period 1 February 2017 to 1 August 2019. Data were extracted using R version 4.1.2 and the analysis was performed using STATA version 17.0.

## Results

### Descriptive analysis

When comparing raw data searches before and after the pandemic announcement on 11 March 2020 (shown in Figure 1), we observe an immediate increase in the relative search volumes of *online doctor, online health, telehealth*, and *telemedicine*. The search volume of *health app* did not indicate an immediate reaction, though the search volumes increased over time. After the immediate increase, we also observed a gradual reduction in the search trends of *online doctor, online health, telehealth*, and *telemedicine*, while *health app* remained stable. Furthermore, there is limited reason to assume “anticipation” as a relevant factor in digital healthcare-related search behaviour, given that search volume substantially increased only after the announcement of the pandemic, and airline traffic data remained constant until the week of 9 March 2020, when the pandemic was announced (47), and daily Google Trends Mobility data of the countries under study remained stable until 11 March 2020 (48). Therefore, changes in human behaviour and mobility appear to emerge shortly after the pandemic declaration (49,50).

![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/30/2022.07.29.22278191/F1.medium.gif)

[Figure 1.](http://medrxiv.org/content/early/2022/07/30/2022.07.29.22278191/F1)

Figure 1. 
Google Trends search volumes before and after the announcement of the COVID-19 pandemic. The vertical axis shows the average search volume (scaled from 0 to 100) before and after the pandemic was announced and after the announcement of conditional market authorisation of the first COVID-19 vaccines.

### Interrupted time-series analysis

Prior to the announcement of the COVID-19 pandemic, *online doctor* and *online health* showed an increase in relative search volume in Canada, the United Kingdom, and the United States, while *telemedicine* only reported an increasing search volume in the United Kingdom. The search volume of *health app* increased in the United Kingdom and the United States. Upon the announcement of the pandemic, Australia, Canada, the United Kingdom, the United States, and Ireland reported an immediate rise of the search volumes of all five keywords under study. In contrast, New Zealand only reported an immediate rise in *online health, telehealth*, and *telemedicine*. After the immediate shock, Australia and the United States reported a declining trend in the search volumes of all keywords under study except *health app*. Canada reported declining search volumes of *online health, telehealth*, and *telemedicine*, yet an increasing search volume in *health app*. New Zealand also reported declining search volumes of *online health, telehealth*, and *telemedicine*. The United Kingdom showed declining search volumes of *online doctor* and *online health*. Finally, Ireland reported a decline in all keywords under study after the immediate shock of the pandemic announcement. Further details are shown in Figure 2 and Table 1.

![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2022/07/30/2022.07.29.22278191/F2.medium.gif)

[Figure 2.](http://medrxiv.org/content/early/2022/07/30/2022.07.29.22278191/F2)

Figure 2. 
Interrupted time-series regression analysis for the relative search volumes of *online doctor, online health, telehealth, telemedicine*, and *health app* before and after the announcement of the COVID-19 pandemic and the announcement of the first COVID-19 vaccines. The models include seven-day moving averages of reported COVID-19 cases and deaths as covariates. The first interruption occurs at 11 March 2020 and the second at 20 December 2020.

View this table:
[Table 1.](http://medrxiv.org/content/early/2022/07/30/2022.07.29.22278191/T1)

Table 1. 
Regression estimates for the relative search volumes of *online doctor, online health, telehealth, telemedicine*, and *health app* before and after the pandemic and the vaccine announcement. The models include seven-day moving averages of reported COVID-19 cases and deaths as covariates.

Following the vaccine announcement on 20 December 2020, the search volume of *telemedicine* in New Zealand reported another immediate rise, while *health app* searches fell. Simultaneously, *health app* search volumes increased in the United Kingdom, the United States, and Ireland. After the immediate impact, Australia reported an upward trend in the search volumes of *online health* and *telemedicine*, while New Zealand reported a positive trend for the search volume of *telehealth*. In the United Kingdom, *telehealth* and *telemedicine* both reported declining trends, as well as *health app* in the United States. Finally, an increasing trend was observed in the search volume of *telemedicine* in Ireland, while *online health* and *health app* decreased. Further details are also shown in Figure 2 and Table 1.

To further check the robustness of our findings, we used data from 2017-2019 and conducted placebo tests with events taking place in the same dates in 2018 (i.e., 08 March 2018, 16 December 2018). eFigure 1 and eTable 1 in Annex A indicate some seasonal shocks in March in *online doctor* and *online health* when using a placebo sample. However, in contrast to our main findings, the shocks in the placebo sample show a downward shock rather than an increase in search volume. While *online health* and *telehealth* reported a positive shock in March in Australia, the main model coefficients are significantly higher than the upper bound of the placebo coefficients’ 95% CI (*online health*, 2.19 - 23.80; *telehealth*, 1.82 - 32.06).

## Discussion

### Summary of key findings

Using Google Trends time-series data, we analysed how digital health search behaviour changed after the pandemic announcement and how it developed during the later stages of the pandemic. We observed an immediate rise in digital health searches in all countries under study, though there is some variability in which keywords were preferred in each country. This is consistent with the claim that healthcare underwent an emergency transition to a digital paradigm when the pandemic started (2). We also observed that – after this immediate spike – digital health search volumes declined again, sometimes towards pre-pandemic levels. The exception to this is the search volume of *health app*, which showed to either remain stable or gradually increase during the pandemic. This can be explained by the notion that all other keywords are meant to substitute traditional healthcare utilisation, while health apps may be used for a wider variety of purposes (e.g., prevention, fitness, managing subclinical health complaints). Additionally, a large volume of health apps was released and advocated for during the pandemic for surveillance purposes (such as recording vaccination status and generating vaccine passports), which may also contribute to explaining the findings of this study. Seeing that vaccines were rolled out in phases across the general population, this can explain why health apps searches remained stable during the entire period under study. Our placebo analysis indicated some statistically significant seasonal trends over the time period 2017-2019. However, these trends were almost exclusively negatively skewed. In the two cases where the trend was positive, our main finding fell outside the confidence interval of the placebo experiment, suggesting that our main results are not driven by an artificial correlation.

### Sustainability of digital health uptake

It is well established that an array of factors need to align in order for a digital innovation to be adopted structurally (2,8), even more so in healthcare which is characteristically one of the slowest domains to adopt digital innovations (51). The beneficial effects of digital health slowly become more apparent and our findings indicate that the general population also seeks out digital healthcare if traditional care is disrupted (2,9). However, our findings indicate that digital healthcare-seeking behaviour is not sustainable at the current stage of progress, seeing how the search volumes under study frequently dropped back to pre-pandemic levels. While this may be partially explained by the return of traditional health services, it also supports previously speculated notions that the digital capacity of the general population may not be sufficiently developed to fully embrace the opportunities brought by digital health (11). Additionally, we highlight how, irrespective of a hypothetical increase in digital competence at a population level over the course of the pandemic, digital skills at later stages in the pandemic continue to be lacking for the purpose of structurally implementing digital health.

This lack of digital readiness extends beyond the general population, as health professionals also continue to voice their concerns regarding the uptake of digital technologies in their practice (7). In light of these reservations, our findings highlight the need for digital (health) training to be developed and implemented in the educational curricula of health professionals. Additionally, training courses for current practitioners may be even more vital, as being informed about the benefits of digital health has been shown to positively influence their decision on using digital health (52). Educational modules and training courses for the general population will also be instrumental in mitigating the effects of the digital health paradox (9).

### Strengths and limitations

This study has a number of strengths. This study is – to our knowledge – the first of its kind that captures the prolonged effect of the COVID-19 pandemic on digital health-seeking behaviour using a comprehensive list of possible search terms, which adequately captures digital health utilisation. While other datasets may exist that capture digital healthcare utilisation, a unique advantage of Google Trends is the high frequency with which it is collected. This allows us to examine both short- and long-term trends from a comparative cross-country perspective. Furthermore, Google Trends data is considered revealed preference data, thus allowing for actual behaviours of Google users to be analysed. Our countries of choice also include a mix of digitally prolific countries (e.g. United Kingdom and Australia) and more digitally hesitant countries (e.g. United States) (9,10), thus providing a balanced representation of search behaviour across countries at various stages of digital health development.

Limitations also need to be considered. While we were not able to ascertain sociodemographic characteristics, we need to consider the possibility of selection bias since young people are more likely to seek health information digitally given their digital skills (11,53). Also, population groups without internet access are unable to contribute to this dataset (11,53). Furthermore, data from only one search engine are included. While Google Trends is the most popular search engine, other search engines may have also been used to seek digital healthcare. Our study does not capture digital health-seeking behaviour of people that directly reached out to their general practitioner or medical contact point. Finally, our data does not allow us to capture the supply-side of digital health delivery (i.e., availability and quality), which may interact with the demand for digital health services.

### Implications for research, policy, practice

Recently, calls for increased and accelerated uptake of digital technologies in healthcare have appeared in various magnitudes (54–57). However, our findings provide a strong signal that such acceleration would be premature and could undermine the real potential of digital health delivery (12). Vulnerable population groups, in particular, may be further disadvantaged and excluded through lack of access to digital infrastructure and underdeveloped digital skills. The recently launched European Health Data Space is an example of how policy frameworks can risk exacerbating existing digital divides if policymakers do not address the underlying lack of access to digital infrastructure and underdeveloped digital skills before structurally rolling out digital health tools (58). As such, capacity-building efforts in the areas of digital infrastructure and skills should be combined with the large-scale introduction of digital health technologies to ensure that citizens and health professionals alike are in the position to recognise the value of digital health tools once they enter the healthcare market at large (7–9,13,52,59). Nevertheless, the deployment of digital health tools should not cease and instead be carefully designed and targeted to specific population groups and health services, because digital health is considered a type of healthcare whose value can only be derived after directly experiencing it (57). Methodically nurturing the readiness and tension for change is, therefore, a vital element in the process of adopting digital health as a mainstream mode of healthcare.

Policy environments are vital to the process of taking up of digital health (9,10). National standards play an important role in creating awareness in markets, setting norms, and safeguarding basic quality dimensions of digital health (10). So long as these factors remain underdeveloped, digital health will face substantial barriers to being implemented and assimilated into national healthcare settings. In fact, the absence of robust market access regulatory frameworks for digital health may also contribute to the current environment where novel tools are stuck in the proof-of-concept or pilot phase and cannot pivot to clinical adoption (3).

Finally, our findings speak to an overarching change in preference for digital health-seeking behaviour. We recognise that the development of digital health is heterogeneous and may differ per health domain. As such, more in-depth analyses of readiness to use digital health are warranted per health domain. Furthermore, educational material should be developed for the purpose of better acquainting the general population with the concept of digital health, building their capacity to harness the potential of digital tools and navigate digital environments (13). More extensive educational material may be prepared for key stakeholder groups, such as patients, health professionals, and policymakers. Closer engagement with the general public and patient groups will further prove to be instrumental in facilitating digital health tools that are widely accessible (9,58).

## Conclusions

This article explores digital health-seeking behaviour in six Anglophone countries at different stages of the COVID-19 pandemic. We show that, after a brief spike, digital health-seeking behaviour frequently declines to levels comparable to before the pandemic; thus, pointing to a lack of readiness to adopt digital health structurally as a mode of healthcare. We highlight potential next steps for digital health integration in the countries under study, while acknowledging the common need for policy innovation, increasing awareness of digital health and its potential, and capacity building (including both digital infrastructure and digital literacy). While the futuristic use of digital technologies in health is promising, we must allow both the policy landscapes and digital health literacy levels to catch up with the rapid advances in technology (2,60). Ultimately, it is evident that more than a pandemic is needed to sustainably implement digital health.

## Supporting information

Annex A [[supplements/278191_file04.docx]](pending:yes)

## Data Availability

Data are publicly available from the Google Trends webpage ([https://trends.google.com/](https://trends.google.com/)). The data extraction, cleaning, and analysis code are openly available: [https://bit.ly/39ZU0oL](https://bit.ly/39ZU0oL)

## Acknowledgements

We would like to thank Ms Alicja Mastylak for her time and effort in proofreading this article.

## Footnotes

*   **Funding** No funding was acquired for this article.

*   **Declaration of interest** No competing interests were reported.

*   **Data availability** Data are publicly available from the Google Trends webpage ([https://trends.google.com/](https://trends.google.com/)). The data extraction, cleaning, and analysis code are openly available: [https://bit.ly/39ZU0oL](https://bit.ly/39ZU0oL).

*   Received July 29, 2022.
*   Revision received July 29, 2022.
*   Accepted July 30, 2022.


*   © 2022, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/)

## References

1.  1.Moynihan R, Sanders S, Michaleff ZA, Scott AM, Clark J, To EJ, et al. Impact of COVID-19 pandemic on utilisation of healthcare services: a systematic review. BMJ Open. 2021 Mar 1;11(3):e045343.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTEvMy9lMDQ1MzQzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMDcvMzAvMjAyMi4wNy4yOS4yMjI3ODE5MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

2.  2.Wong BLH, Maaß L, Vodden A, van Kessel R, Sorbello S, Buttigieg S, et al. The dawn of digital public health in Europe: Implications for public health policy and practice. Lancet Reg Health –Eur. 2022 Mar 1;14:100316.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.lanepe.2022.100316&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=35132399&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

3.  3.Egermark M, Blasiak A, Remus A, Sapanel Y, Ho D. Overcoming Pilotitis in Digital Medicine at the Intersection of Data, Clinical Evidence, and Adoption. Adv Intell Syst. 2022;2200056.
    
    

4.  4.Leigh S, Daly R, Stevens S, Lapajne L, Clayton C, Andrews T, et al. Web-based internet searches for digital health products in the United Kingdom before and during the COVID-19 pandemic: a time-series analysis using app libraries from the Organisation for the Review of Care and Health Applications (ORCHA). BMJ Open. 2021 Oct 1;11(10):e053891.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMzoiMTEvMTAvZTA1Mzg5MSI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIyLzA3LzMwLzIwMjIuMDcuMjkuMjIyNzgxOTEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

5.  5.Auerbach AD, Neinstein A, Khanna R. Balancing Innovation and Safety When Integrating Digital Tools Into Health Care. Ann Intern Med. 2018 Oct 16;169(8):592.
    
    

6.  6.Higgins JP. Smartphone Applications for Patients’ Health and Fitness. Am J Med. 2016 Jan 1;129(1):11–9.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

7.  7.Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci. 2016 Oct 26;11(1):146.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

8.  8.Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A’Court C, et al. Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies. J Med Internet Res. 2017 Nov 1;19(11):e8775.
    
    

9.  9.van Kessel R, Hrzic R, O’Nuallain E, Weir E, Wong BLH, Anderson M, et al. Digital Health Paradox: International Policy Perspectives to Address Increased Health Inequalities for People Living With Disabilities. J Med Internet Res. 2022 Feb 22;24(2):e33819.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2196/33819&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=35191848&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

10. 10.Essén A, Stern AD, Haase CB, Car J, Greaves F, Paparova D, et al. Health app policy: international comparison of nine countries’ approaches. Npj Digit Med. 2022 Mar 18;5(1):1–10.
    
    

11. 11.van Kessel R, Wong BLH, Rubinić I, O’Nuallain E, Czabanowska K. Is Europe prepared to go digital? making the case for developing digital capacity: An exploratory analysis of Eurostat survey data. PLOS Digit Health. 2022 Feb 17;1(2):e0000013.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pdig.0000013&link_type=DOI) 

12. 12.Honeyman M, Maguire D, Evans H, Davies A. Digital technology and health inequalities: a scoping review [Internet]. Cardiff: Public Health Wales NHS Trust; 2020 p. 44. Available from: [https://dx.plos.org/10.1371/journal.pdig.0000013](https://dx.plos.org/10.1371/journal.pdig.0000013)
    
    

13. 13.van Kessel R, Wong BLH, Clemens T, Brand H. Digital health literacy as a super determinant of health: More than simply the sum of its parts. Internet Interv. 2022 Mar 1;27:100500.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.invent.2022.100500&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=35242586&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

14. 14.Wharton GA, Sood HS, Sissons A, Mossialos E. Virtual primary care: fragmentation or integration? Lancet Digit Health. 2019 Nov 1;1(7):e330–1.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S2589-7500(19)30152-9&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33323207&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

15. 15.Kubb C, Foran HM. Online Health Information Seeking by Parents for Their Children: Systematic Review and Agenda for Further Research. J Med Internet Res. 2020 Aug 25;22(8):e19985.
    
    

16. 16.Jacobs W, Amuta AO, Jeon KC. Health information seeking in the digital age: An analysis of health information seeking behavior among US adults. Alvares C, editor. Cogent Soc Sci. 2017 Jan 1;3(1):1302785.
    
    

17. 17.de Albuquerque Veloso Machado M, Roberts B, Wong BLH, van Kessel R, Mossialos E. The Relationship Between the COVID-19 Pandemic and Vaccine Hesitancy: A Scoping Review of Literature Until August 2021. Front Public Health. 2021;9:747787.
    
    

18. 18.Howell EL, Brossard D. (Mis)informed about what? What it means to be a science-literate citizen in a digital world. Proc Natl Acad Sci. 2021 Apr 13;118(15):e1912436117.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoicG5hcyI7czo1OiJyZXNpZCI7czoxODoiMTE4LzE1L2UxOTEyNDM2MTE3IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMDcvMzAvMjAyMi4wNy4yOS4yMjI3ODE5MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

19. 19.Arora VS, McKee M, Stuckler D. Google Trends: Opportunities and limitations in health and health policy research. Health Policy. 2019 Mar 1;123(3):338–41.
    
    

20. 20.Zhao X, Coxe SJ, Timmons AC, Frazier SL. Mental Health Information Seeking Online: A Google Trends Analysis of ADHD. Adm Policy Ment Health Ment Health Serv Res [Internet]. 2021 Sep 22 [cited 2022 Feb 22]; Available from: [https://doi.org/10.1007/s10488-021-01168-w](https://doi.org/10.1007/s10488-021-01168-w)
    
    

21. 21.Nuti SV, Wayda B, Ranasinghe I, Wang S, Dreyer RP, Chen SI, et al. The Use of Google Trends in Health Care Research: A Systematic Review. PLOS ONE. 2014 Oct 22;9(10):e109583.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0109583&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25337815&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

22. 22.Brodeur A, Clark AE, Fleche S, Powdthavee N. COVID-19, lockdowns and well-being: Evidence from Google Trends. J Public Econ. 2021 Jan;193:104346.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jpubeco.2020.104346&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

23. 23.Mavragani A, Ochoa G. Google Trends in Infodemiology and Infoveillance: Methodology Framework. JMIR Public Health Surveill. 2019 May 29;5(2):e13439.
    
    

24. 24.1.  Ryan M, 
    2.  Gerard K, 
    3.  Amaya-Amaya M
    
    Mark T, Swait J. Using Stated Preference and Revealed Preference Data Fusion Modelling in Health Care. In: Ryan M, Gerard K, Amaya-Amaya M, editors. Using Discrete Choice Experiments to Value Health and Health Care [Internet]. Dordrecht: Springer Netherlands; 2008 [cited 2022 Jun 4]. p. 217–34. (The Economics of Non-Market Goods and Resources). Available from: [https://doi.org/10.1007/978-1-4020-5753-3_10](https://doi.org/10.1007/978-1-4020-5753-3_10)
    
    

25. 25.Mavragani A, Ochoa G, Tsagarakis KP. Assessing the Methods, Tools, and Statistical Approaches in Google Trends Research: Systematic Review. J Med Internet Res. 2018 Nov 6;20(11):e9366.
    
    

26. 26.Jun SP, Yoo HS, Choi S. Ten years of research change using Google Trends: From the perspective of big data utilizations and applications. Technol Forecast Soc Change. 2018 May 1;130:69–87.
    
    

27. 27.Rovetta A, Castaldo L. A new infodemiological approach through Google Trends: longitudinal analysis of COVID-19 scientific and infodemic names in Italy. BMC Med Res Methodol. 2022 Jan 30;22(1):33.
    
    

28. 28.Pullan S, Dey M. Vaccine hesitancy and anti-vaccination in the time of COVID-19: A Google Trends analysis. Vaccine. 2021 Apr 1;39(14):1877–81.
    
    

29. 29.Kalhori SRN, Bahaadinbeigy K, Deldar K, Gholamzadeh M, Hajesmaeel-Gohari S, Ayyoubzadeh SM. Digital Health Solutions to Control the COVID-19 Pandemic in Countries With High Disease Prevalence: Literature Review. J Med Internet Res. 2021 Mar 10;23(3):e19473.
    
    

30. 30.Gunasekeran DV, Tseng RMWW, Tham YC, Wong TY. Applications of digital health for public health responses to COVID-19: a systematic scoping review of artificial intelligence, telehealth and related technologies. Npj Digit Med. 2021 Feb 26;4(1):1–6.
    
    

31. 31.Widmer RJ, Collins NM, Collins CS, West CP, Lerman LO, Lerman A. Digital Health Interventions for the Prevention of Cardiovascular Disease: A Systematic Review and Meta-analysis. Mayo Clin Proc. 2015 Apr 1;90(4):469–80.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.mayocp.2014.12.026&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25841251&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

32. 32.StatCounter. Search Engine Market Share Europe [Internet]. StatCounter Global Stats. 2022 [cited 2022 May 12]. Available from: [https://gs.statcounter.com/search-engine-market-share/all/europe](https://gs.statcounter.com/search-engine-market-share/all/europe)
    
    

33. 33.StatCounter. Search Engine Market Share North America [Internet]. StatCounter Global Stats. 2022 [cited 2022 May 12]. Available from: [https://gs.statcounter.com/search-engine-market-share/all/north-america](https://gs.statcounter.com/search-engine-market-share/all/north-america)
    
    

34. 34.StatCounter. Search Engine Market Share Oceania [Internet]. StatCounter Global Stats. 2022 [cited 2022 May 12]. Available from: [https://gs.statcounter.com/search-engine-market-share/all/oceania](https://gs.statcounter.com/search-engine-market-share/all/oceania)
    
    

35. 35.Bernal JL, Cummins S, Gasparrini A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. 2017 Feb;46(1):348–55.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ije/dyw098&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27283160&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

36. 36.Hategeka C, Ruton H, Karamouzian M, Lynd LD, Law MR. Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review. BMJ Glob Health. 2020 Oct 1;5(10):e003567.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.ncbi.nlm.nih.gov/pubmed/33055094&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

37. 37.Taljaard M, McKenzie JE, Ramsay CR, Grimshaw JM. The use of segmented regression in analysing interrupted time series studies: an example in pre-hospital ambulance care. Implement Sci. 2014 Jun 19;9(1):77.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1748-5908-9-77&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24943919&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

38. 38.Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002 Aug;27(4):299–309.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1046/j.1365-2710.2002.00430.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12174032&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000177464200009&link_type=ISI) 

39. 39.Perez-Arce F, Angrisani M, Bennett D, Darling J, Kapteyn A, Thomas K. COVID-19 vaccines and mental distress. PLOS ONE. 2021 Sep 8;16(9):e0256406.
    
    

40. 40.Ansari F, Gray K, Nathwani D, Phillips G, Ogston S, Ramsay C, et al. Outcomes of an intervention to improve hospital antibiotic prescribing: interrupted time series with segmented regression analysis. J Antimicrob Chemother. 2003 Nov 1;52(5):842–8.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jac/dkg459&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=14563900&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000186446100016&link_type=ISI) 

41. 41.Karaivanov A, Kim D, Lu SE, Shigeoka H. COVID-19 vaccination mandates and vaccine uptake. Nat Hum Behav. 2022 Jun 2;1–10.
    
    

42. 42.Hadeiy SK, Zamani N, McDonald R, Rezaei O, Kolahi AA, Gholami N, et al. An interrupted time series analysis of hospital admissions due to alcohol intoxication during the COVID-19 pandemic in Tehran, Iran. Sci Rep. 2022 Jun 22;12(1):10574.
    
    

43. 43.Larson PS, Bergmans RS. Impact of the COVID-19 pandemic on temporal patterns of mental health and substance abuse related mortality in Michigan: An interrupted time series analysis. Lancet Reg Health –Am [Internet]. 2022 Jun 1 [cited 2022 Jul 1];10. Available from: [https://www.thelancet.com/journals/lanam/article/PIIS2667-193X(22)00035-7/fulltext](https://www.thelancet.com/journals/lanam/article/PIIS2667-193X(22)00035-7/fulltext)
    
    

44. 44.Hale T, Angrist N, Goldszmidt R, Kira B, Petherick A, Phillips T, et al. A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat Hum Behav. 2021 Apr;5(4):529–38.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

45. 45.Linden A. Conducting Interrupted Time-series Analysis for Single-and Multiple-group Comparisons. Stata J. 2015 Jun 1;15(2):480–500.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1536867x1501500208&link_type=DOI) 

46. 46.Cumby RE, Huizinga J. Testing the Autocorrelation Structure of Disturbances in Ordinary Least Squares and Instrumental Variables Regressions. Econometrica. 1992;60(1):185–95.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2307/2951684&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1992HD77300011&link_type=ISI) 

47. 47.EUROCONTROL. COVID-19 impact on the European air traffic network [Internet]. 2022 [cited 2022 Jun 3]. Available from: [https://www.eurocontrol.int/covid19](https://www.eurocontrol.int/covid19)
    
    

48. 48.Ritchie H, Mathieu E, Rodés-Guirao L, Appel C, Giattino C, Ortiz-Ospina E, et al. COVID-19: Google Mobility Trends. Our World Data [Internet]. 2020 Mar 5 [cited 2022 Jun 3]; Available from: [https://ourworldindata.org/covid-google-mobility-trends](https://ourworldindata.org/covid-google-mobility-trends)
    
    

49. 49.Jun SP, Yoo HS, Lee JS. The impact of the pandemic declaration on public awareness and behavior: Focusing on COVID-19 google searches. Technol Forecast Soc Change. 2021 May 1;166:120592.
    
    

50. 50.Chan HF, Skali A, Savage DA, Stadelmann D, Torgler B. Risk attitudes and human mobility during the COVID-19 pandemic. Sci Rep. 2020 Nov 16;10(1):19931.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F07%2F30%2F2022.07.29.22278191.atom) 

51. 51.Bartlett R, Somauroo A, Zerbi C. How the medtech industry can capture value from digital health. McKinsey&Company; 2021 May p. 9.
    
    

52. 52.Vonken L, Hussein H, Crutzen R, Vluggen S. Perceptions of Dutch general practitioners towards eHealth for patients with type-2 diabetes: a qualitative study. Fam Pract. 2022 Jun 25;cmac066.
    
    

53. 53.Kickbusch I, Piselli D, Agrawal A, Balicer R, Banner O, Adelhardt M, et al. The Lancet and Financial Times Commission on governing health futures 2030: growing up in a digital world. The Lancet. 2021 Nov 6;398(10312):1727–76.
    
    

54. 54.Scott BK, Miller GT, Fonda SJ, Yeaw RE, Gaudaen JC, Pavliscsak HH, et al. Advanced Digital Health Technologies for COVID-19 and Future Emergencies. Telemed E-Health [Internet]. 2020 Oct 1 [cited 2022 May 26]; Available from: [https://www.liebertpub.com/doi/full/10.1089/tmj.2020.0140](https://www.liebertpub.com/doi/full/10.1089/tmj.2020.0140)
    
    

55. 55.Mosnaim GS, Stempel H, Van Sickle D, Stempel DA. The Adoption and Implementation of Digital Health Care in the Post–COVID-19 Era. J Allergy Clin Immunol Pract. 2020 Sep;8(8):2484–6.
    
    

56. 56.Inkster B, O’Brien R, Selby E, Joshi S, Subramanian V, Kadaba M, et al. Digital Health Management During and Beyond the COVID-19 Pandemic: Opportunities, Barriers, and Recommendations. JMIR Ment Health. 2020 Jul 6;7(7):e19246.
    
    

57. 57.Busso M, Gonzalez MP, Scartascini C. On the demand for telemedicine: Evidence from the COVID-19 pandemic. Health Econ [Internet]. 2022 [cited 2022 May 26]; Available from: [https://onlinelibrary.wiley.com/doi/abs/10.1002/hec.4523](https://onlinelibrary.wiley.com/doi/abs/10.1002/hec.4523)
    
    

58. 58.van Kessel R, Wong BLH, Forman R, Gabrani J, Mossialos E. The European Health Data Space fails to bridge digital divides. BMJ. 2022 Jul 8;378:e071913.
    
    [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE5OiIzNzgvanVsMDhfMy9lMDcxOTEzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMDcvMzAvMjAyMi4wNy4yOS4yMjI3ODE5MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

59. 59.Tan YR, Agrawal A, Matsoso MP, Katz R, Davis SLM, Winkler AS, et al. A call for citizen science in pandemic preparedness and response: beyond data collection. BMJ Glob Health. 2022 Jun 1;7(6):e009389.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiYm1qZ2giO3M6NToicmVzaWQiO3M6MTE6IjcvNi9lMDA5Mzg5IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjIvMDcvMzAvMjAyMi4wNy4yOS4yMjI3ODE5MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

60. 60.Sheikh A, Anderson M, Albala S, Casadei B, Franklin BD, Richards M, et al. Health information technology and digital innovation for national learning health and care systems. Lancet Digit Health. 2021 Jun 1;3(6):e383–96.