Forecasting admissions in psychiatric hospitals before and during Covid-19
==========================================================================

* Jan Wolff
* Ansgar Klimke
* Michael Marschollek
* Tim Kacprowski

## Abstract

**Introduction** The COVID-19 pandemic has strong effects on most health care systems and individual services providers. Forecasting of admissions can help for the efficient organisation of hospital care. We aimed to forecast the number of admissions to psychiatric hospitals before and during the COVID-19 pandemic and we compared the performance of machine learning models and time series models. This would eventually allow to support timely resource allocation for optimal treatment of patients.

**Methods** We used admission data from 9 psychiatric hospitals in Germany between 2017 and 2020. We compared machine learning models with time series models in weekly, monthly and yearly forecasting before and during the COVID-19 pandemic. Our models were trained and validated with data from the first two years and tested in prospectively sliding time-windows in the last two years.

**Results** A total of 90,686 admissions were analysed. The models explained up to 90% of variance in hospital admissions in 2019 and 75% in 2020 with the effects of the COVID-19 pandemic. The best models substantially outperformed a one-step seasonal naïve forecast (seasonal mean absolute scaled error (sMASE) 2019: 0.59, 2020: 0.76). The best model in 2019 was a machine learning model (elastic net, mean absolute error (MAE): 7.25). The best model in 2020 was a time series model (exponential smoothing state space model with Box-Cox transformation, ARMA errors and trend and seasonal components, MAE: 10.44), which adjusted more quickly to the shock effects of the COVID-19 pandemic. Models forecasting admissions one week in advance did not perform better than monthly and yearly models in 2019 but they did in 2020. The most important features for the machine learning models were calendrical variables.

**Conclusion** Model performance did not vary much between different modelling approaches before the COVID-19 pandemic and established forecasts were substantially better than one-step seasonal naïve forecasts. However, weekly time series models adjusted quicker to the COVID-19 related shock effects. In practice, different forecast horizons could be used simultaneously to allow both early planning and quick adjustments to external effects.

Index Terms
*   Forecasting
*   Psychiatry
*   Hospitals
*   Covid-19

## I. Introduction

**H**ealth care systems need to balance potentially unlimited demand for services with scarce health care resources [1]. This balance might become even more difficult in the future due to an aging population that leads to increased demand for health care services and reduced medical work force [2]. Hospitals often consume a large part of total health care budgets [3]. A possible way to mitigate resource constraints in hospitals is the use of modern technologies to make services more cost-effective [4]. For instance, the increasing availability of data allows to support medical decision making in hospitals with information derived from machine learning algorithms [5, 6].

Efficient resource allocation in hospitals requires the management of volatile demand and available resources. This management is more critical in hospitals than in other areas since lack of timely and sufficient services can lead to negative patient outcomes [7]. Insufficient staff can lead to increased morbidity and mortality [8, 9]. Due to the shortage of trained medical staff in many health care systems, securing sufficient medical staff to permanently meet patient needs becomes a critical objective [10]. Inpatient mental health care is more staff intensive than other medical disciplines due to the personal nature of many interventions [11, 12].

The Covid-19 pandemic has strong effects on most health care systems and individual service providers [13]. A sudden surge in patients requiring intensive respiratory care and the possible shortage of ICU capacities led to political supply side interventions in many health care systems [14]. In Hesse, Germany, somatic and psychiatric hospitals were required to restrict new admissions to urgent care cases and avoid elective patients from 16th March 2020. Furthermore, new hospital hygiene regulations and the requirement to quarantine patients reduced hospital capacities.

Forecasting of admissions can help for the efficient organisation of hospital care and for the adjustment of resources to sudden changes in patient volumes. We aimed to forecast the number of admissions to psychiatric hospitals before and during the COVID-19 pandemic and we compared the performance of machine learning models and time series models.

## II. Methods

### A. Data

We included all inpatient admissions from 01. January 2017 to 31. December 2020 to nine hospitals in Hesse, Germany. These hospitals are part of a common service provider and account for about half of all inpatient mental health care in the state of Hesse. Aggregated admission numbers per day were obtained from the hospital administrations and did not contain individual patient data. Returns after planned interruptions, such as home leave, were excluded. Multiple separate admissions of the same patient were counted individually. Admissions to the departments of child and adolescent psychiatry and admissions to the departments for psychosomatic medicine were excluded.

We obtained weather and climate data from the Climate Data Centre of Germany’s National Meteorological Service [15]. We used the gtrendsR package to query Google trend data for Hesse, Germany [16]. School holidays and public holidays were obtained from publicly available calendars.

### B. Analyses

We used machine learning and time series models to predict the number of hospital admissions for each day of year 2019 and 2020. The machine learning models were a) gradient boosting with trees (XGB) [17], b) support vector machines (SVM) [18] and c) elastic nets [19]. The time series models were a) exponential smoothing state space models (ETS) [20], b) exponential smoothing state space models with screening for Box-Cox transformation, ARMA errors and trend and seasonal components (TBATS) [21] and c) additive models with non-linear trends fitted by seasonal effects (PROPHET) [22]. We compared models forecasting a week in advance, a month in advance and a whole year one week in advance.

### C. Features

Our machine learning model used calendrical variables, climate and weather data, google trend data, Fourier terms and lagged number of admissions as features. The calendrical features were day of week, weekend, public holiday, school holiday, quarter of the year, month of the year, bridge days, i.e. days between a public holiday and the weekend and the end of the year, i.e. the days between Christmas and new year’s eve. The climate and weather data were wind speed, cloudiness, air pressure, precipitation depth and type, duration of sunshine, snow height, air temperature and humidity. Since the weather of future days was unknown at the point of prediction we used lagged values, i.e. the weekly model used the weather 7 days before the predicted day and the monthly models used the weather data 28 days before the predicted day. We did not use weather data for the yearly model.

We used the following keywords in google trend data: depression, sadness, suicide, mania, fear, panic, addiction, dependence, alcohol, drugs, schizophrenia, psychosis and hallucinations. As for the weather data, we used lagged values of google trend data. The weekly models used the number of admissions 14 days before the predicted day, because the number of admissions was not known yet on day 7 before prediction, as additional feature and the monthly model used these values with a lag of 35. Our time series models did not use feature variables.

### D. Training and testing

We used prospectively sliding time windows to validate (2018) and test (2019 and 2020) model performance. The final weekly models predicted each day of one full week of hospital admissions seven days in advance. We tested one model for each week and study site in 2019 and 2020, thereby incrementally prolonging the training period and forwarding the 7-day testing period each by one week. The monthly models each predicted 28 days of hospital admissions in advance and the incremental slides were 28 days. In the yearly models, we predicted the whole year of 2019 and 2020 each one week before the years started.

We compared model performance with the Root-Mean-Squared-Error (RMSE), the R2, the Mean Absolute Error (MAE) and a seasonal Mean Absolute Scaled Error (sMASE). The sMASE [23] was calculated by dividing the MAE of our weekly, monthly and yearly forecasts by the MAE derived from a naïve forecast based on the number of admissions observed 14 days, 35 days and 364 days before the predicted day, respectively.

## III. Results

The number of admissions showed a relatively strong weekly seasonality and a yearly seasonality. Figure 1 provides the results of a multiple seasonal decomposition of the number of daily admissions by loess [24]. There was no strong trend in admission numbers during the first three years, until the commencement of the Corona hospital regulation on March 16th had a clear negative effect on the number of admissions.

![Fig. 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/20/2021.07.16.21260200/F1.medium.gif)

[Fig. 1.](http://medrxiv.org/content/early/2021/07/20/2021.07.16.21260200/F1)

Fig. 1. 
Multiple seasonal decomposition by loess. The y-axes are scaled differently.

Table I shows the forecasting performance in 2019 and in 2020 at all study sites combined. The naïve seasonal forecasts were based on the number of admissions 14, 35 and 364 days before the predicted day for the weekly, monthly and yearly models, respectively. In absolute terms, the best model in 2019 was the weekly elastic net, which achieved a MAE of 7.25 days and an explained variance of 90%. Compared to a naïve forecast based on the number of admissions two weeks in advance, this model achieved a forecast improvement of 38% (sMASE=0.62). In absolute terms, the best model in 2020 was the weekly TBATS model. However, compared to the monthly possible naïve forecast, i.e. the number of admission 35 days in advance, the highest improvement was achieved with the monthly SVM.

View this table:
[TABLE I.](http://medrxiv.org/content/early/2021/07/20/2021.07.16.21260200/T1)

TABLE I. 
Forecasting performance in 2019 and in 2020. Best values per column are in boldface. RMSE= Root-Mean-Squared-Error, MAE= Mean Absolute Error, sMASE= Seasonal Mean Absolute Scaled Error. 1: The naïve seasonal forecasts were based on the number of admissions 14, 35 and 364 days before the predicted day for the weekly, monthly and yearly models, respectively.

The error accumulation in 2019 and 2020 at all study sites combined is shown in Figure 2. While model performance was relatively similar in 2019, errors diverged after commencement of the Corona hospital regulation on March 16th, 2020. Weekly time series models adjusted quicker to the new circumstances and accumulated less error until the end of year 2020.

![Fig. 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/20/2021.07.16.21260200/F2.medium.gif)

[Fig. 2.](http://medrxiv.org/content/early/2021/07/20/2021.07.16.21260200/F2)

Fig. 2. 
Cumulated mean absolute error in 2019 and 2020 by machine learning and time series models. XGB= Gradient boosting with trees, SVM= Support vector machines, ETS= Exponential smoothing state space models, TBATS= Exponential smoothing state space models with Box-Cox transformation, ARMA errors and trend and seasonal components, PROPHET= Additive models with non-linear trends fitted by seasonal effects.

The forecasting models showed variation in performance between study sites. Figure 3 shows differences in percentage errors between study sites per week derived from the overall best performing weekly machine learning and time-series models (see Table I), respectively. Both models performed similar in year 2019. However, the elastic net caused less error peaks, for instance at easter Monday and during Christmas time because it had these holidays as features. In contrast, the TBATS model adjusted quicker to the corona regulations and adjusted to the new level of admission numbers during the rest of 2020 better than the elastic net.

![Fig. 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/20/2021.07.16.21260200/F3.medium.gif)

[Fig. 3.](http://medrxiv.org/content/early/2021/07/20/2021.07.16.21260200/F3)

Fig. 3. 
Variation of percentage error between study sites. TBATS= Exponential smoothing state space models with Box-Cox transformation, ARMA errors and trend and seasonal components, IQR= Interquartile range.

Figure 4 shows the top 25 feature variables ordered by their importance in forecasting the number of admissions with the elastic net, which was the best performing machine learning algorithm in our comparison. Variable importance represents the influence of each feature on the forecast performance relative to the other variables [25]. The strongest influence on forecast performance was found in calendrical variables.

![Fig. 4.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/07/20/2021.07.16.21260200/F4.medium.gif)

[Fig. 4.](http://medrxiv.org/content/early/2021/07/20/2021.07.16.21260200/F4)

Fig. 4. 
Variable importance of TOP 25 features in machine leaning models. Dec= December.

## IV. Discussion

### A. Key findings

We aimed to forecast the number of admissions to psychiatric hospitals before and during the COVID-19 pandemic and we compared the performance of machine learning models and time series models. This would eventually allow to support timely resource allocation for optimal treatment of patients. Model performance did not vary much between different modelling approaches before the COVID-19 pandemic. Established forecasts were substantially better than seasonal naïve forecasts. The most important features were calendrical variables that did not require short term adjustments in weekly and monthly models. However, weekly time series models adjusted quicker to the COVID-19 related shock effects than monthly and yearly models and the machine learning models.

### B. Strength and weaknesses

A strength of our study were the data of four years from nine hospitals representing about half of all inpatient psychiatric admissions in Hesse, Germany. This allowed both to give a representative picture of inpatient psychiatric care in Germany and to show how the forecasting approaches work at different study sites. Furthermore, it was possible to analyse the effect of sudden changes in hospital admissions to the performance of different modelling approaches due to the commencement of the Corona hospital regulation in March 2020.

A limitation of our study was the lack of data to differentiate between causes of reduced hospital admissions after the corona regulation came into effect in March 2020. The reduced admissions could have been a result of different supply side and demand side effects, such as avoidance of elective admissions, reduced capacities due to isolation and quarantine requirements and unwillingness of patients to enter hospitals during the Corona crisis. Another limitation of our study was its restriction to one large German provider of inpatient mental health care, which requires a lot of care when translating to different healthcare systems or different clinical settings.

### C. Comparison to previous research

Previous studies often focused on emergency departments [26] and there were no previous studies that analysed forecasting of psychiatric hospital admissions comparable to our study in scale and scope.

Vollmer *et al* 2021 predicted admission numbers in the emergency departments of two hospitals in London with data from 2011 to 2018 [27]. They compared machine learning models to more traditional time series models to make forecasts of admissions one, three and seven days in advance. The forecasts of different time horizons, i.e., one, three and seven days in advance, performed very similar. This is comparable to our findings of relatively similar results between weekly, monthly and yearly predictions, although at a different scale. In contrast to our study, lagged admissions from previous weeks were among the strongest predictors, probably related to the stronger increase and decrease of admission number levels during the study period at the different hospitals in comparison to our study. As in our study, Vollmer *et al* also found that calendrical variables were among the features with the strongest influence on forecasting performance. Weather and climate data and google search data had a relatively low influence on forecasting performance.

Similar results were found by Boutsioli *et al*, who used a simple OLS regression to forecast hospital admissions to the emergency departments of ten public hospitals in Greece [28]. They only used the calendrical variables weekend, summer holiday, public holiday and the participation in emergency care in their model and explained a relatively high variance in hospital admissions of up to 88%.

Jones *et al* forecasted the admission numbers at three emergency departments in the USA one, seven, fourteen, twenty-one and thirty days in advance [29]. They used autoregressive integrated moving average (ARIMA) models, time series regression, exponential smoothing, and artificial neural network models to predict admissions per day. They also found that admissions were characterised by yearly and weekly seasonality (see for comparison our Figure 1). As in our study, they found a relatively low improvement in forecasting performance in the shorter forecasting horizons in comparison to the longer horizons. Similar to our study and to the study of Vollmer *et al* [27], weather and climate had a relatively low influence on forecasting performance.

McCoy *et al* forecasted hospital discharge numbers at two academic medical centers in the USA [30]. They compared the performance of a PROPHET model with a seasonal ARIMA model and a one-step naïve seasonal forecast and compared monthly models to yearly models. The best performance was achieved by a PROPHET model. Comparable to our study, they also found relatively low to none improvement of forecasting accuracy in refitting their models monthly in comparison to yearly models.

## V. Conclusions

Accurate forecasting of hospital admissions can help for the efficient organisation of hospital care and for the adjustment of resources to sudden changes in patient volumes. We found a substantial improvement of forecasting accuracy in comparison to a seasonally adjusted naïve baseline forecast. Model performance did not vary much between different modelling approaches and different forecasting horizons before the COVID-19 pandemic. However, weekly time series models adjusted quicker to the COVID-19 related shock effects. In practice, different forecast horizons could be used simultaneously to allow both early planning and quick adjustments to external effects.

## Data Availability

Data can be provided upon reasonable request.

*   Received July 16, 2021.
*   Revision received July 16, 2021.
*   Accepted July 20, 2021.


*   © 2021, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution 4.0 International), CC BY 4.0, as described at [http://creativecommons.org/licenses/by/4.0/](http://creativecommons.org/licenses/by/4.0/)

## References

1.  1.Hurst, K. & Smith, A. Temporary nursing staff - cost and quality issues. Journal of advanced nursing 67, 287–296. ISSN: 1365-2648. doi:10.1111/j.1365-2648.2010.05471.x (Feb. 2011).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1365-2648.2010.05471.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20969614&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F20%2F2021.07.16.21260200.atom) 

2.  2.Dall, T. M. et al. An Aging Population And Growing Disease Burden Will Require ALarge And Specialized Health Care Workforce By 2025. Health Affairs 32, 2013–2020. ISSN: 0278-2715, 1544-5208. doi:10.1377/hlthaff.2013.0714. [http://www.healthaffairs.org/doi/10.1377/hlthaff.2013.0714](http://www.healthaffairs.org/doi/10.1377/hlthaff.2013.0714) (2021) (Nov. 2013).
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjEwOiIzMi8xMS8yMDEzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDcvMjAvMjAyMS4wNy4xNi4yMTI2MDIwMC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

3.  3.WHO. Global spending on health 2020: weathering the storm. Geneva: World Health Organization 2020.
    
    

4.  4.Ekman, B. Cost Analysis of a Digital Health Care Model in Sweden. PharmacoEconomics - Open 2, 347–354. ISSN: 2509-4262, 2509-4254. doi:10.1007/s41669-017-0059-7. [http://link.springer.com/10.1007/s41669-017-0059-7](http://link.springer.com/10.1007/s41669-017-0059-7) (2021) (Sept. 2018).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s41669-017-0059-7&link_type=DOI) 

5.  5.Wolff, J. et al. Predicting patient outcomes in psychiatric hospitals with routine data: a machine learning approach. BMC Medical Informatics and Decision Making 20, 21. ISSN: 1472-6947. doi:10.1186/s12911-020-1042-2. [https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-020-1042-2](https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-020-1042-2) (2020) (Dec. 2020).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12911-020-1042-2&link_type=DOI) 

6.  6.Wolff, J. et al. Predicting the risk of drug–drug interactions in psychiatric hospitals: a retrospective longitudinal pharmacovigilance study. BMJ Open 11, e045276. doi:10.1136/bmjopen-2020-045276. [http://bmjopen.bmj.com/content/11/4/e045276.abstract](http://bmjopen.bmj.com/content/11/4/e045276.abstract) (Apr. 2021).
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMjoiMTEvNC9lMDQ1Mjc2IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDcvMjAvMjAyMS4wNy4xNi4yMTI2MDIwMC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

7.  7.Jack, E. P. & Powers, T. L. A review and synthesis of demand management, capacity management and performance in health-care services. International Journal of Management Reviews 11, 149–174. ISSN: 14608545, 14682370. doi:10.1111/j.1468-2370.2008.00235.x. [http://doi.wiley.com/10.1111/j.1468-2370.2008.00235.x](http://doi.wiley.com/10.1111/j.1468-2370.2008.00235.x) (2021) (June 2009).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1468-2370.2008.00235.x&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000266550300002&link_type=ISI) 

8.  8.Bernstein, S. L. et al. The Effect of Emergency Department Crowding on Clinically Oriented Outcomes. Academic Emergency Medicine 16, 1–10. ISSN: 1553-2712. doi:10.1111/j.1553-2712.2008.00295.x. [https://www.onlinelibrary.wiley.com/doi/abs/10.1111/j.1553-2712.2008.00295.x](https://www.onlinelibrary.wiley.com/doi/abs/10.1111/j.1553-2712.2008.00295.x) (2019) (2009).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1553-2712.2008.00295.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19007346&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F20%2F2021.07.16.21260200.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000262047000001&link_type=ISI) 

9.  9.Needleman, J., Buerhaus, P., Mattke, S., Stewart, M. & Zelevinsky, K. Nurse-Staffing Levels and the Quality of Care in Hospitals. New England Journal of Medicine 346, 1715–1722. ISSN: 0028-4793. doi:10.1056/NEJMsa012247. [https://doi.org/10.1056/NEJMsa012247](https://doi.org/10.1056/NEJMsa012247) (2019) (May 2002).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMsa012247&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12037152&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F20%2F2021.07.16.21260200.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000175834100007&link_type=ISI) 

10. 10.Michel, J.-P. & Ecarnot, F. The shortage of skilled workers in Europe: its impact on geriatric medicine. European Geriatric Medicine 11, 345–347. ISSN: 1878-7657. doi:10.1007/s41999-020-00323-0. [https://link.springer.com/10.1007/s41999-020-00323-0](https://link.springer.com/10.1007/s41999-020-00323-0) (2021) (June 2020).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s41999-020-00323-0&link_type=DOI) 

11. 11.Wolff, J. et al. Work-Time Distribution of Physicians at a German University Hospital. Deutsches Arzteblatt International 114, 705–711. ISSN: 1866-0452. doi:10.3238/arztebl.2017.0705 (2017).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3238/arztebl.2017.0705&link_type=DOI) 

12. 12.Wolff, J. et al. A work time study analysing differences in resource use between psychiatric inpatients. Social Psychiatry and Psychiatric Epidemiology 50, 1309– 1315. ISSN: 1433-9285. doi:10.1007/s00127-015-1041-2 (2015).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00127-015-1041-2&link_type=DOI) 

13. 13.Blumenthal, D., Fowler, E. J., Abrams, M. & Collins, S. R. Covid-19 — Implications for the Health Care System. New England Journal of Medicine 383, 1483– 1488. ISSN: 0028-4793, 1533-4406. doi:10.1056/NEJMsb2021088. [http://www.nejm.org/doi/10.1056/NEJMsb2021088](http://www.nejm.org/doi/10.1056/NEJMsb2021088) (2021) (Oct. 2020).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMsb2021088&link_type=DOI) 

14. 14.Netters, S. et al. Pandemic ICU triage challenge and medical ethics. BMJ Supportive & Palliative Care 11, 133–137. ISSN: 2045-435X, 2045-4368. doi:10.1136/bmjspcare-2020-002793. [https://spcare.bmj.com/lookup/doi/10.1136/bmjspcare-2020-002793](https://spcare.bmj.com/lookup/doi/10.1136/bmjspcare-2020-002793) (2021) (June 2021).
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiYm1qc3BjYXJlIjtzOjU6InJlc2lkIjtzOjg6IjExLzIvMTMzIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDcvMjAvMjAyMS4wNy4xNi4yMTI2MDIwMC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

15. 15.Service, N. M. Climate Data Centre. [https://cdc.dwd.de/portal/](https://cdc.dwd.de/portal/) 2021.
    
    

16. 16.Massicotte, P. & Eddelbuettel, D. gtrendsR: Perform and Display Google Trends Queries. Version 1.4.8. Feb. 2021. [https://CRAN.R-project.org/package=gtrendsR](https://CRAN.R-project.org/package=gtrendsR).
    
    

17. 17.Friedman, J. H. Greedy Function Approximation: A Gradient Boosting Machine. The Annals of Statistics 29, 1189–1232. ISSN: 0090-5364. [https://www.jstor.org/stable/2699986](https://www.jstor.org/stable/2699986) (2019) (2001).
    
    

18. 18.Hearst, M., Dumais, S., Osuna, E., Platt, J. & Scholkopf, B. Support vector machines. IEEE Intelligent Systems and their Applications 13, 18–28. ISSN: 1094-7167. doi:10.1109/5254.708428. [http://ieeexplore.ieee.org/document/708428/](http://ieeexplore.ieee.org/document/708428/) (2021) (July 1998).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1109/5254.708428&link_type=DOI) 

19. 19.Zou, H. & Hastie, T. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67, 301–320. ISSN: 1369-7412, 1467-9868. doi:10.1111/j.1467-9868.2005.00503.x. [http://doi.wiley.com/10.1111/j.1467-9868.2005.00503.x](http://doi.wiley.com/10.1111/j.1467-9868.2005.00503.x) (2021) (Apr. 2005).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1467-9868.2005.00503.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=WOS:00022749&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F20%2F2021.07.16.21260200.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000227498200007&link_type=ISI) 

20. 20.Hyndman, R. J., Koehler, A. B., Snyder, R. D. & Grose, S. A state space framework for automatic forecasting using exponential smoothing methods. International Journal of Forecasting 18, 439–454. ISSN: 01692070. doi:10.1016/S0169-2070(01)00110-8. [https://linkinghub.elsevier.com/retrieve/pii/S0169207001001108](https://linkinghub.elsevier.com/retrieve/pii/S0169207001001108) (2021) (July 2002).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0169-2070(01)00110-8&link_type=DOI) 

21. 21.De Livera, A. M., Hyndman, R. J. & Snyder, R. D. Forecasting Time Series With Complex Seasonal Patterns Using Exponential Smoothing. Journal of the American Statistical Association 106, 1513–1527. ISSN: 0162-1459, 1537-274X. doi:10.1198/jasa.2011.tm09771. [http://www.tandfonline.com/doi/abs/10.1198/jasa.2011.tm09771](http://www.tandfonline.com/doi/abs/10.1198/jasa.2011.tm09771) (2021) (Dec. 2011).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1198/jasa.2011.tm09771&link_type=DOI) 

22. 22.Taylor, S. J. & Letham, B. Forecasting at scale preprint (PeerJ Preprints, Sept. 2017). doi:10.7287/peerj.preprints.3190v2. [https://peerj.com/preprints/3190v2](https://peerj.com/preprints/3190v2) (2021).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7287/peerj.preprints.3190v2&link_type=DOI) 

23. 23.Hyndman, R. J. & Koehler, A. B. Another look at measures of forecast accuracy. International Journal of Forecasting 22, 679–688. ISSN: 01692070. doi:10.1016/j.ijforecast.2006.03.001. [https://linkinghub.elsevier.com/retrieve/pii/S0169207006000239](https://linkinghub.elsevier.com/retrieve/pii/S0169207006000239) (2021) (Oct. 2006).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ijforecast.2006.03.001&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000241935600006&link_type=ISI) 

24. 24.Cleveland, R., Cleveland, W. S., McRae, J. & Terpenning, I. STL: A Seasonal-Trend Decomposition Procedure Based on Loess. Journal of Official Statistics 6, 3–37 (1990).
    
    

25. 25.Greenwell, B. M., Boehmke, B. C. & McCarthy, A. J. A Simple and Effective Model-Based Variable Importance Measure. 1805.04755 [cs, stat]. arxiv:1805.04755. [http://arxiv.org/abs/1805.04755](http://arxiv.org/abs/1805.04755) (2021) (May 2018).
    
    

26. 26.Gul, M. & Celik, E. An exhaustive review and analysis on applications of statistical forecasting in hospital emergency departments. Health Systems 9, 263–284. ISSN: 2047-6965, 2047-6973. doi:10.1080/20476965.2018.1547348. [https://www.tandfonline.com/doi/full/10.1080/20476965.2018.1547348](https://www.tandfonline.com/doi/full/10.1080/20476965.2018.1547348) (2021) (Oct. 2020).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/20476965.2018.1547348&link_type=DOI) 

27. 27.Vollmer, M. A. et al. A unified machine learning approach to time series forecasting applied to demand at emergency departments. BMC Emergency Medicine 21, 9. ISSN: 1471-227X. doi:10.1186/s12873-020-00395-y. [https://bmcemergmed.biomedcentral.com/articles/10.1186/s12873-020-00395-y](https://bmcemergmed.biomedcentral.com/articles/10.1186/s12873-020-00395-y) (2021) (Dec. 2021).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12873-020-00395-y&link_type=DOI) 

28. 28.Boutsioli, Z. Forecasting the stochastic demand for inpatient care: the case of the Greek national health system. Health Services Management Research 23, 116–120. ISSN: 0951-4848, 1758-1044. doi:10.1258/hsmr.2009.009025. [http://journals.sagepub.com/doi/10.1258/hsmr.2009.009025](http://journals.sagepub.com/doi/10.1258/hsmr.2009.009025) (2021) (Aug. 2010).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1258/hsmr.2009.009025&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20702888&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F20%2F2021.07.16.21260200.atom) 

29. 29.Jones, S. S. et al. Forecasting Daily Patient Volumes in the Emergency Department. Academic Emergency Medicine 15, 159–170. ISSN: 10696563, 15532712. doi:10.1111/j.1553-2712.2007.00032.x. [http://doi.wiley.com/10.1111/j.1553-2712.2007.00032.x](http://doi.wiley.com/10.1111/j.1553-2712.2007.00032.x) (2021) (Feb. 2008).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1553-2712.2007.00032.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18275446&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F07%2F20%2F2021.07.16.21260200.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000253041300007&link_type=ISI) 

30. 30.McCoy, T. H., Pellegrini, A. M. & Perlis, R. H. Assessment of Time-Series Machine Learning Methods for Forecasting Hospital Discharge Volume. JAMA Network Open 1, e184087. ISSN: 2574-3805. doi:10.1001/jamanetworkopen.2018.4087. [http://jamanetworkopen.jamanetwork.com/article.aspx?doi=10.1001/jamanetworkopen.2018.4087](http://jamanetworkopen.jamanetwork.com/article.aspx?doi=10.1001/jamanetworkopen.2018.4087) (2021) (Nov. 2018).
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2018.4087&link_type=DOI)