ABSTRACT
Objective There is a paucity of evidence for the implementation of remote home monitoring for COVID-19 infection. The aims of this study were to: identify the key characteristics of remote home monitoring models for COVID-19 infection, explore the experiences of staff implementing these models, understand the use of data for monitoring progress against outcomes, and document variability in staffing and resource allocation.
Methods This was a multi-site mixed methods study that combined qualitative and quantitative approaches to analyse the implementation and impact of remote home monitoring models during the first wave of the COVID-19 pandemic (March to August 2020). The study combined interviews (n=22) with staff delivering these models across eight sites in England with the collection and analysis of data on staffing models and resource allocation.
Results The models varied in relation to the healthcare settings and mechanisms used for patient triage, monitoring and escalation. Implementation was embedded in existing staff workloads and budgets. Good communication within clinical teams, culturally-appropriate information for patients/carers and the combination of multiple approaches for patient monitoring (app and paper-based) were considered facilitators in implementation. The mean cost per monitored patient varied from £400 to £553, depending on the model.
Conclusions It is necessary to provide the means for evaluating the effectiveness of these models, for example, by establishing comparator data. Future research should also focus on the sustainability of the models and patient experience (considering the extent to which some of the models exacerbate existing inequalities in access to care).
STRENGTHS AND LIMITATIONS OF THE STUDY
The study makes a contribution to existing evidence on remote home monitoring models by exploring the design and implementation of these models for confirmed or suspected COVID-19 cases.
The study was carried out across eight remote home monitoring models implemented in England, capturing variability in the mechanisms used for triage, monitoring and escalation.
Limited evidence was available to assess the effectiveness of the remote home monitoring models.
No comparator data were available for the absence of remote home monitoring.
The study was designed as a rapid evaluation and only captured experiences and processes of implementation in a convenience sample of eight models implemented during the first wave of the pandemic in England.
BACKGROUND
Delays in the presentation of patients with COVID-19 has led to patients arriving in acute care emergency departments with very low oxygen saturations, often without accompanying breathlessness (‘silent hypoxia’)1. As a result, many patients have been admitted with an advanced course of the disease, requiring extended hospital length of stays, invasive treatment, potential admission to Intensive Care Units (ICU) and death2. Remote home monitoring models (sometimes referred to as ‘virtual wards’), using pulse oximetery and other measurements (i.e. temperature), seek to remotely monitor patients considered at high-risk of deterioration at home to: 1) avoid unnecessary hospital admissions and promote appropriate admissions (appropriate care at the appropriate place), and 2) escalate cases of deterioration at an earlier stage to avoid invasive ventilation and ICU admission (by referring patients to emergency services or asking them to visit primary care)3. Remote home monitoring models have been implemented for confirmed or suspected COVID-19 cases in the US, Australia, Canada, Ireland, China, The Netherlands, India and the UK, with some variation in the frequency of patient monitoring, modality (telephone or video calls and use of applications or online portals), staffing, patient criteria and use of pulse oximetry to measure oxygen saturation levels4-8.
In England, a number of remote home monitoring models were set-up during the first wave of the COVID-19 pandemic with the aim outlined above but with a high degree of variability in relation to the mechanisms implemented for patient assessment and triage, monitoring and escalation9. Some models have been led by secondary care while others are mainly based in primary care. Furthermore, some have been designed as pre-hospital models (admitting patients from the community or emergency department) while others have functioned as step-down ‘wards’ (facilitating early discharge from hospital)39,10.
Despite previous research on the use of remote home monitoring models for other conditions and their widespread use during the COVID-19 pandemic, studies on their implementation and impact are rare11. This mixed-methods evaluation of remote home monitoring models in England sought to address this gap by capturing the processes of implementing remote home monitoring models for confirmed or suspected COVID-19 cases, variability in staffing and resource allocation and the experiences of staff designing and delivering these models.
STUDY AIMS AND CONCEPTUAL FRAMEWORK
The aims of this study were to: identify the key characteristics of remote home monitoring models, explore the experiences of staff implementing these models, understand the use of data for monitoring progress against outcomes, and document variability in staffing and resource allocation. We studied models with the following characteristics:
Implemented to monitor confirmed or suspected COVID-19 cases (retrospective as we focused on sites implemented during the first wave of the pandemic)
Focused on monitoring patients prior to hospital admission as well as step-down wards
Delivered from primary and secondary care settings
Include some element of patient recording of oxygen saturation using pulse oximetry.
RESEARCH QUESTIONS
The research questions guiding the study included
What were the aims and main designs of the remote home monitoring models implemented for confirmed or suspected COVID-19 cases?
What were the factors that acted as barriers and facilitators in the implementation of these models during wave 1 of the pandemic?
What were the expected outcomes of these models?
What data were collected by study sites and how did these help them monitor progress against their expected outcomes?
How were the resources allocated (including staffing models) to implement the remote home monitoring models during wave 1 of the pandemic?
What were the lessons learned from implementing remote home monitoring models during wave 1 of the pandemic?
DESIGN
This was a multi-site study that combined qualitative and quantitative approaches to analyse the implementation and impact of remote home monitoring models implemented for confirmed or suspected COVID-19 cases. The study was carried out from July to September 2020.
Rapid qualitative study of first wave
Qualitative fieldwork was based on telephone and online semi-structured interviews with a purposive sample of staff from eight pilot sites implemented during wave 1 of the pandemic and documentary analysis of internal documents developed by these sites. Data collection followed a rapid qualitative research design involving teams of field researchers, participatory approaches, and iterative data collection and analysis12,13. Most of the interviews were carried out by pairs of researchers with different disciplinary backgrounds (i.e. qualitative and quantitative research) to obtain insight into different aspects of the model and their implementation. The interviews were guided by an interview topic guide and focused on capturing the main components of remote home monitoring models, staff experiences of implementing the models during wave 1 of the pandemic, and processes used to implement the models (including factors that acted as barriers and enablers).
We obtained information on the data collected during the first wave (including the data fields, numbers of patients covered and aggregate outcomes, and the extent to which the sites used both bespoke and standard data collection). We gathered information on whether they used other available quantitative evidence to help inform clinical decisions. We used this information to assess the value of the data in helping the sites monitor progress against outcomes, understand the impact of virtual wards, and identify the resources used in their implementation.
Documentary analysis was used to capture changes in design and implementation over time as well as to understand key broad contextual factors such as population served, geography and availability of other services.
Interview sampling
The eight study sites were selected using a combination of purposive and convenience sampling (Table 1). We sought to include sites that functioned as pre-hospital and step-down wards, covered different population sizes and areas of the country and had different combinations of monitoring approaches (app, paper-based with telephone calls, and both). In order to be included in the study, the site had to be fully operational during wave 1 of the pandemic.
Sample of remote home monitoring pilot sites included in the rapid qualitative study
The interviews were carried out with a purposive sample of study participants based on the sampling framework outlined in Table 2. We aimed to carry out interviews with participants who led the design and implementation of the models, staff in charge of monitoring and escalation and staff with knowledge of data collection and use.
Sampling framework for interviews with pilot site participants
Recruitment
Site leads guided the researchers in the identification of potential interview participants. The researchers contacted potential participants via email and sent them a participant information sheet (informing them about the study, the anonymity of the collected data and that their participation would be voluntary). Participants were given 48 hours to review the information and ask questions about the study. If the participant agreed to take part in the study, they were asked to sign the consent form. The researcher then arranged a time to carry out the interview over the phone or online.
Data analysis
Detailed notes from the telephone/online interviews were added to Rapid Assessment Procedure (RAP) sheets that sought to summarise the main findings from each study site. We have described their use in detail in Vindrola-Padros et al.13. Multiple researchers cross-checked the data included in the RAP sheets to ensure consistency in data reduction and synthesis. The interviews were not transcribed verbatim, however, the team held weekly meetings throughout the study to discuss emerging findings The team used the RAP sheets to identify the key findings and look for patterns across the study sites. An iterative approach to data analysis was followed, engaging with published literature, the wider NHS Covid community and case study sites for member validation and additional insight into the study findings.
The number of patients and the impact
Seven out of eight sites provided information on the total number of patients served during the period March – August 2020. Using a data collection template (Appendix 1), the total number of patients was collected separately by activitiy (triaged, monitored, deteriorated and escalated). For one site, limited information on the number of patients was obtained from the interviews. In addition, the number of deaths and the number of patients discharged from the virtual wards were also collected.
Staffing models and resource allocation
We collected data on the staffing models and allocation of resources. This included data on the number of staff, their function/ seniority and the number of hours worked. Data on all other resources used in setting up the pilots and running them were also collected. These included medical equipment used, information materials, tools for flagging deterioration, etc. (see Appendix 1). The sites also reported the associated costs to these resources. Six out of eight sites provided the requested information.
Data analysis
The information from the pre-hospital and step-down models was analysed separately. The set-up costs were reported as a mean cost per site (as the running period may be longer than the one observed in this study). For the running costs, the mean costs per patient were calculated for the pilot period (March-August 2020).
RESULTS
The design of remote home monitoring models
The main aim of these models was to monitor patients considered high-risk who could be safely managed at home to: 1) avoid unnecessary hospital admissions (appropriate care at the appropriate place), and 2) escalate cases of deterioration at an earlier stage. In most cases, staff drew from experiences of implementing previous remote home monitoring or ambulatory care pathways and used staff familiar with these pathways to deliver the home monitoring.
The steps or main components of these models included (see also Figure 1)
Assessment of patient for suitability for remote home monitoring. The patient was triaged through emergency telephone numbers, GP practice, (or ED for those pilots in secondary care). Some of these models functioned as ‘step-down’ wards in the sense that they were used to monitor patients who had been admitted to hospital but were thought to be fit enough to complete their recovery at home.
The patient was provided with a pulse oximeter, patient information (including escalation warning signs and what to do) and a mechanism for recording observations regularly (app or paper diary) such as changes in symptoms, pulse, heart rate, temperature and blood oxygen levels. In some cases, patients were risk-stratified and this informed the frequency of the monitoring.
The patient received regular monitoring calls from staff (either primary or secondary care depending on the site) capturing changes in symptoms and trends in oxygen saturation. The modality and frequency of surveillance was based on clinician discretion and calls were used to identify cases of deterioration and inform the patient of next steps. The sites using apps for patient monitoring had an additional safety mechanism, where the app alerted the patient when recorded data indicated potential deterioration and the patient was instructed about next steps (emergency telephone numbers, GP call, etc.). Alerts could have also triggered action by the clinical team monitoring patient information on electronic dashboards, mainly through a phone call to the patient to indicate that they would need to visit a local ED or dial the emergency phone line.
Patients were expected to be discharged from remote home monitoring around 14 days (when recovery was expected), but this varied by study site. In some cases, patients were followed-up in person at their local GP practice. Some study sites had linked data to monitor outcomes in patients admitted to hospital.
Patient experience and engagement
Seven of the eight study sites documented patient experience through surveys or questionnaires. In general, patient experience was described as positive. Staff described high levels of patient engagement and reports by patients that the service provided reassurance. It is also important to note that staff also reported some cases of increased patient anxiety and the reduction in patient engagement during later follow-up calls or at later stages of the first wave of the pandemic.
Data and evidence
The study sites collected combinations of demographics, clinical readings, patient experience and outcomes data. Common outcomes collected included hospital and ICU admissions or readmissions, ED attendances, mortality rates and patient satisfaction measures. The need to act quickly at the start of the pandemic meant that there was little time to carefully plan data collection. Study sites aspired to share data between and within primary, secondary and other care sectors, but had made limited progress. Data quality was reported in some sites to be good, while others acknowledged limitations, especially early on. Data collection outside the apps could be cumbersome, and study sites found it challenging to integrate data from apps into their existing patient administration systems. There was relatively little external evidence available on the effectiveness of different approaches that could inform the set-up of the models.
Use of data
Some study sites reported that analysis of their own data had helped to inform improvements to their service, and most were monitoring key outcomes over time. One site with a small number of patients acknowledged the difficulty of measuring the impact of their service, and queried whether combining data across different study sites would be possible. No study site had been able to identify an appropriate group to use as a comparator at the time of the study.
More sophisticated analyses of the data were reported to have started with partner health and academic organisations, for example, two sites were collaborating on the development of a prediction model linked to oximetry readings. Further measures that sites would like to have collected included data on the longer-term effects of COVID-19 on patients, and data on the mental health impacts on both staff and patients.
Patient numbers and impact
The information on the number of patients monitored for the period March-August 2020 for seven out of eight sites (for both the pre-hospital and the step-down models) is presented in Table 3. The total number of escalated patients was 10% for the pre-hospital and 12.2% for the step-down model, from which the majority of patients was seen in ED (76.7% and 91.8% respectively) and/or admitted to hospital (52.7% and 74.5%). The number of escalated patients admitted to ICU was 2.0% for the pre-hospital and 8.5% for the step-down model. The reported mortality rates (1.1% and 0.9% respectively) were lower than the Case Fatality Rate (CFR) of 6.57 reported in the national data during the first peak14.
The number of patients
Staffing models and costs
Data on the staff involved in setting-up and running the models did not show clear patterns in terms of specialisation or seniority of the staff involved. The total number of the staff involved in setting up and running the models varied by site. Staff involved were a mix of consultants, ED staff, GP partners, nurses, ANPs, and medical students (see Appendix 2 for more information on the staff’s level of seniority and the number of hours spent).
Table 4 gives detailed information on the resources used during the setting-up and running periods for both pre-hospital and step-down models. As expected, the set-up period was more resource intensive, while the running was more staff-time intensive. The running costs depended on the time the pilot site was operating and the number of patients seen. The mean costs per monitored patient were higher in the pre-hospital (£553 per patient) than in the step-down model (£400/patient).
Costs of pre-hospital and step-down models
Implementation facilitators
Implementation was facilitated by the active role played by dedicated clinical leaders in establishing the remote home monitoring models. Significant support and ‘buy in’ from senior management within acute trusts and across Clinical Comissioning Groups (CCGs) to set up the models was documented across all sites. Acute hospitals that had previous pathways in place (i.e. ambulatory care) or digital protocols that could be repurposed by IT teams were able to set-up these models at a quicker pace. In both primary and secondary care-led models, participants indicated that monitoring could be delivered by nurses with minimal senior oversight, maintaining clear communication with delivery teams. Good communication between members of the clinical team was identified as a key facilitator. It is important to note that during the first wave of the pandemic, staff were available to play a role in the delivery of care in these models due to the cancellation of elective care and other activities in the NHS. Volunteers were also used. Participants expressed concern that these staff members would not be available during future surges in patient cases.
In general, patients experienced positive engagement with the remote home monitoring models. Paper and video patient information (as well as using digital platforms) was very useful to explain the concept of the remote home monitoring models and how to take measurements using pulse oximeters.
Implementation barriers
Early on, referral criteria and processes were unclear, which led to patients being referred to these models who might have have been ineligible in other circumstances. In part, this was caused by evolving criteria for patient referrals. Staff found it difficult to carry out non-verbal assessments using telephone and video consultation alone. Some patient groups were more difficult to monitor remotely (e.g. homeless community) and staff reflected that monitoring using an app only model might not be suitable for all populations, as this approach could exclude patients with low levels of health and technology literacy. The availability of culturally appropriate patient information in different community languages was identified as a key component of patient engagement, but not all study sites were able to develop these materials.
Lack of administrative/project management support and resources meant that essential equipment such as pulse oximeters could not be obtained quickly. Staff also found it challenging to deliver a seven day service due to workforce availability. There was a lack of published data to support the design of the remote monitoring models and study sites found it challenging and time consuming to collect the desired data, even when using commercially available apps. The integration of service data with existing patient administration systems was generally poor, and it was not feasible to arrange data sharing between and within sectors in the time available. Additionally, there was no link between NHS Test and Trace systems and the study sites’ referral processes.
DISCUSSION
The monitoring of patients remotely was perceived by staff as a safe way to ensure patients received the appropriate care at the right place. The mortality rate (1%) appeared low, especially when compared to other COVID-19-related mortality rates; but caution needs to be taken when drawing comparisons as populations, pre-existing conditions and levels of severity are likely to be different. Furthermore, these models were not able to establish control groups to compare effectiveness. This is consistent with the published evidence on the use of remote home monitoring models for COVID-19 cases in other countries11.
Patient and carer training were identified as the key to the success of these models and some concerns were expressed in relation to models that only relied on digital solutions such as apps for the monitoring of patients and the identification of cases of deterioration. Technological barriers have been reported in studies of remote home monitoring for other conditions and authors have highlighted the impact of these barriers on patient engagement and outcomes15,16.
The development of inclusive models was discussed frequently, and, to some extent, this was reliant on the availability of patient information that was culturally appropriate and in different languages. Some published examples of the use of pulse oximetry in remote home monitoring models have indicated that good patient training is associated with more accurate readings and more responsive care5. Personalised support might be required to avoid patient anxiety and reach those who may be difficult to monitor remotely. Remote home monitoring models in the US and Canada, for instance, also screened patients for mental health and social care needs, providing a more holistic model of patient care4,17.
Our study also found concerns regarding the sustainability of services as the models we analysed were implemented during the first wave of the pandemic, when some members of staff were released from clinical responsibilities and could be redeployed to monitor patients remotely. Furthermore, additional sources of funding were made available at this time and staff were allowed to use discretionary funds through fast-approval processes established in both primary and secondary care. The sustainability of these models during subsequent surges in patient cases and for other conditions will require more stable flows of funding as well as clinical and administrative management support. The detailed information on the use of resources was limited and more resources could have been used for set-up and running of the services, but not reported. It was also difficult to conclude if the differences in resource use between the sites were a result of barriers/availability of resources or reflected the specific patterns of use in these different contexts.
The impact of remote home monitoring on patient outcomes and their cost-effectiveness should also be assessed through the use of more standardised data and appropriate comparators. Future research will also need to explore the benefits and limitations of models led by primary versus secondary care to determine if primary care is able to deliver more integrated care for patients and reduce the demand on hospital services. The analysis of patients’ and carers’ experiences with these models should be a core component of future studies, and may shed further light on how best to engage and support patients to maximise the effectiveness of remote home monitoring. As the healthcare landscape continues to adapt to the challenges generated by COVID-19 and the NHS enters different seasonal peaks, future studies will need to analyse the transformation of these models over time, beyond the pressures and resources made available during the first wave of the pandemic and considering changes implemented in the risk stratification of patients and Test and Trace programmes.
Data Availability
All data relevant to the study are included in the article or uploaded as supplementary information.
FUNDING
The study was funded by the National Institute for Health Research-NIHR (Health Services and Delivery Research, 16/138/17 – Rapid Service Evaluation Research Team; or The Birmingham, RAND and Cambridge Evaluation (BRACE) Centre Team (HSDR16/138/31) and NJF is an NIHR Senior Investigator; no authors have financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no authors have other relationships or activities that could appear to have influenced the submitted work. The views and opinions expressed therein are those of the authors and do not necessarily reflect those of the HS&DR, NIHR, NHS or the Department of Health and Social Care.
COMPETING INTERESTS
The authors do not have any competing interests to declare.
PATIENT AND PUBLIC INVOLVEMENT STAGES
Neither patients nor the public were involved in the design, conduct, reporting or dissemination plans of our research.
ETHICAL APPROVAL STATEMENT
The study was reviewed and classified as a service evaluation by the HRA decision tool and the UCL/UCLH Joint Research Office. It was also reviewed by the University of Birmingham Research Ethics Committee (REC): ERN_13-1085AP37.
CONTRIBUTORSHIP STATEMENT
NJF, CSJ, TG, KS, MS, SMT, SM and CVP contributed to the design of the study. MS, KS, CSJ, TG, SMT, NJF and CVP participated in data collection for the study. NJF, JE, and SM reviewed and provided feedback on the manuscript. All authors approved the final version of the manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
DATA SHARING AGREEMENT
All data relevant to the study are included in the article or uploaded as supplementary information.
ACKNOWLEDGEMENTS
We would like to thank the support provided by Matt Inada-Kim, Allison Streetly and Karen Kirkham throughout the duration of the study. We would also like to thank all of the staff members who gave of their time to participate in the study.
Footnotes
c.vindrola{at}ucl.ac.uk, m.s.sidhu{at}bham.ac.uk, theo.georghiou{at}nuffieldtrust.org.uk, chris.sherlaw-johnson{at}nuffieldtrust.org.uk, k.e.singh{at}bham.ac.uk, s.tomini{at}ucl.ac.uk, j.l.ellins{at}bham.ac.uk, sm2428{at}medschl.cam.ac.uk, n.fulop{at}ucl.ac.uk